2024-03-14T12:02:19.996 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-14T12:02:19.996 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin} --verbose --name jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi --archive /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019 --owner scheduled_jcollin@teuthology -- /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/orig.config.yaml 2024-03-14T12:02:20.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T12:02:20.119 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019 branch: main description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin} email: jcollin@redhat.com first_in_suite: false job_id: '7600019' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: lockdep: true ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - missing required features sha1: 71e6beb6de9ad34b7d44be0ccdf38721ae92202d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 71e6beb6de9ad34b7d44be0ccdf38721ae92202d kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: origin-jcollin-testing sha1: ab5e0cba092761c4e4548739d8748ec40a6ffd68 owner: scheduled_jcollin@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8468 sha1: 71e6beb6de9ad34b7d44be0ccdf38721ae92202d sleep_before_teardown: 0 suite: fs:functional suite_branch: origin-jcollin-testing suite_path: /home/teuthworker/src/github.com_joscollin_ceph_ab5e0cba092761c4e4548739d8748ec40a6ffd68/qa suite_relpath: qa suite_repo: https://github.com/joscollin/ceph.git suite_sha1: ab5e0cba092761c4e4548739d8748ec40a6ffd68 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtnm40UtVIa2GFsWgd9HtRYh6qFTSDq+2CKW1kvAiwvrH0JAREZrFSK3nki8uLh601My7RmfbPU0NlKowuAyZo= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFD15MB9iYeb6MrVg/H9dAD9sptnJFZ1tfewH5VGxfYwEJX0gSLWLl1rzNrxUSikF25Rg8xYpZjDfqYrDMW2F5Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_admin.TestLabeledPerfCounters teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-14_11:45:06 tube: smithi user: jcollin verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-14T12:02:20.119 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_joscollin_ceph_ab5e0cba092761c4e4548739d8748ec40a6ffd68/qa; will attempt to use it 2024-03-14T12:02:20.120 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_joscollin_ceph_ab5e0cba092761c4e4548739d8748ec40a6ffd68/qa/tasks 2024-03-14T12:02:20.120 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-14T12:02:20.122 INFO:teuthology.task.internal:Checking packages... 2024-03-14T12:02:20.145 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '71e6beb6de9ad34b7d44be0ccdf38721ae92202d' 2024-03-14T12:02:20.145 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-14T12:02:20.145 INFO:teuthology.packaging:ref: None 2024-03-14T12:02:20.145 INFO:teuthology.packaging:tag: None 2024-03-14T12:02:20.146 INFO:teuthology.packaging:branch: main 2024-03-14T12:02:20.146 INFO:teuthology.packaging:sha1: 71e6beb6de9ad34b7d44be0ccdf38721ae92202d 2024-03-14T12:02:20.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-03-14T12:02:20.386 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2052.g6185d084 2024-03-14T12:02:20.388 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-14T12:02:20.395 INFO:teuthology.task.internal:no buildpackages task found 2024-03-14T12:02:20.395 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-14T12:02:20.482 INFO:teuthology.task.internal:Saving configuration 2024-03-14T12:02:20.499 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-14T12:02:20.505 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T12:02:20.528 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 11:57:05.778361', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtnm40UtVIa2GFsWgd9HtRYh6qFTSDq+2CKW1kvAiwvrH0JAREZrFSK3nki8uLh601My7RmfbPU0NlKowuAyZo='} 2024-03-14T12:02:20.549 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 11:57:05.776750', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFD15MB9iYeb6MrVg/H9dAD9sptnJFZ1tfewH5VGxfYwEJX0gSLWLl1rzNrxUSikF25Rg8xYpZjDfqYrDMW2F5Y='} 2024-03-14T12:02:20.549 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-14T12:02:20.555 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-14T12:02:20.555 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-14T12:02:20.556 INFO:teuthology.run_tasks:Running task console_log... 2024-03-14T12:02:20.632 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3c55c710d0>, signals=[15]) 2024-03-14T12:02:20.633 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-14T12:02:20.638 INFO:teuthology.task.internal:Opening connections... 2024-03-14T12:02:20.638 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:02:20.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:02:20.732 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:02:20.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:02:20.820 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-14T12:02:20.827 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-03-14T12:02:20.845 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-03-14T12:02:20.846 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-03-14T12:02:20.902 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T12:02:20.903 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-14T12:02:20.904 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-14T12:02:20.904 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-03-14T12:02:20.995 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-03-14T12:02:21.014 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-03-14T12:02:21.014 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-03-14T12:02:21.070 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-14T12:02:21.071 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-14T12:02:21.072 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-03-14T12:02:21.101 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-14T12:02:21.112 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-14T12:02:21.123 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-14T12:02:21.123 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-03-14T12:02:21.126 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-03-14T12:02:21.145 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-14T12:02:21.151 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-14T12:02:21.151 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-03-14T12:02:21.184 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-03-14T12:02:21.225 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-14T12:02:21.278 INFO:teuthology.run_tasks:Running task kernel... 2024-03-14T12:02:21.291 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-14T12:02:21.291 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-14T12:02:21.291 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-14T12:02:21.292 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-14T12:02:21.292 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-14T12:02:21.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:02:21.308 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-03-14T12:02:21.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:02:21.309 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-03-14T12:02:21.364 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-361.el9.x86_64 2024-03-14T12:02:21.365 INFO:teuthology.task.kernel:Running kernel on smithi082: 5.14.0-361.el9.x86_64 2024-03-14T12:02:21.365 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2024-03-14T12:02:21.366 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-361.el9.x86_64 2024-03-14T12:02:21.366 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-361.el9.x86_64 2024-03-14T12:02:21.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-03-14T12:02:23.495 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-14T12:02:23.698 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 9.3 MB/s | 6.7 MB 00:00 2024-03-14T12:02:24.962 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-03-14T12:02:25.299 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-14T12:02:26.294 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-03-14T12:02:27.673 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:01 2024-03-14T12:02:29.303 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-14T12:02:29.976 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-14T12:02:34.591 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.1 MB 00:01 2024-03-14T12:02:36.424 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2024-03-14T12:02:38.126 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-14T12:02:40.911 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 772 kB/s | 6.1 MB 00:08 2024-03-14T12:02:42.504 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2024-03-14T12:02:43.042 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-14T12:02:45.198 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-14T12:02:45.198 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-14T12:02:45.342 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:02:45.350 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-14T12:02:45.351 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-14T12:02:45.351 INFO:teuthology.orchestra.run.smithi082.stdout:Install 4 Packages 2024-03-14T12:02:45.351 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:02:45.351 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 100 M 2024-03-14T12:02:45.351 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 123 M 2024-03-14T12:02:45.351 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-03-14T12:02:46.126 INFO:teuthology.orchestra.run.smithi082.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 9.0 MB/s | 6.0 MB 00:00 2024-03-14T12:02:46.951 INFO:teuthology.orchestra.run.smithi082.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 14 MB/s | 21 MB 00:01 2024-03-14T12:02:47.893 INFO:teuthology.orchestra.run.smithi082.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-03-14T12:02:48.418 INFO:teuthology.orchestra.run.smithi082.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 15 MB/s | 33 MB 00:02 2024-03-14T12:02:48.418 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-03-14T12:02:48.418 INFO:teuthology.orchestra.run.smithi082.stdout:Total 32 MB/s | 100 MB 00:03 2024-03-14T12:02:49.304 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-03-14T12:02:49.675 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-03-14T12:02:49.675 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-03-14T12:02:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-03-14T12:02:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-03-14T12:02:53.704 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-03-14T12:02:54.339 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-14T12:02:54.460 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-14T12:02:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-14T12:02:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-14T12:03:02.296 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-14T12:03:02.414 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:04.647 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 958 kB/s | 21 MB 00:22 2024-03-14T12:03:07.122 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:09.569 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-14T12:03:11.734 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-14T12:03:11.734 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-14T12:03:11.821 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-03-14T12:03:11.826 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-03-14T12:03:11.827 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-03-14T12:03:11.827 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:03:11.827 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 100 M 2024-03-14T12:03:11.827 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 123 M 2024-03-14T12:03:11.827 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-03-14T12:03:12.370 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 17 MB/s | 6.0 MB 00:00 2024-03-14T12:03:12.645 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 33 MB/s | 21 MB 00:00 2024-03-14T12:03:13.587 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 25 MB/s | 39 MB 00:01 2024-03-14T12:03:13.879 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 22 MB/s | 33 MB 00:01 2024-03-14T12:03:13.879 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-03-14T12:03:13.880 INFO:teuthology.orchestra.run.smithi164.stdout:Total 48 MB/s | 100 MB 00:02 2024-03-14T12:03:14.770 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-03-14T12:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-03-14T12:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-03-14T12:03:16.792 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-03-14T12:03:16.792 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-03-14T12:03:19.513 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-03-14T12:03:20.152 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-14T12:03:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-14T12:03:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-14T12:03:23.274 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-14T12:03:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-14T12:03:28.158 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:32.921 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:36.376 INFO:teuthology.orchestra.run.smithi082.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-14T12:03:36.376 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:03:36.460 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:36.947 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:36.947 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-14T12:03:36.948 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-14T12:03:36.948 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:03:37.343 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-14T12:03:37.858 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-03-14T12:03:37.996 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-14T12:03:37.996 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-14T12:03:37.996 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-14T12:03:37.996 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-14T12:03:37.997 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-14T12:03:37.997 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-14T12:03:37.997 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=None, version=distro) 2024-03-14T12:03:37.997 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-03-14T12:03:38.117 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-14T12:03:38.117 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-14T12:03:38.117 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -qi grub2-tools 2024-03-14T12:03:38.154 INFO:teuthology.orchestra.run.smithi082.stdout:Name : grub2-tools 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Epoch : 1 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Version : 2.06 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Release : 68.el9 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: x86_64 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Group : Unspecified 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Size : 8264254 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:License : GPLv3+ 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Packager : builder@centos.org 2024-03-14T12:03:38.155 INFO:teuthology.orchestra.run.smithi082.stdout:Vendor : CentOS 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:Summary : Support tools for GRUB. 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:Description : 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:hardware devices. 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:03:38.156 INFO:teuthology.orchestra.run.smithi082.stdout:This subpackage provides tools for support of all platforms. 2024-03-14T12:03:38.157 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-14T12:03:38.157 INFO:teuthology.task.kernel:Updating grub on smithi082 to boot 5.14.0-427.el9.x86_64 2024-03-14T12:03:38.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-14T12:03:38.784 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-03-14T12:03:40.257 INFO:teuthology.orchestra.run.smithi082.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-14T12:03:40.290 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-03-14T12:03:40.292 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-03-14T12:03:40.307 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.Ha6QKb37Bz 2024-03-14T12:03:40.308 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ha6QKb37Bz 2024-03-14T12:03:40.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.Ha6QKb37Bz 2024-03-14T12:03:40.514 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.Ha6QKb37Bz is 6KB 2024-03-14T12:03:40.563 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.Ha6QKb37Bz 2024-03-14T12:03:40.577 DEBUG:teuthology.orchestra.run.smithi082:> sudo /bin/ls /boot/loader/entries || true 2024-03-14T12:03:40.643 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-14T12:03:40.644 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-14T12:03:40.644 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-14T12:03:40.644 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-14T12:03:40.645 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-14T12:03:40.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2024-03-14T12:04:02.306 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:04:02.306 INFO:teuthology.orchestra.run.smithi164.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-14T12:04:02.306 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:04:02.390 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:04:02.803 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:04:02.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-14T12:04:02.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-14T12:04:02.803 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-14T12:04:03.198 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-14T12:04:03.198 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:04:03.198 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-03-14T12:04:03.199 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-14T12:04:03.199 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-14T12:04:03.199 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-14T12:04:03.199 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-14T12:04:03.199 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:04:03.199 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-03-14T12:04:03.749 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-03-14T12:04:03.891 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-14T12:04:03.891 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-14T12:04:03.891 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-14T12:04:03.891 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-14T12:04:03.891 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-14T12:04:03.891 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-14T12:04:03.891 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-03-14T12:04:03.892 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-03-14T12:04:04.012 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-14T12:04:04.013 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-14T12:04:04.013 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 68.el9 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-03-14T12:04:04.048 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8264254 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:04:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-03-14T12:04:04.050 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-14T12:04:04.050 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-427.el9.x86_64 2024-03-14T12:04:04.050 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-14T12:04:04.675 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-03-14T12:04:06.066 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-14T12:04:06.096 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-03-14T12:04:06.099 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-03-14T12:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.xNjGkxG4RJ 2024-03-14T12:04:06.114 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xNjGkxG4RJ 2024-03-14T12:04:06.183 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.xNjGkxG4RJ 2024-03-14T12:04:06.328 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.xNjGkxG4RJ is 6KB 2024-03-14T12:04:06.376 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.xNjGkxG4RJ 2024-03-14T12:04:06.391 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-03-14T12:04:06.455 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-14T12:04:06.455 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-14T12:04:06.455 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-14T12:04:06.455 INFO:teuthology.orchestra.run.smithi164.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-14T12:04:06.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-14T12:04:06.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-03-14T12:04:10.814 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-14T12:04:10.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-14T12:04:10.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:04:36.638 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-14T12:04:36.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-14T12:04:36.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:05:10.851 DEBUG:teuthology.orchestra.remote:timed out 2024-03-14T12:05:11.330 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-03-14T12:05:19.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-14T12:05:19.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:05:20.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-14T12:05:20.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:05:21.280 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-03-14T12:05:21.776 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-14T12:05:21.776 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-14T12:05:21.777 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-03-14T12:05:21.791 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-427.el9.x86_64 2024-03-14T12:05:21.791 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-14T12:05:21.791 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-14T12:05:21.791 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-14T12:05:22.793 DEBUG:teuthology.parallel:result is None 2024-03-14T12:06:20.334 DEBUG:teuthology.orchestra.remote:timed out 2024-03-14T12:06:32.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-14T12:06:32.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:06:32.916 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-03-14T12:06:33.319 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-14T12:06:33.319 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-427.el9.x86_64"... 2024-03-14T12:06:33.320 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-03-14T12:06:33.335 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-427.el9.x86_64 2024-03-14T12:06:33.336 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-14T12:06:33.336 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-14T12:06:33.336 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-14T12:06:34.337 DEBUG:teuthology.parallel:result is None 2024-03-14T12:06:34.338 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-14T12:06:34.345 INFO:teuthology.task.internal:Creating test directory... 2024-03-14T12:06:34.345 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-14T12:06:34.349 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-14T12:06:34.367 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-14T12:06:34.374 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-14T12:06:34.381 INFO:teuthology.task.internal:Creating archive directory... 2024-03-14T12:06:34.381 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-14T12:06:34.409 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-14T12:06:34.511 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-14T12:06:34.517 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-14T12:06:34.518 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-14T12:06:34.520 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-14T12:06:34.596 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T12:06:34.624 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T12:06:34.625 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T12:06:34.658 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T12:06:34.660 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-14T12:06:34.667 INFO:teuthology.task.internal:Configuring sudo... 2024-03-14T12:06:34.668 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-14T12:06:34.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-14T12:06:34.700 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-14T12:06:34.708 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-14T12:06:34.708 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-14T12:06:34.741 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-14T12:06:34.757 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T12:06:34.842 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T12:06:34.896 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T12:06:34.980 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T12:06:35.023 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:06:35.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-14T12:06:35.092 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T12:06:35.136 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T12:06:35.196 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T12:06:35.281 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T12:06:35.325 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:06:35.325 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-14T12:06:35.395 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-03-14T12:06:35.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-03-14T12:06:35.447 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T12:06:35.473 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T12:06:35.989 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-14T12:06:35.998 INFO:teuthology.task.internal:Starting timer... 2024-03-14T12:06:35.998 INFO:teuthology.run_tasks:Running task pcp... 2024-03-14T12:06:36.009 INFO:teuthology.run_tasks:Running task selinux... 2024-03-14T12:06:36.017 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-14T12:06:36.017 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-03-14T12:06:36.091 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-03-14T12:06:36.093 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-03-14T12:06:36.162 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-03-14T12:06:36.164 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-14T12:06:36.164 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-03-14T12:06:36.190 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-03-14T12:06:36.190 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-03-14T12:06:36.214 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-03-14T12:06:36.214 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-03-14T12:06:36.214 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-14T12:06:36.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:06:36.260 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-14T12:06:36.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:06:36.289 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-14T12:06:36.289 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-03-14T12:06:36.350 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-03-14T12:06:36.391 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-14T12:06:36.400 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-14T12:06:36.416 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-14T12:06:36.418 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-14T12:10:46.750 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-03-14T12:10:46.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-14T12:10:46.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:10:46.843 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-03-14T12:10:46.940 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-14T12:10:46.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-14T12:10:46.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T12:10:47.028 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-03-14T12:10:47.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-14T12:10:47.111 INFO:teuthology.run_tasks:Running task clock... 2024-03-14T12:10:47.121 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-14T12:10:47.121 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-14T12:10:47.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T12:10:47.124 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-14T12:10:47.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T12:10:47.152 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-14T12:10:47.169 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-14T12:10:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-14T12:10:47.197 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-14T12:10:47.198 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-03-14T12:10:47.211 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-03-14T12:10:47.226 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-03-14T12:10:47.228 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-14T12:10:47.241 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-03-14T12:10:47.244 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-14T12:10:47.257 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-14T12:10:47.274 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-14T12:10:47.303 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-03-14T12:10:47.306 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T12:10:47.306 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-03-14T12:10:47.306 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.307 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.307 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.307 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.325 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-03-14T12:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T12:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-03-14T12:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:10:47.329 INFO:teuthology.run_tasks:Running task install... 2024-03-14T12:10:47.338 DEBUG:teuthology.task.install:project ceph 2024-03-14T12:10:47.338 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d'}} 2024-03-14T12:10:47.338 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d'} 2024-03-14T12:10:47.339 INFO:teuthology.task.install:Using flavor: default 2024-03-14T12:10:47.344 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-14T12:10:47.344 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-14T12:10:47.345 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d', 'tag': None, 'wait_for_package': False} 2024-03-14T12:10:47.345 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=71e6beb6de9ad34b7d44be0ccdf38721ae92202d 2024-03-14T12:10:47.347 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d', 'tag': None, 'wait_for_package': False} 2024-03-14T12:10:47.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=71e6beb6de9ad34b7d44be0ccdf38721ae92202d 2024-03-14T12:10:47.489 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/ 2024-03-14T12:10:47.489 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2023.g71e6beb6 2024-03-14T12:10:47.742 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-14T12:10:47.742 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:10:47.742 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-14T12:10:47.770 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-14T12:10:47.771 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-14T12:10:47.792 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/ 2024-03-14T12:10:47.792 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2023.g71e6beb6 2024-03-14T12:10:47.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-14T12:10:47.902 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-14T12:10:47.902 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:10:47.902 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-14T12:10:47.919 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-14T12:10:47.931 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-14T12:10:47.931 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/71e6beb6de9ad34b7d44be0ccdf38721ae92202d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-14T12:10:47.986 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-03-14T12:10:47.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-03-14T12:10:47.999 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-14T12:10:48.079 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-14T12:10:48.146 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-03-14T12:10:48.147 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-03-14T12:10:48.223 INFO:teuthology.orchestra.run.smithi164.stdout:74 files removed 2024-03-14T12:10:48.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-14T12:10:48.411 INFO:teuthology.orchestra.run.smithi082.stdout:74 files removed 2024-03-14T12:10:48.440 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-14T12:10:48.987 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 195 kB/s | 82 kB 00:00 2024-03-14T12:10:49.168 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 198 kB/s | 82 kB 00:00 2024-03-14T12:10:49.329 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 34 kB/s | 10 kB 00:00 2024-03-14T12:10:49.601 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-03-14T12:10:49.679 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-03-14T12:10:49.994 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-03-14T12:10:50.688 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-14T12:10:51.057 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - AppStream 5.0 MB/s | 6.7 MB 00:01 2024-03-14T12:10:52.405 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-03-14T12:10:52.574 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2024-03-14T12:10:54.656 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-03-14T12:10:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-14T12:11:00.944 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.0 MB 00:08 2024-03-14T12:11:01.174 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.1 MB 00:01 2024-03-14T12:11:02.667 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 15 kB 00:00 2024-03-14T12:11:05.038 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-03-14T12:11:09.876 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-14T12:11:12.446 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T12:11:12.447 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T12:11:12.459 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-14T12:11:12.460 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-14T12:11:12.461 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-14T12:11:12.462 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-14T12:11:12.464 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-14T12:11:12.466 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-14T12:11:12.467 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-14T12:11:12.548 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-03-14T12:11:12.569 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 6.6 k 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 5.4 M 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 888 k 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 145 k 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 1.5 M 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 139 k 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 1.8 M 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 7.4 M 2024-03-14T12:11:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 49 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 9.9 M 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 48 M 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 264 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 748 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 220 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 27 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 18 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 31 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 745 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 127 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-14T12:11:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 162 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 320 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 298 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 99 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 85 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 3.1 M 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 170 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-14T12:11:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 3.4 M 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 3.1 M 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-14T12:11:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 20 M 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 34 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 2.1 M 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 248 k 2024-03-14T12:11:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 4.7 M 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 17 M 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 16 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 25 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 163 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-14T12:11:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 501 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 5.2 M 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-14T12:11:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 45 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 131 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-14T12:11:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-14T12:11:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-14T12:11:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-14T12:11:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:11:12.581 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-03-14T12:11:12.582 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================ 2024-03-14T12:11:12.582 INFO:teuthology.orchestra.run.smithi164.stdout:Install 150 Packages 2024-03-14T12:11:12.582 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 24 Packages 2024-03-14T12:11:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:11:12.582 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 224 M 2024-03-14T12:11:12.582 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-03-14T12:11:13.994 INFO:teuthology.orchestra.run.smithi164.stdout:(1/174): ceph-19.0.0-2023.g71e6beb6.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-03-14T12:11:14.978 INFO:teuthology.orchestra.run.smithi164.stdout:(2/174): ceph-common-19.0.0-2023.g71e6beb6.el9. 17 MB/s | 20 MB 00:01 2024-03-14T12:11:15.011 INFO:teuthology.orchestra.run.smithi164.stdout:(3/174): ceph-fuse-19.0.0-2023.g71e6beb6.el9.x8 873 kB/s | 888 kB 00:01 2024-03-14T12:11:15.036 INFO:teuthology.orchestra.run.smithi164.stdout:(4/174): ceph-immutable-object-cache-19.0.0-202 2.4 MB/s | 145 kB 00:00 2024-03-14T12:11:15.145 INFO:teuthology.orchestra.run.smithi164.stdout:(5/174): ceph-mgr-19.0.0-2023.g71e6beb6.el9.x86 14 MB/s | 1.5 MB 00:00 2024-03-14T12:11:15.420 INFO:teuthology.orchestra.run.smithi164.stdout:(6/174): ceph-mon-19.0.0-2023.g71e6beb6.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-14T12:11:16.146 INFO:teuthology.orchestra.run.smithi164.stdout:(7/174): ceph-mds-19.0.0-2023.g71e6beb6.el9.x86 1.9 MB/s | 2.1 MB 00:01 2024-03-14T12:11:16.304 INFO:teuthology.orchestra.run.smithi164.stdout:(8/174): ceph-osd-19.0.0-2023.g71e6beb6.el9.x86 19 MB/s | 17 MB 00:00 2024-03-14T12:11:16.379 INFO:teuthology.orchestra.run.smithi164.stdout:(9/174): ceph-base-19.0.0-2023.g71e6beb6.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-03-14T12:11:16.405 INFO:teuthology.orchestra.run.smithi164.stdout:(10/174): ceph-selinux-19.0.0-2023.g71e6beb6.el 251 kB/s | 25 kB 00:00 2024-03-14T12:11:16.455 INFO:teuthology.orchestra.run.smithi164.stdout:(11/174): cephfs-mirror-19.0.0-2023.g71e6beb6.e 4.3 MB/s | 220 kB 00:00 2024-03-14T12:11:16.505 INFO:teuthology.orchestra.run.smithi164.stdout:(12/174): cephfs-shell-19.0.0-2023.g71e6beb6.el 541 kB/s | 27 kB 00:00 2024-03-14T12:11:16.555 INFO:teuthology.orchestra.run.smithi164.stdout:(13/174): libcephfs-devel-19.0.0-2023.g71e6beb6 626 kB/s | 31 kB 00:00 2024-03-14T12:11:16.639 INFO:teuthology.orchestra.run.smithi164.stdout:(14/174): libcephfs2-19.0.0-2023.g71e6beb6.el9. 8.7 MB/s | 745 kB 00:00 2024-03-14T12:11:16.706 INFO:teuthology.orchestra.run.smithi164.stdout:(15/174): libcephsqlite-19.0.0-2023.g71e6beb6.e 2.4 MB/s | 163 kB 00:00 2024-03-14T12:11:16.756 INFO:teuthology.orchestra.run.smithi164.stdout:(16/174): librados-devel-19.0.0-2023.g71e6beb6. 2.5 MB/s | 127 kB 00:00 2024-03-14T12:11:16.840 INFO:teuthology.orchestra.run.smithi164.stdout:(17/174): libradosstriper1-19.0.0-2023.g71e6beb 5.9 MB/s | 501 kB 00:00 2024-03-14T12:11:17.033 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:15 2024-03-14T12:11:17.207 INFO:teuthology.orchestra.run.smithi164.stdout:(18/174): librgw2-19.0.0-2023.g71e6beb6.el9.x86 14 MB/s | 5.2 MB 00:00 2024-03-14T12:11:17.265 INFO:teuthology.orchestra.run.smithi164.stdout:(19/174): python3-ceph-argparse-19.0.0-2023.g71 777 kB/s | 45 kB 00:00 2024-03-14T12:11:17.324 INFO:teuthology.orchestra.run.smithi164.stdout:(20/174): python3-ceph-common-19.0.0-2023.g71e6 2.2 MB/s | 131 kB 00:00 2024-03-14T12:11:17.382 INFO:teuthology.orchestra.run.smithi164.stdout:(21/174): python3-cephfs-19.0.0-2023.g71e6beb6. 2.7 MB/s | 162 kB 00:00 2024-03-14T12:11:17.449 INFO:teuthology.orchestra.run.smithi164.stdout:(22/174): python3-rados-19.0.0-2023.g71e6beb6.e 4.7 MB/s | 320 kB 00:00 2024-03-14T12:11:17.508 INFO:teuthology.orchestra.run.smithi164.stdout:(23/174): python3-rbd-19.0.0-2023.g71e6beb6.el9 5.0 MB/s | 298 kB 00:00 2024-03-14T12:11:17.575 INFO:teuthology.orchestra.run.smithi164.stdout:(24/174): python3-rgw-19.0.0-2023.g71e6beb6.el9 1.5 MB/s | 99 kB 00:00 2024-03-14T12:11:17.683 INFO:teuthology.orchestra.run.smithi164.stdout:(25/174): ceph-radosgw-19.0.0-2023.g71e6beb6.el 6.4 MB/s | 9.9 MB 00:01 2024-03-14T12:11:17.709 INFO:teuthology.orchestra.run.smithi164.stdout:(26/174): rbd-fuse-19.0.0-2023.g71e6beb6.el9.x8 639 kB/s | 85 kB 00:00 2024-03-14T12:11:17.768 INFO:teuthology.orchestra.run.smithi164.stdout:(27/174): rbd-nbd-19.0.0-2023.g71e6beb6.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-14T12:11:17.834 INFO:teuthology.orchestra.run.smithi164.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2023.g 511 kB/s | 34 kB 00:00 2024-03-14T12:11:17.901 INFO:teuthology.orchestra.run.smithi164.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2023.g71e6beb 2.0 MB/s | 139 kB 00:00 2024-03-14T12:11:17.968 INFO:teuthology.orchestra.run.smithi164.stdout:(30/174): rbd-mirror-19.0.0-2023.g71e6beb6.el9. 11 MB/s | 3.1 MB 00:00 2024-03-14T12:11:18.027 INFO:teuthology.orchestra.run.smithi164.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2023.g71e6b 14 MB/s | 1.8 MB 00:00 2024-03-14T12:11:18.094 INFO:teuthology.orchestra.run.smithi164.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2023.g71 3.6 MB/s | 248 kB 00:00 2024-03-14T12:11:18.153 INFO:teuthology.orchestra.run.smithi164.stdout:(33/174): ceph-mgr-rook-19.0.0-2023.g71e6beb6.e 848 kB/s | 49 kB 00:00 2024-03-14T12:11:18.211 INFO:teuthology.orchestra.run.smithi164.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2023.g7 282 kB/s | 16 kB 00:00 2024-03-14T12:11:18.287 INFO:teuthology.orchestra.run.smithi164.stdout:(35/174): ceph-volume-19.0.0-2023.g71e6beb6.el9 3.4 MB/s | 264 kB 00:00 2024-03-14T12:11:18.379 INFO:teuthology.orchestra.run.smithi164.stdout:(36/174): cephadm-19.0.0-2023.g71e6beb6.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-14T12:11:18.445 INFO:teuthology.orchestra.run.smithi164.stdout:(37/174): cephfs-top-19.0.0-2023.g71e6beb6.el9. 274 kB/s | 18 kB 00:00 2024-03-14T12:11:18.529 INFO:teuthology.orchestra.run.smithi164.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-14T12:11:18.679 INFO:teuthology.orchestra.run.smithi164.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 662 kB 00:00 2024-03-14T12:11:18.721 INFO:teuthology.orchestra.run.smithi164.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.4 MB/s | 920 kB 00:00 2024-03-14T12:11:18.746 INFO:teuthology.orchestra.run.smithi164.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-14T12:11:18.772 INFO:teuthology.orchestra.run.smithi164.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-14T12:11:18.797 INFO:teuthology.orchestra.run.smithi164.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-03-14T12:11:18.889 INFO:teuthology.orchestra.run.smithi164.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-14T12:11:18.914 INFO:teuthology.orchestra.run.smithi164.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-14T12:11:18.940 INFO:teuthology.orchestra.run.smithi164.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-03-14T12:11:18.965 INFO:teuthology.orchestra.run.smithi164.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-14T12:11:18.990 INFO:teuthology.orchestra.run.smithi164.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-14T12:11:19.015 INFO:teuthology.orchestra.run.smithi164.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-14T12:11:19.041 INFO:teuthology.orchestra.run.smithi164.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-14T12:11:19.066 INFO:teuthology.orchestra.run.smithi164.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 937 kB/s | 46 kB 00:00 2024-03-14T12:11:19.091 INFO:teuthology.orchestra.run.smithi164.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-14T12:11:19.116 INFO:teuthology.orchestra.run.smithi164.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-14T12:11:19.142 INFO:teuthology.orchestra.run.smithi164.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-14T12:11:19.167 INFO:teuthology.orchestra.run.smithi164.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-14T12:11:19.268 INFO:teuthology.orchestra.run.smithi164.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-14T12:11:19.309 INFO:teuthology.orchestra.run.smithi164.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-14T12:11:19.335 INFO:teuthology.orchestra.run.smithi164.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-14T12:11:19.360 INFO:teuthology.orchestra.run.smithi164.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-14T12:11:19.477 INFO:teuthology.orchestra.run.smithi164.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-03-14T12:11:19.511 INFO:teuthology.orchestra.run.smithi164.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-03-14T12:11:19.536 INFO:teuthology.orchestra.run.smithi164.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-14T12:11:19.661 INFO:teuthology.orchestra.run.smithi164.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 5.8 MB 00:00 2024-03-14T12:11:19.686 INFO:teuthology.orchestra.run.smithi164.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-14T12:11:19.728 INFO:teuthology.orchestra.run.smithi164.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 271 kB/s | 18 kB 00:00 2024-03-14T12:11:19.754 INFO:teuthology.orchestra.run.smithi164.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-14T12:11:19.779 INFO:teuthology.orchestra.run.smithi164.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-14T12:11:19.904 INFO:teuthology.orchestra.run.smithi164.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-14T12:11:19.929 INFO:teuthology.orchestra.run.smithi164.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-14T12:11:19.955 INFO:teuthology.orchestra.run.smithi164.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-14T12:11:19.980 INFO:teuthology.orchestra.run.smithi164.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-14T12:11:20.005 INFO:teuthology.orchestra.run.smithi164.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-14T12:11:20.031 INFO:teuthology.orchestra.run.smithi164.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-14T12:11:20.056 INFO:teuthology.orchestra.run.smithi164.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-14T12:11:20.081 INFO:teuthology.orchestra.run.smithi164.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-14T12:11:20.126 INFO:teuthology.orchestra.run.smithi164.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 47 kB 00:00 2024-03-14T12:11:20.156 INFO:teuthology.orchestra.run.smithi164.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-14T12:11:20.332 INFO:teuthology.orchestra.run.smithi164.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 223 kB/s | 39 kB 00:00 2024-03-14T12:11:20.482 INFO:teuthology.orchestra.run.smithi164.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-14T12:11:20.508 INFO:teuthology.orchestra.run.smithi164.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 237 kB/s | 41 kB 00:00 2024-03-14T12:11:20.533 INFO:teuthology.orchestra.run.smithi164.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-14T12:11:20.558 INFO:teuthology.orchestra.run.smithi164.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-14T12:11:20.577 INFO:teuthology.orchestra.run.smithi164.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-14T12:11:20.592 INFO:teuthology.orchestra.run.smithi164.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-14T12:11:20.617 INFO:teuthology.orchestra.run.smithi164.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-14T12:11:20.634 INFO:teuthology.orchestra.run.smithi164.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-14T12:11:20.676 INFO:teuthology.orchestra.run.smithi164.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-14T12:11:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-14T12:11:20.727 INFO:teuthology.orchestra.run.smithi164.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-14T12:11:20.744 INFO:teuthology.orchestra.run.smithi164.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-03-14T12:11:20.886 INFO:teuthology.orchestra.run.smithi164.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-14T12:11:20.927 INFO:teuthology.orchestra.run.smithi164.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-03-14T12:11:20.961 INFO:teuthology.orchestra.run.smithi164.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-03-14T12:11:20.978 INFO:teuthology.orchestra.run.smithi164.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-14T12:11:21.003 INFO:teuthology.orchestra.run.smithi164.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-03-14T12:11:21.029 INFO:teuthology.orchestra.run.smithi164.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-03-14T12:11:21.033 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-14T12:11:21.087 INFO:teuthology.orchestra.run.smithi164.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-03-14T12:11:21.154 INFO:teuthology.orchestra.run.smithi164.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 27 MB/s | 4.1 MB 00:00 2024-03-14T12:11:21.180 INFO:teuthology.orchestra.run.smithi164.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-03-14T12:11:21.205 INFO:teuthology.orchestra.run.smithi164.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-14T12:11:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 694 kB/s | 35 kB 00:00 2024-03-14T12:11:21.255 INFO:teuthology.orchestra.run.smithi164.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-03-14T12:11:21.281 INFO:teuthology.orchestra.run.smithi164.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-14T12:11:21.373 INFO:teuthology.orchestra.run.smithi164.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-03-14T12:11:21.823 INFO:teuthology.orchestra.run.smithi164.stdout:(105/174): ceph-test-19.0.0-2023.g71e6beb6.el9. 8.7 MB/s | 48 MB 00:05 2024-03-14T12:11:21.931 INFO:teuthology.orchestra.run.smithi164.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-03-14T12:11:22.340 INFO:teuthology.orchestra.run.smithi164.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-03-14T12:11:22.460 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 15 kB 00:00 2024-03-14T12:11:22.482 INFO:teuthology.orchestra.run.smithi164.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 494 kB/s | 548 kB 00:01 2024-03-14T12:11:22.541 INFO:teuthology.orchestra.run.smithi164.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 447 kB/s | 309 kB 00:00 2024-03-14T12:11:22.591 INFO:teuthology.orchestra.run.smithi164.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-03-14T12:11:22.683 INFO:teuthology.orchestra.run.smithi164.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 342 kB/s | 49 kB 00:00 2024-03-14T12:11:22.700 INFO:teuthology.orchestra.run.smithi164.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-03-14T12:11:22.825 INFO:teuthology.orchestra.run.smithi164.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-14T12:11:22.917 INFO:teuthology.orchestra.run.smithi164.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-03-14T12:11:23.026 INFO:teuthology.orchestra.run.smithi164.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-03-14T12:11:23.051 INFO:teuthology.orchestra.run.smithi164.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-03-14T12:11:23.126 INFO:teuthology.orchestra.run.smithi164.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-03-14T12:11:23.143 INFO:teuthology.orchestra.run.smithi164.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-03-14T12:11:23.160 INFO:teuthology.orchestra.run.smithi164.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-14T12:11:23.235 INFO:teuthology.orchestra.run.smithi164.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-03-14T12:11:23.294 INFO:teuthology.orchestra.run.smithi164.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-14T12:11:23.319 INFO:teuthology.orchestra.run.smithi164.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-03-14T12:11:23.345 INFO:teuthology.orchestra.run.smithi164.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 323 kB/s | 35 kB 00:00 2024-03-14T12:11:23.420 INFO:teuthology.orchestra.run.smithi164.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-14T12:11:23.462 INFO:teuthology.orchestra.run.smithi164.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-14T12:11:23.529 INFO:teuthology.orchestra.run.smithi164.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-03-14T12:11:23.579 INFO:teuthology.orchestra.run.smithi164.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-03-14T12:11:23.629 INFO:teuthology.orchestra.run.smithi164.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-03-14T12:11:23.655 INFO:teuthology.orchestra.run.smithi164.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-03-14T12:11:23.688 INFO:teuthology.orchestra.run.smithi164.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-03-14T12:11:23.738 INFO:teuthology.orchestra.run.smithi164.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 180 kB/s | 20 kB 00:00 2024-03-14T12:11:23.805 INFO:teuthology.orchestra.run.smithi164.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-03-14T12:11:23.839 INFO:teuthology.orchestra.run.smithi164.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-03-14T12:11:23.864 INFO:teuthology.orchestra.run.smithi164.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-03-14T12:11:23.914 INFO:teuthology.orchestra.run.smithi164.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-03-14T12:11:24.031 INFO:teuthology.orchestra.run.smithi164.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-03-14T12:11:24.057 INFO:teuthology.orchestra.run.smithi164.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 85 kB/s | 16 kB 00:00 2024-03-14T12:11:24.082 INFO:teuthology.orchestra.run.smithi164.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 538 kB/s | 90 kB 00:00 2024-03-14T12:11:24.149 INFO:teuthology.orchestra.run.smithi164.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-03-14T12:11:24.174 INFO:teuthology.orchestra.run.smithi164.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-03-14T12:11:24.199 INFO:teuthology.orchestra.run.smithi164.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-03-14T12:11:24.266 INFO:teuthology.orchestra.run.smithi164.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-03-14T12:11:24.291 INFO:teuthology.orchestra.run.smithi164.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-03-14T12:11:24.317 INFO:teuthology.orchestra.run.smithi164.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-03-14T12:11:24.392 INFO:teuthology.orchestra.run.smithi164.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-14T12:11:24.417 INFO:teuthology.orchestra.run.smithi164.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-03-14T12:11:24.451 INFO:teuthology.orchestra.run.smithi164.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-03-14T12:11:24.501 INFO:teuthology.orchestra.run.smithi164.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-03-14T12:11:24.551 INFO:teuthology.orchestra.run.smithi164.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-14T12:11:24.877 INFO:teuthology.orchestra.run.smithi164.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-03-14T12:11:24.935 INFO:teuthology.orchestra.run.smithi164.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-14T12:11:25.136 INFO:teuthology.orchestra.run.smithi164.stdout:(152/174): librados2-19.0.0-2023.g71e6beb6.el9. 5.3 MB/s | 3.4 MB 00:00 2024-03-14T12:11:25.377 INFO:teuthology.orchestra.run.smithi164.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 5.2 MB/s | 2.3 MB 00:00 2024-03-14T12:11:25.445 INFO:teuthology.orchestra.run.smithi164.stdout:(154/174): NetworkManager-libnm-1.46.0-1.el9.x8 5.9 MB/s | 1.8 MB 00:00 2024-03-14T12:11:25.470 INFO:teuthology.orchestra.run.smithi164.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 462 kB/s | 42 kB 00:00 2024-03-14T12:11:25.495 INFO:teuthology.orchestra.run.smithi164.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-03-14T12:11:25.520 INFO:teuthology.orchestra.run.smithi164.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-14T12:11:25.546 INFO:teuthology.orchestra.run.smithi164.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 726 kB/s | 36 kB 00:00 2024-03-14T12:11:25.579 INFO:teuthology.orchestra.run.smithi164.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-14T12:11:25.613 INFO:teuthology.orchestra.run.smithi164.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-03-14T12:11:25.638 INFO:teuthology.orchestra.run.smithi164.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 392 kB/s | 23 kB 00:00 2024-03-14T12:11:25.663 INFO:teuthology.orchestra.run.smithi164.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-14T12:11:25.688 INFO:teuthology.orchestra.run.smithi164.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-14T12:11:25.714 INFO:teuthology.orchestra.run.smithi164.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-14T12:11:25.731 INFO:teuthology.orchestra.run.smithi164.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-14T12:11:25.747 INFO:teuthology.orchestra.run.smithi164.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 859 kB/s | 29 kB 00:00 2024-03-14T12:11:25.839 INFO:teuthology.orchestra.run.smithi164.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 24 MB/s | 2.2 MB 00:00 2024-03-14T12:11:25.873 INFO:teuthology.orchestra.run.smithi164.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-03-14T12:11:25.924 INFO:teuthology.orchestra.run.smithi164.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 27 MB/s | 2.3 MB 00:00 2024-03-14T12:11:25.949 INFO:teuthology.orchestra.run.smithi164.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-14T12:11:25.965 INFO:teuthology.orchestra.run.smithi164.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 748 kB/s | 31 kB 00:00 2024-03-14T12:11:26.007 INFO:teuthology.orchestra.run.smithi164.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 74 kB 00:00 2024-03-14T12:11:26.058 INFO:teuthology.orchestra.run.smithi164.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-03-14T12:11:27.422 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 21 MB 00:04 2024-03-14T12:11:28.091 INFO:teuthology.orchestra.run.smithi164.stdout:(174/174): librbd1-19.0.0-2023.g71e6beb6.el9.x8 908 kB/s | 3.1 MB 00:03 2024-03-14T12:11:28.098 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-03-14T12:11:28.099 INFO:teuthology.orchestra.run.smithi164.stdout:Total 14 MB/s | 224 MB 00:15 2024-03-14T12:11:28.569 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-03-14T12:11:28.885 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-03-14T12:11:28.885 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-03-14T12:11:30.899 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-03-14T12:11:30.899 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-03-14T12:11:32.310 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-14T12:11:34.865 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T12:11:34.866 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T12:11:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-14T12:11:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-14T12:11:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-14T12:11:34.883 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-14T12:11:34.885 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-14T12:11:34.888 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-14T12:11:34.889 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-14T12:11:34.969 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-14T12:11:34.983 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 6.6 k 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 5.4 M 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 888 k 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 145 k 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 1.5 M 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 139 k 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 1.8 M 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 7.4 M 2024-03-14T12:11:34.984 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 49 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 9.9 M 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 48 M 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 264 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 748 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 220 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 27 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 18 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 31 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 745 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 127 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-14T12:11:34.985 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 162 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 320 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 298 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 99 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 85 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 3.1 M 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 170 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-14T12:11:34.986 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-14T12:11:34.987 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 3.4 M 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 3.1 M 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 20 M 2024-03-14T12:11:34.988 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 34 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 2.1 M 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 248 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 4.7 M 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 17 M 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2023.g71e6beb6.el9 ceph-noarch 16 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 25 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-14T12:11:34.989 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 163 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 501 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 5.2 M 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-14T12:11:34.990 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-14T12:11:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 45 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.0.0-2023.g71e6beb6.el9 ceph 131 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-14T12:11:34.992 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-14T12:11:34.993 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-14T12:11:34.994 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-14T12:11:34.995 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-14T12:11:34.996 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-03-14T12:11:34.997 INFO:teuthology.orchestra.run.smithi082.stdout:Install 150 Packages 2024-03-14T12:11:34.997 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 24 Packages 2024-03-14T12:11:34.997 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:11:34.997 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 224 M 2024-03-14T12:11:34.997 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-03-14T12:11:35.894 INFO:teuthology.orchestra.run.smithi082.stdout:(1/174): ceph-19.0.0-2023.g71e6beb6.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-03-14T12:11:36.344 INFO:teuthology.orchestra.run.smithi082.stdout:(2/174): ceph-fuse-19.0.0-2023.g71e6beb6.el9.x8 1.9 MB/s | 888 kB 00:00 2024-03-14T12:11:36.411 INFO:teuthology.orchestra.run.smithi082.stdout:(3/174): ceph-immutable-object-cache-19.0.0-202 2.1 MB/s | 145 kB 00:00 2024-03-14T12:11:36.853 INFO:teuthology.orchestra.run.smithi082.stdout:(4/174): ceph-mds-19.0.0-2023.g71e6beb6.el9.x86 4.8 MB/s | 2.1 MB 00:00 2024-03-14T12:11:37.061 INFO:teuthology.orchestra.run.smithi082.stdout:(5/174): ceph-mgr-19.0.0-2023.g71e6beb6.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-03-14T12:11:37.570 INFO:teuthology.orchestra.run.smithi082.stdout:(6/174): ceph-mon-19.0.0-2023.g71e6beb6.el9.x86 9.2 MB/s | 4.7 MB 00:00 2024-03-14T12:11:37.867 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-03-14T12:11:37.987 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-14T12:11:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-14T12:11:38.145 INFO:teuthology.orchestra.run.smithi082.stdout:(7/174): ceph-common-19.0.0-2023.g71e6beb6.el9. 8.3 MB/s | 20 MB 00:02 2024-03-14T12:11:38.220 INFO:teuthology.orchestra.run.smithi082.stdout:(8/174): ceph-base-19.0.0-2023.g71e6beb6.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-03-14T12:11:38.320 INFO:teuthology.orchestra.run.smithi082.stdout:(9/174): ceph-selinux-19.0.0-2023.g71e6beb6.el9 252 kB/s | 25 kB 00:00 2024-03-14T12:11:38.368 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-14T12:11:38.383 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-14T12:11:38.444 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-14T12:11:38.500 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-14T12:11:38.539 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-14T12:11:38.564 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-14T12:11:38.618 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-14T12:11:38.871 INFO:teuthology.orchestra.run.smithi082.stdout:(10/174): ceph-radosgw-19.0.0-2023.g71e6beb6.el 14 MB/s | 9.9 MB 00:00 2024-03-14T12:11:38.937 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-14T12:11:38.946 INFO:teuthology.orchestra.run.smithi082.stdout:(11/174): cephfs-mirror-19.0.0-2023.g71e6beb6.e 2.9 MB/s | 220 kB 00:00 2024-03-14T12:11:39.004 INFO:teuthology.orchestra.run.smithi082.stdout:(12/174): cephfs-shell-19.0.0-2023.g71e6beb6.el 464 kB/s | 27 kB 00:00 2024-03-14T12:11:39.017 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 10/198 2024-03-14T12:11:39.071 INFO:teuthology.orchestra.run.smithi082.stdout:(13/174): libcephfs-devel-19.0.0-2023.g71e6beb6 469 kB/s | 31 kB 00:00 2024-03-14T12:11:39.155 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 10/198 2024-03-14T12:11:39.185 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 11/198 2024-03-14T12:11:39.247 INFO:teuthology.orchestra.run.smithi082.stdout:(14/174): ceph-osd-19.0.0-2023.g71e6beb6.el9.x8 10 MB/s | 17 MB 00:01 2024-03-14T12:11:39.276 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 11/198 2024-03-14T12:11:39.289 INFO:teuthology.orchestra.run.smithi082.stdout:(15/174): libcephfs2-19.0.0-2023.g71e6beb6.el9. 3.4 MB/s | 745 kB 00:00 2024-03-14T12:11:39.309 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-2023.g71e6beb6.el9.x86_64 12/198 2024-03-14T12:11:39.314 INFO:teuthology.orchestra.run.smithi082.stdout:(16/174): libcephsqlite-19.0.0-2023.g71e6beb6.e 2.4 MB/s | 163 kB 00:00 2024-03-14T12:11:39.339 INFO:teuthology.orchestra.run.smithi082.stdout:(17/174): librados-devel-19.0.0-2023.g71e6beb6. 2.5 MB/s | 127 kB 00:00 2024-03-14T12:11:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-14T12:11:39.373 INFO:teuthology.orchestra.run.smithi082.stdout:(18/174): libradosstriper1-19.0.0-2023.g71e6beb 8.4 MB/s | 501 kB 00:00 2024-03-14T12:11:39.431 INFO:teuthology.orchestra.run.smithi082.stdout:(19/174): python3-ceph-argparse-19.0.0-2023.g71 776 kB/s | 45 kB 00:00 2024-03-14T12:11:39.507 INFO:teuthology.orchestra.run.smithi082.stdout:(20/174): python3-ceph-common-19.0.0-2023.g71e6 1.7 MB/s | 131 kB 00:00 2024-03-14T12:11:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-14T12:11:39.590 INFO:teuthology.orchestra.run.smithi082.stdout:(21/174): python3-cephfs-19.0.0-2023.g71e6beb6. 1.9 MB/s | 162 kB 00:00 2024-03-14T12:11:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-14T12:11:39.632 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 16/198 2024-03-14T12:11:39.682 INFO:teuthology.orchestra.run.smithi082.stdout:(22/174): python3-rados-19.0.0-2023.g71e6beb6.e 3.4 MB/s | 320 kB 00:00 2024-03-14T12:11:39.727 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 16/198 2024-03-14T12:11:39.749 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86 17/198 2024-03-14T12:11:39.749 INFO:teuthology.orchestra.run.smithi082.stdout:(23/174): librgw2-19.0.0-2023.g71e6beb6.el9.x86 13 MB/s | 5.2 MB 00:00 2024-03-14T12:11:39.774 INFO:teuthology.orchestra.run.smithi082.stdout:(24/174): python3-rbd-19.0.0-2023.g71e6beb6.el9 3.2 MB/s | 298 kB 00:00 2024-03-14T12:11:39.800 INFO:teuthology.orchestra.run.smithi082.stdout:(25/174): python3-rgw-19.0.0-2023.g71e6beb6.el9 1.9 MB/s | 99 kB 00:00 2024-03-14T12:11:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86 17/198 2024-03-14T12:11:39.833 INFO:teuthology.orchestra.run.smithi082.stdout:(26/174): rbd-fuse-19.0.0-2023.g71e6beb6.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-14T12:11:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-14T12:11:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-14T12:11:39.891 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-14T12:11:39.917 INFO:teuthology.orchestra.run.smithi082.stdout:(27/174): rbd-nbd-19.0.0-2023.g71e6beb6.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-14T12:11:40.000 INFO:teuthology.orchestra.run.smithi082.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2023.g 408 kB/s | 34 kB 00:00 2024-03-14T12:11:40.059 INFO:teuthology.orchestra.run.smithi082.stdout:(29/174): rbd-mirror-19.0.0-2023.g71e6beb6.el9. 12 MB/s | 3.1 MB 00:00 2024-03-14T12:11:40.084 INFO:teuthology.orchestra.run.smithi082.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2023.g71e6beb 1.6 MB/s | 139 kB 00:00 2024-03-14T12:11:40.085 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-14T12:11:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-14T12:11:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-14T12:11:40.285 INFO:teuthology.orchestra.run.smithi082.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2023.g71e6b 8.0 MB/s | 1.8 MB 00:00 2024-03-14T12:11:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-14T12:11:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2023.g71 3.7 MB/s | 248 kB 00:00 2024-03-14T12:11:40.419 INFO:teuthology.orchestra.run.smithi082.stdout:(33/174): ceph-mgr-rook-19.0.0-2023.g71e6beb6.e 740 kB/s | 49 kB 00:00 2024-03-14T12:11:40.477 INFO:teuthology.orchestra.run.smithi082.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2023.g7 283 kB/s | 16 kB 00:00 2024-03-14T12:11:40.544 INFO:teuthology.orchestra.run.smithi082.stdout:(35/174): ceph-volume-19.0.0-2023.g71e6beb6.el9 3.9 MB/s | 264 kB 00:00 2024-03-14T12:11:40.686 INFO:teuthology.orchestra.run.smithi082.stdout:(36/174): cephadm-19.0.0-2023.g71e6beb6.el9.noa 5.2 MB/s | 748 kB 00:00 2024-03-14T12:11:40.700 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-14T12:11:40.745 INFO:teuthology.orchestra.run.smithi082.stdout:(37/174): cephfs-top-19.0.0-2023.g71e6beb6.el9. 313 kB/s | 18 kB 00:00 2024-03-14T12:11:40.760 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-14T12:11:40.884 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-14T12:11:40.886 INFO:teuthology.orchestra.run.smithi082.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2024-03-14T12:11:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-14T12:11:41.029 INFO:teuthology.orchestra.run.smithi082.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 662 kB 00:00 2024-03-14T12:11:41.095 INFO:teuthology.orchestra.run.smithi082.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2024-03-14T12:11:41.120 INFO:teuthology.orchestra.run.smithi082.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-03-14T12:11:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-14T12:11:41.146 INFO:teuthology.orchestra.run.smithi082.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-14T12:11:41.171 INFO:teuthology.orchestra.run.smithi082.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-14T12:11:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-14T12:11:41.313 INFO:teuthology.orchestra.run.smithi082.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-03-14T12:11:41.326 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-14T12:11:41.338 INFO:teuthology.orchestra.run.smithi082.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 103 kB/s | 17 kB 00:00 2024-03-14T12:11:41.366 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-14T12:11:41.372 INFO:teuthology.orchestra.run.smithi082.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 303 kB/s | 10 kB 00:00 2024-03-14T12:11:41.397 INFO:teuthology.orchestra.run.smithi082.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-14T12:11:41.398 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-14T12:11:41.414 INFO:teuthology.orchestra.run.smithi082.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-14T12:11:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-14T12:11:41.456 INFO:teuthology.orchestra.run.smithi082.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-14T12:11:41.481 INFO:teuthology.orchestra.run.smithi082.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-03-14T12:11:41.498 INFO:teuthology.orchestra.run.smithi082.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-14T12:11:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-14T12:11:41.523 INFO:teuthology.orchestra.run.smithi082.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-14T12:11:41.549 INFO:teuthology.orchestra.run.smithi082.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2024-03-14T12:11:41.563 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-14T12:11:41.574 INFO:teuthology.orchestra.run.smithi082.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-14T12:11:41.599 INFO:teuthology.orchestra.run.smithi082.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-14T12:11:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-14T12:11:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-14T12:11:41.710 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-14T12:11:41.750 INFO:teuthology.orchestra.run.smithi082.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-14T12:11:41.753 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-14T12:11:41.783 INFO:teuthology.orchestra.run.smithi082.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 8.3 MB/s | 1.5 MB 00:00 2024-03-14T12:11:41.786 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-14T12:11:41.808 INFO:teuthology.orchestra.run.smithi082.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2024-03-14T12:11:41.814 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-14T12:11:41.834 INFO:teuthology.orchestra.run.smithi082.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-14T12:11:41.882 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-14T12:11:41.919 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-2023.g71e6beb6.el9. 39/198 2024-03-14T12:11:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-2023.g71e6beb6.el 40/198 2024-03-14T12:11:41.976 INFO:teuthology.orchestra.run.smithi082.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-14T12:11:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-2023.g71e6beb6.el9.x86_6 41/198 2024-03-14T12:11:42.009 INFO:teuthology.orchestra.run.smithi082.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-14T12:11:42.034 INFO:teuthology.orchestra.run.smithi082.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-14T12:11:42.059 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-14T12:11:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-14T12:11:42.187 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-14T12:11:42.327 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-14T12:11:42.374 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-14T12:11:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:(63/174): ceph-test-19.0.0-2023.g71e6beb6.el9.x 11 MB/s | 48 MB 00:04 2024-03-14T12:11:42.610 INFO:teuthology.orchestra.run.smithi082.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 9.6 MB/s | 5.8 MB 00:00 2024-03-14T12:11:42.635 INFO:teuthology.orchestra.run.smithi082.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 41 kB/s | 24 kB 00:00 2024-03-14T12:11:42.660 INFO:teuthology.orchestra.run.smithi082.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 123 kB/s | 18 kB 00:00 2024-03-14T12:11:42.686 INFO:teuthology.orchestra.run.smithi082.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-14T12:11:42.719 INFO:teuthology.orchestra.run.smithi082.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 545 kB/s | 32 kB 00:00 2024-03-14T12:11:42.811 INFO:teuthology.orchestra.run.smithi082.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-14T12:11:42.836 INFO:teuthology.orchestra.run.smithi082.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-14T12:11:42.861 INFO:teuthology.orchestra.run.smithi082.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-14T12:11:42.887 INFO:teuthology.orchestra.run.smithi082.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 922 kB/s | 69 kB 00:00 2024-03-14T12:11:42.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-14T12:11:42.912 INFO:teuthology.orchestra.run.smithi082.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-03-14T12:11:42.937 INFO:teuthology.orchestra.run.smithi082.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-14T12:11:42.962 INFO:teuthology.orchestra.run.smithi082.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-14T12:11:42.969 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-14T12:11:42.988 INFO:teuthology.orchestra.run.smithi082.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-03-14T12:11:43.013 INFO:teuthology.orchestra.run.smithi082.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-14T12:11:43.038 INFO:teuthology.orchestra.run.smithi082.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 567 kB/s | 43 kB 00:00 2024-03-14T12:11:43.059 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-14T12:11:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-14T12:11:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-14T12:11:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-14T12:11:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-14T12:11:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-2023.g71e6beb6.el9.x86_6 53/198 2024-03-14T12:11:43.289 INFO:teuthology.orchestra.run.smithi082.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 39 kB 00:00 2024-03-14T12:11:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-14T12:11:43.314 INFO:teuthology.orchestra.run.smithi082.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 150 kB/s | 41 kB 00:00 2024-03-14T12:11:43.456 INFO:teuthology.orchestra.run.smithi082.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-03-14T12:11:43.481 INFO:teuthology.orchestra.run.smithi082.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-14T12:11:43.506 INFO:teuthology.orchestra.run.smithi082.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 297 kB/s | 56 kB 00:00 2024-03-14T12:11:43.531 INFO:teuthology.orchestra.run.smithi082.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-14T12:11:43.556 INFO:teuthology.orchestra.run.smithi082.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 834 kB/s | 63 kB 00:00 2024-03-14T12:11:43.582 INFO:teuthology.orchestra.run.smithi082.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-14T12:11:43.607 INFO:teuthology.orchestra.run.smithi082.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-14T12:11:43.624 INFO:teuthology.orchestra.run.smithi082.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2024-03-14T12:11:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-14T12:11:43.658 INFO:teuthology.orchestra.run.smithi082.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-14T12:11:43.685 INFO:teuthology.orchestra.run.smithi082.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2024-03-14T12:11:43.708 INFO:teuthology.orchestra.run.smithi082.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-14T12:11:43.839 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-14T12:11:43.883 INFO:teuthology.orchestra.run.smithi082.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 936 kB/s | 187 kB 00:00 2024-03-14T12:11:43.908 INFO:teuthology.orchestra.run.smithi082.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 977 kB/s | 195 kB 00:00 2024-03-14T12:11:43.934 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-14T12:11:43.942 INFO:teuthology.orchestra.run.smithi082.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-03-14T12:11:43.967 INFO:teuthology.orchestra.run.smithi082.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-03-14T12:11:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-14T12:11:44.067 INFO:teuthology.orchestra.run.smithi082.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 135 kB/s | 21 kB 00:00 2024-03-14T12:11:44.093 INFO:teuthology.orchestra.run.smithi082.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 169 kB/s | 25 kB 00:00 2024-03-14T12:11:44.116 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-14T12:11:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-14T12:11:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-14T12:11:44.251 INFO:teuthology.orchestra.run.smithi082.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-03-14T12:11:44.276 INFO:teuthology.orchestra.run.smithi082.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-03-14T12:11:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-14T12:11:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-14T12:11:44.335 INFO:teuthology.orchestra.run.smithi082.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 600 kB/s | 35 kB 00:00 2024-03-14T12:11:44.360 INFO:teuthology.orchestra.run.smithi082.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.5 MB/s | 216 kB 00:00 2024-03-14T12:11:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-14T12:11:44.394 INFO:teuthology.orchestra.run.smithi082.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 760 kB/s | 44 kB 00:00 2024-03-14T12:11:44.415 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-14T12:11:44.419 INFO:teuthology.orchestra.run.smithi082.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-14T12:11:44.451 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-14T12:11:44.494 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-14T12:11:44.536 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-14T12:11:44.561 INFO:teuthology.orchestra.run.smithi082.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-03-14T12:11:44.636 INFO:teuthology.orchestra.run.smithi082.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-03-14T12:11:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-14T12:11:44.717 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-14T12:11:44.747 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-14T12:11:44.770 INFO:teuthology.orchestra.run.smithi082.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.2 MB/s | 4.1 MB 00:00 2024-03-14T12:11:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-14T12:11:44.891 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-14T12:11:44.895 INFO:teuthology.orchestra.run.smithi082.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-14T12:11:44.920 INFO:teuthology.orchestra.run.smithi082.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-14T12:11:44.931 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-14T12:11:44.945 INFO:teuthology.orchestra.run.smithi082.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-03-14T12:11:44.971 INFO:teuthology.orchestra.run.smithi082.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 513 kB/s | 26 kB 00:00 2024-03-14T12:11:44.971 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-14T12:11:44.996 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-14T12:11:45.005 INFO:teuthology.orchestra.run.smithi082.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 823 kB/s | 49 kB 00:00 2024-03-14T12:11:45.021 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-14T12:11:45.030 INFO:teuthology.orchestra.run.smithi082.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-14T12:11:45.072 INFO:teuthology.orchestra.run.smithi082.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-14T12:11:45.100 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-14T12:11:45.220 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-14T12:11:45.264 INFO:teuthology.orchestra.run.smithi082.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-03-14T12:11:45.289 INFO:teuthology.orchestra.run.smithi082.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-03-14T12:11:45.314 INFO:teuthology.orchestra.run.smithi082.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 863 kB/s | 43 kB 00:00 2024-03-14T12:11:45.320 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-14T12:11:45.339 INFO:teuthology.orchestra.run.smithi082.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-03-14T12:11:45.363 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-14T12:11:45.365 INFO:teuthology.orchestra.run.smithi082.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-03-14T12:11:45.411 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-14T12:11:45.457 INFO:teuthology.orchestra.run.smithi082.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-03-14T12:11:45.467 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-14T12:11:45.565 INFO:teuthology.orchestra.run.smithi082.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-03-14T12:11:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-14T12:11:45.632 INFO:teuthology.orchestra.run.smithi082.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 529 kB/s | 35 kB 00:00 2024-03-14T12:11:45.649 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-14T12:11:45.708 INFO:teuthology.orchestra.run.smithi082.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-14T12:11:45.717 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-14T12:11:45.794 INFO:teuthology.orchestra.run.smithi082.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-14T12:11:45.842 INFO:teuthology.orchestra.run.smithi082.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-03-14T12:11:45.844 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-14T12:11:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-14T12:11:45.900 INFO:teuthology.orchestra.run.smithi082.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 304 kB/s | 18 kB 00:00 2024-03-14T12:11:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-14T12:11:45.942 INFO:teuthology.orchestra.run.smithi082.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 557 kB/s | 23 kB 00:00 2024-03-14T12:11:45.953 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-14T12:11:45.976 INFO:teuthology.orchestra.run.smithi082.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-03-14T12:11:45.981 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-14T12:11:46.016 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-14T12:11:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 472 kB/s | 20 kB 00:00 2024-03-14T12:11:46.054 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-14T12:11:46.086 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-14T12:11:46.116 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-14T12:11:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-14T12:11:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-14T12:11:46.260 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-14T12:11:46.276 INFO:teuthology.orchestra.run.smithi082.stdout:(130/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-03-14T12:11:46.301 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-14T12:11:46.327 INFO:teuthology.orchestra.run.smithi082.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-03-14T12:11:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-14T12:11:46.352 INFO:teuthology.orchestra.run.smithi082.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-03-14T12:11:46.377 INFO:teuthology.orchestra.run.smithi082.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-14T12:11:46.411 INFO:teuthology.orchestra.run.smithi082.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-03-14T12:11:46.418 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-14T12:11:46.436 INFO:teuthology.orchestra.run.smithi082.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-14T12:11:46.503 INFO:teuthology.orchestra.run.smithi082.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 247 kB/s | 16 kB 00:00 2024-03-14T12:11:46.506 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-14T12:11:46.550 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-14T12:11:46.561 INFO:teuthology.orchestra.run.smithi082.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 720 kB/s | 90 kB 00:00 2024-03-14T12:11:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-14T12:11:46.586 INFO:teuthology.orchestra.run.smithi082.stdout:(138/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-03-14T12:11:46.603 INFO:teuthology.orchestra.run.smithi082.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-03-14T12:11:46.620 INFO:teuthology.orchestra.run.smithi082.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-03-14T12:11:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-14T12:11:46.671 INFO:teuthology.orchestra.run.smithi082.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 880 kB/s | 59 kB 00:00 2024-03-14T12:11:46.688 INFO:teuthology.orchestra.run.smithi082.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-03-14T12:11:46.722 INFO:teuthology.orchestra.run.smithi082.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-03-14T12:11:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-14T12:11:46.738 INFO:teuthology.orchestra.run.smithi082.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 760 kB/s | 51 kB 00:00 2024-03-14T12:11:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-14T12:11:46.788 INFO:teuthology.orchestra.run.smithi082.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-14T12:11:46.856 INFO:teuthology.orchestra.run.smithi082.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-03-14T12:11:46.873 INFO:teuthology.orchestra.run.smithi082.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-03-14T12:11:47.006 INFO:teuthology.orchestra.run.smithi082.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-03-14T12:11:47.032 INFO:teuthology.orchestra.run.smithi082.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.4 MB/s | 427 kB 00:00 2024-03-14T12:11:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-14T12:11:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-14T12:11:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-14T12:11:47.623 INFO:teuthology.orchestra.run.smithi082.stdout:(150/174): librados2-19.0.0-2023.g71e6beb6.el9. 5.5 MB/s | 3.4 MB 00:00 2024-03-14T12:11:47.638 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-14T12:11:47.690 INFO:teuthology.orchestra.run.smithi082.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-03-14T12:11:47.725 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-14T12:11:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-14T12:11:47.932 INFO:teuthology.orchestra.run.smithi082.stdout:(152/174): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-03-14T12:11:47.938 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-14T12:11:47.977 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-14T12:11:48.057 INFO:teuthology.orchestra.run.smithi082.stdout:(153/174): librbd1-19.0.0-2023.g71e6beb6.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-03-14T12:11:48.062 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-14T12:11:48.150 INFO:teuthology.orchestra.run.smithi082.stdout:(154/174): NetworkManager-1.46.0-1.el9.x86_64.r 5.0 MB/s | 2.3 MB 00:00 2024-03-14T12:11:48.191 INFO:teuthology.orchestra.run.smithi082.stdout:(155/174): NetworkManager-libnm-1.46.0-1.el9.x8 7.0 MB/s | 1.8 MB 00:00 2024-03-14T12:11:48.242 INFO:teuthology.orchestra.run.smithi082.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-14T12:11:48.267 INFO:teuthology.orchestra.run.smithi082.stdout:(157/174): NetworkManager-team-1.46.0-1.el9.x86 202 kB/s | 42 kB 00:00 2024-03-14T12:11:48.309 INFO:teuthology.orchestra.run.smithi082.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 544 kB/s | 36 kB 00:00 2024-03-14T12:11:48.368 INFO:teuthology.orchestra.run.smithi082.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-14T12:11:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:(160/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.0 MB/s | 248 kB 00:00 2024-03-14T12:11:48.418 INFO:teuthology.orchestra.run.smithi082.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 457 kB/s | 23 kB 00:00 2024-03-14T12:11:48.469 INFO:teuthology.orchestra.run.smithi082.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-14T12:11:48.502 INFO:teuthology.orchestra.run.smithi082.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 995 kB/s | 108 kB 00:00 2024-03-14T12:11:48.527 INFO:teuthology.orchestra.run.smithi082.stdout:(164/174): elfutils-libelf-0.190-2.el9.x86_64.r 740 kB/s | 192 kB 00:00 2024-03-14T12:11:48.552 INFO:teuthology.orchestra.run.smithi082.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-14T12:11:48.611 INFO:teuthology.orchestra.run.smithi082.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 586 kB/s | 64 kB 00:00 2024-03-14T12:11:48.644 INFO:teuthology.orchestra.run.smithi082.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-14T12:11:48.670 INFO:teuthology.orchestra.run.smithi082.stdout:(168/174): libuuid-2.37.4-18.el9.x86_64.rpm 201 kB/s | 29 kB 00:00 2024-03-14T12:11:48.711 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-14T12:11:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 21 MB/s | 2.3 MB 00:00 2024-03-14T12:11:48.822 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-14T12:11:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 374 kB/s | 31 kB 00:00 2024-03-14T12:11:48.998 INFO:teuthology.orchestra.run.smithi082.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-03-14T12:11:49.121 INFO:teuthology.orchestra.run.smithi082.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 595 kB/s | 74 kB 00:00 2024-03-14T12:11:49.271 INFO:teuthology.orchestra.run.smithi082.stdout:(173/174): util-linux-core-2.37.4-18.el9.x86_64 774 kB/s | 465 kB 00:00 2024-03-14T12:11:49.547 INFO:teuthology.orchestra.run.smithi082.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.3 MB/s | 2.2 MB 00:00 2024-03-14T12:11:49.554 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-03-14T12:11:49.554 INFO:teuthology.orchestra.run.smithi082.stdout:Total 15 MB/s | 224 MB 00:14 2024-03-14T12:11:50.016 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-03-14T12:11:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-14T12:11:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-14T12:11:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-14T12:11:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-14T12:11:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-14T12:11:50.326 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-03-14T12:11:50.326 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-03-14T12:11:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-14T12:11:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 124/198 2024-03-14T12:11:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 124/198 2024-03-14T12:11:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 125/198 2024-03-14T12:11:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-14T12:11:51.192 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-14T12:11:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 128/198 2024-03-14T12:11:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 128/198 2024-03-14T12:11:51.317 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 129/198 2024-03-14T12:11:52.358 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-03-14T12:11:52.358 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-03-14T12:11:53.478 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 130/198 2024-03-14T12:11:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 130/198 2024-03-14T12:11:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 130/198 2024-03-14T12:11:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 131/198 2024-03-14T12:11:54.184 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 131/198 2024-03-14T12:11:54.184 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-14T12:11:54.184 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-14T12:11:54.185 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:11:54.218 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 132/198 2024-03-14T12:11:59.282 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-03-14T12:11:59.383 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-14T12:11:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-14T12:11:59.456 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-14T12:11:59.465 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-14T12:11:59.505 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-14T12:11:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-14T12:11:59.592 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-14T12:11:59.626 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-14T12:11:59.669 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-14T12:11:59.995 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-14T12:12:00.053 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 10/198 2024-03-14T12:12:00.172 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 10/198 2024-03-14T12:12:00.203 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 11/198 2024-03-14T12:12:00.294 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 11/198 2024-03-14T12:12:00.334 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.0.0-2023.g71e6beb6.el9.x86_64 12/198 2024-03-14T12:12:00.377 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-14T12:12:00.558 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-14T12:12:00.623 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-14T12:12:00.667 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 16/198 2024-03-14T12:12:00.771 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 16/198 2024-03-14T12:12:00.793 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86 17/198 2024-03-14T12:12:00.858 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86 17/198 2024-03-14T12:12:00.906 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-14T12:12:00.928 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-14T12:12:00.943 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-14T12:12:01.129 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-14T12:12:01.160 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-14T12:12:01.191 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-14T12:12:01.390 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-14T12:12:01.510 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-14T12:12:01.570 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-14T12:12:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-14T12:12:01.729 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-14T12:12:01.933 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-14T12:12:02.043 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-14T12:12:02.136 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-14T12:12:02.167 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-14T12:12:02.194 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-14T12:12:02.229 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-14T12:12:02.303 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-14T12:12:02.365 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-14T12:12:02.404 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-14T12:12:02.467 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-14T12:12:02.503 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-14T12:12:02.546 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-14T12:12:02.580 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-14T12:12:02.607 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-14T12:12:02.676 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-14T12:12:02.712 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.0.0-2023.g71e6beb6.el9. 39/198 2024-03-14T12:12:02.747 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.0.0-2023.g71e6beb6.el 40/198 2024-03-14T12:12:02.775 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.0.0-2023.g71e6beb6.el9.x86_6 41/198 2024-03-14T12:12:02.843 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-14T12:12:02.892 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-14T12:12:02.964 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-14T12:12:03.099 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-14T12:12:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-14T12:12:03.690 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-14T12:12:03.773 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-14T12:12:03.850 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-14T12:12:03.902 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-14T12:12:03.910 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-14T12:12:03.943 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-14T12:12:03.982 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-14T12:12:04.020 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.0.0-2023.g71e6beb6.el9.x86_6 53/198 2024-03-14T12:12:04.059 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-14T12:12:04.332 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-14T12:12:04.493 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-14T12:12:04.578 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-14T12:12:04.621 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-14T12:12:05.014 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-14T12:12:05.058 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-14T12:12:05.114 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-14T12:12:05.186 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-14T12:12:05.225 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-14T12:12:05.253 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-14T12:12:05.300 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-14T12:12:05.336 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-14T12:12:05.379 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-14T12:12:05.421 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-14T12:12:05.555 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-14T12:12:05.602 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-14T12:12:05.632 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-14T12:12:05.673 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-14T12:12:05.767 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-14T12:12:05.807 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-14T12:12:05.848 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-14T12:12:05.873 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-14T12:12:05.898 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 132/198 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-03-14T12:12:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:05.986 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-14T12:12:06.107 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-14T12:12:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 133/198 2024-03-14T12:12:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 133/198 2024-03-14T12:12:06.130 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:06.130 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-14T12:12:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:06.207 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-14T12:12:06.256 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-14T12:12:06.305 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-14T12:12:06.360 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-14T12:12:06.544 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-14T12:12:06.585 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-14T12:12:06.714 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-14T12:12:06.759 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-14T12:12:06.790 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-14T12:12:06.821 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-14T12:12:06.849 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-14T12:12:06.892 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-14T12:12:06.931 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-14T12:12:06.962 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-14T12:12:06.993 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-14T12:12:07.069 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-14T12:12:07.111 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-14T12:12:07.145 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-14T12:12:07.186 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-14T12:12:07.241 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-14T12:12:07.326 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-14T12:12:07.408 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-14T12:12:07.452 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-14T12:12:07.483 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-14T12:12:07.530 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-14T12:12:07.628 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-14T12:12:07.636 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 134/198 2024-03-14T12:12:07.661 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 134/198 2024-03-14T12:12:07.661 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:07.661 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-14T12:12:07.661 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T12:12:07.661 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T12:12:07.661 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:07.675 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-14T12:12:07.870 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 135/198 2024-03-14T12:12:07.892 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 135/198 2024-03-14T12:12:07.892 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:07.892 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-14T12:12:07.892 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T12:12:07.892 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T12:12:07.892 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:08.329 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 136/198 2024-03-14T12:12:08.351 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 136/198 2024-03-14T12:12:08.351 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:08.351 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-14T12:12:08.351 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T12:12:08.352 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T12:12:08.352 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:08.396 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-14T12:12:08.439 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-14T12:12:08.448 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-14T12:12:08.483 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-14T12:12:08.530 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-14T12:12:08.610 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-14T12:12:08.642 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-14T12:12:08.822 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-14T12:12:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 138/198 2024-03-14T12:12:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 138/198 2024-03-14T12:12:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2023.g71e6beb6.e 139/198 2024-03-14T12:12:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2023.g71e6beb6. 140/198 2024-03-14T12:12:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noa 141/198 2024-03-14T12:12:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-14T12:12:09.188 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-14T12:12:09.217 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noa 141/198 2024-03-14T12:12:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.n 142/198 2024-03-14T12:12:09.921 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-14T12:12:10.031 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-14T12:12:10.212 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.n 142/198 2024-03-14T12:12:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e 143/198 2024-03-14T12:12:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e 143/198 2024-03-14T12:12:10.499 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2023.g71e6beb6.el 144/198 2024-03-14T12:12:10.531 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 145/198 2024-03-14T12:12:10.554 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 145/198 2024-03-14T12:12:10.554 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:10.554 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-14T12:12:10.554 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T12:12:10.554 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T12:12:10.554 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:10.598 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 146/198 2024-03-14T12:12:10.609 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 146/198 2024-03-14T12:12:10.663 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-2023.g71e6beb6.el9.x86_64 147/198 2024-03-14T12:12:11.253 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-14T12:12:11.343 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-14T12:12:11.374 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-14T12:12:11.405 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-14T12:12:11.433 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-14T12:12:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-14T12:12:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 149/198 2024-03-14T12:12:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 149/198 2024-03-14T12:12:11.581 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:11.581 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-14T12:12:11.581 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T12:12:11.581 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T12:12:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2023.g71e6b 150/198 2024-03-14T12:12:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2023.g71e6b 150/198 2024-03-14T12:12:11.648 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:11.648 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-14T12:12:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:11.693 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 151/198 2024-03-14T12:12:11.715 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 151/198 2024-03-14T12:12:11.716 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:11.716 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-14T12:12:11.716 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T12:12:11.716 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T12:12:11.716 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:11.745 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-14T12:12:11.780 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 124/198 2024-03-14T12:12:11.853 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 124/198 2024-03-14T12:12:11.884 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 125/198 2024-03-14T12:12:11.921 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-14T12:12:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 152/198 2024-03-14T12:12:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 152/198 2024-03-14T12:12:12.041 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:12.041 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-14T12:12:12.041 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T12:12:12.041 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T12:12:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-14T12:12:12.416 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 128/198 2024-03-14T12:12:12.470 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 128/198 2024-03-14T12:12:12.493 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 129/198 2024-03-14T12:12:14.621 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 130/198 2024-03-14T12:12:14.656 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 130/198 2024-03-14T12:12:15.194 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 130/198 2024-03-14T12:12:15.226 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 131/198 2024-03-14T12:12:15.298 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 131/198 2024-03-14T12:12:15.298 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-14T12:12:15.298 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-14T12:12:15.299 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:15.335 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 132/198 2024-03-14T12:12:16.516 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-2023.g71e6beb6.el9.x86_64 153/198 2024-03-14T12:12:16.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 154/198 2024-03-14T12:12:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 155/198 2024-03-14T12:12:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-14T12:12:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-2023.g71e6beb6.el9.x86_64 157/198 2024-03-14T12:12:16.945 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-14T12:12:16.997 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 159/198 2024-03-14T12:12:17.044 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-14T12:12:17.103 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-14T12:12:17.130 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-14T12:12:17.163 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-2023.g71e6beb6.el9.x86_ 163/198 2024-03-14T12:12:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-14T12:12:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-14T12:12:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-14T12:12:17.372 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-14T12:12:17.398 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-14T12:12:17.921 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-14T12:12:17.971 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-14T12:12:17.991 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-14T12:12:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-2023.g71e6beb6.el9.noarch 170/198 2024-03-14T12:12:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-14T12:12:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-14T12:12:19.033 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-14T12:12:19.064 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-14T12:12:19.089 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-14T12:12:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-14T12:12:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-14T12:12:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-14T12:12:19.221 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-14T12:12:19.250 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-14T12:12:19.251 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-14T12:12:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-14T12:12:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-14T12:12:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-14T12:12:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-14T12:12:19.445 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-14T12:12:19.480 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-14T12:12:19.518 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-14T12:12:19.550 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-14T12:12:19.575 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-14T12:12:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-14T12:12:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-14T12:12:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-14T12:12:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-14T12:12:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-14T12:12:19.837 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-14T12:12:19.837 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-14T12:12:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-14T12:12:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-14T12:12:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-14T12:12:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-14T12:12:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-14T12:12:20.096 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-14T12:12:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-14T12:12:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-2023.g71e6beb6.el9.x86_64 1/198 2024-03-14T12:12:25.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 2/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 3/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 4/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2023.g71e6b 5/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 6/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 7/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 8/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 9/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 10/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 11/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-2023.g71e6beb6.el9.x86_64 12/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 13/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-2023.g71e6beb6.el9.x86_64 14/198 2024-03-14T12:12:25.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-2023.g71e6beb6.el9.x86_ 15/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 16/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 17/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-2023.g71e6beb6.el9.x86_6 18/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86 19/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 20/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2023.g71e6beb6.el 21/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-2023.g71e6beb6.el9. 22/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-2023.g71e6beb6.el9.x86_6 23/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-2023.g71e6beb6.el9.x86_64 24/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 25/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 26/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 27/198 2024-03-14T12:12:25.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 28/198 2024-03-14T12:12:25.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 29/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2023.g71e6beb6. 30/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noa 31/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.n 32/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e 33/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2023.g71e6beb6.el 34/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 35/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2023.g71e6beb6.e 36/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 37/198 2024-03-14T12:12:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 38/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-2023.g71e6beb6.el9.noarch 39/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-14T12:12:25.103 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-14T12:12:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-14T12:12:25.105 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-14T12:12:25.106 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-14T12:12:25.107 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-14T12:12:25.108 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-14T12:12:25.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-14T12:12:25.110 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 151/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 153/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-14T12:12:25.111 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-14T12:12:25.112 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-14T12:12:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-14T12:12:25.114 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 132/198 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-03-14T12:12:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:27.065 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 133/198 2024-03-14T12:12:27.080 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 133/198 2024-03-14T12:12:27.080 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:27.080 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-14T12:12:27.080 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:27.897 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-14T12:12:27.897 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: acl-2.3.1-4.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-14T12:12:27.898 INFO:teuthology.orchestra.run.smithi164.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.899 INFO:teuthology.orchestra.run.smithi164.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.900 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.901 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-14T12:12:27.902 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: jq-1.6-16.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-14T12:12:27.903 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-14T12:12:27.904 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.11-1.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.905 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-14T12:12:27.906 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-14T12:12:27.907 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-14T12:12:27.909 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-14T12:12:27.910 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-14T12:12:27.911 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-14T12:12:27.911 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-14T12:12:27.911 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:27.911 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-03-14T12:12:28.368 DEBUG:teuthology.parallel:result is None 2024-03-14T12:12:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 134/198 2024-03-14T12:12:28.577 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 134/198 2024-03-14T12:12:28.577 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:28.577 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-14T12:12:28.577 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T12:12:28.577 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T12:12:28.577 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:28.786 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 135/198 2024-03-14T12:12:28.808 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 135/198 2024-03-14T12:12:28.809 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:28.809 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-14T12:12:28.809 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T12:12:28.809 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T12:12:28.809 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:29.245 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 136/198 2024-03-14T12:12:29.267 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 136/198 2024-03-14T12:12:29.267 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:29.267 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-14T12:12:29.267 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T12:12:29.267 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T12:12:29.267 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:29.355 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-14T12:12:29.731 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 138/198 2024-03-14T12:12:29.762 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 138/198 2024-03-14T12:12:29.794 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2023.g71e6beb6.e 139/198 2024-03-14T12:12:29.853 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2023.g71e6beb6. 140/198 2024-03-14T12:12:29.877 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noa 141/198 2024-03-14T12:12:30.109 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noa 141/198 2024-03-14T12:12:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.n 142/198 2024-03-14T12:12:31.096 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.n 142/198 2024-03-14T12:12:31.128 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e 143/198 2024-03-14T12:12:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e 143/198 2024-03-14T12:12:31.398 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2023.g71e6beb6.el 144/198 2024-03-14T12:12:31.430 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 145/198 2024-03-14T12:12:31.453 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 145/198 2024-03-14T12:12:31.453 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:31.453 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-14T12:12:31.453 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T12:12:31.453 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T12:12:31.453 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:31.497 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 146/198 2024-03-14T12:12:31.508 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 146/198 2024-03-14T12:12:31.556 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.0.0-2023.g71e6beb6.el9.x86_64 147/198 2024-03-14T12:12:32.420 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-14T12:12:32.458 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 149/198 2024-03-14T12:12:32.480 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 149/198 2024-03-14T12:12:32.480 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:32.480 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-14T12:12:32.481 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T12:12:32.481 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T12:12:32.481 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:32.525 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2023.g71e6b 150/198 2024-03-14T12:12:32.546 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2023.g71e6b 150/198 2024-03-14T12:12:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-14T12:12:32.547 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:32.592 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 151/198 2024-03-14T12:12:32.614 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 151/198 2024-03-14T12:12:32.614 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:32.614 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-14T12:12:32.615 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T12:12:32.615 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T12:12:32.615 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:32.918 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 152/198 2024-03-14T12:12:32.940 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 152/198 2024-03-14T12:12:32.941 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T12:12:32.941 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-14T12:12:32.941 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T12:12:32.941 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T12:12:32.941 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.0.0-2023.g71e6beb6.el9.x86_64 153/198 2024-03-14T12:12:37.480 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 154/198 2024-03-14T12:12:37.510 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 155/198 2024-03-14T12:12:37.538 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-14T12:12:37.737 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.0.0-2023.g71e6beb6.el9.x86_64 157/198 2024-03-14T12:12:37.865 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-14T12:12:37.911 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 159/198 2024-03-14T12:12:37.961 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-14T12:12:38.030 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-14T12:12:38.063 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-14T12:12:38.096 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.0.0-2023.g71e6beb6.el9.x86_ 163/198 2024-03-14T12:12:38.145 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-14T12:12:38.174 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-14T12:12:38.249 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-14T12:12:38.315 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-14T12:12:38.347 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-14T12:12:38.899 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-14T12:12:38.937 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-14T12:12:38.957 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-14T12:12:39.784 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.0.0-2023.g71e6beb6.el9.noarch 170/198 2024-03-14T12:12:39.868 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-14T12:12:39.908 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-14T12:12:39.998 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-14T12:12:40.030 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-14T12:12:40.055 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-14T12:12:40.080 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-14T12:12:40.107 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-14T12:12:40.107 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-14T12:12:40.186 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-14T12:12:40.216 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-14T12:12:40.216 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-14T12:12:40.284 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-14T12:12:40.310 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-14T12:12:40.343 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-14T12:12:40.369 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-14T12:12:40.394 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-14T12:12:40.428 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-14T12:12:40.466 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-14T12:12:40.498 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-14T12:12:40.524 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-14T12:12:40.549 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-14T12:12:40.575 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-14T12:12:40.600 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-14T12:12:40.606 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-14T12:12:40.606 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-14T12:12:40.795 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-14T12:12:40.795 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-14T12:12:40.877 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-14T12:12:40.902 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-14T12:12:40.927 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-14T12:12:40.960 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-14T12:12:41.003 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-14T12:12:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-14T12:12:45.903 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-14T12:12:45.903 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.0.0-2023.g71e6beb6.el9.x86_64 1/198 2024-03-14T12:12:45.903 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 2/198 2024-03-14T12:12:45.903 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 3/198 2024-03-14T12:12:45.903 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 4/198 2024-03-14T12:12:45.903 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2023.g71e6b 5/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 6/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 7/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 8/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 9/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 10/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 11/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.0.0-2023.g71e6beb6.el9.x86_64 12/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 13/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.0.0-2023.g71e6beb6.el9.x86_64 14/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.0.0-2023.g71e6beb6.el9.x86_ 15/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 16/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 17/198 2024-03-14T12:12:45.904 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.0.0-2023.g71e6beb6.el9.x86_6 18/198 2024-03-14T12:12:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86 19/198 2024-03-14T12:12:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 20/198 2024-03-14T12:12:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2023.g71e6beb6.el 21/198 2024-03-14T12:12:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.0.0-2023.g71e6beb6.el9. 22/198 2024-03-14T12:12:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.0.0-2023.g71e6beb6.el9.x86_6 23/198 2024-03-14T12:12:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.0.0-2023.g71e6beb6.el9.x86_64 24/198 2024-03-14T12:12:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 25/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 26/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 27/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 28/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 29/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2023.g71e6beb6. 30/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noa 31/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.n 32/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e 33/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2023.g71e6beb6.el 34/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 35/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2023.g71e6beb6.e 36/198 2024-03-14T12:12:45.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 37/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 38/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.0.0-2023.g71e6beb6.el9.noarch 39/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-14T12:12:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-14T12:12:45.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-14T12:12:45.925 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-14T12:12:45.926 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-14T12:12:45.927 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-14T12:12:45.928 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-14T12:12:45.929 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-14T12:12:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-14T12:12:45.931 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 151/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 153/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-14T12:12:45.932 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-14T12:12:45.933 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-14T12:12:45.934 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-14T12:12:45.935 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-14T12:12:45.935 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-14T12:12:45.935 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-14T12:12:45.935 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-14T12:12:45.935 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-14T12:12:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-14T12:12:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: acl-2.3.1-4.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-14T12:12:48.892 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.0.0-2023.g71e6beb6.el9.noarch 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-16.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-14T12:12:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-14T12:12:48.896 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.11-1.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-14T12:12:48.897 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-14T12:12:48.898 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-14T12:12:48.899 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-14T12:12:48.900 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-14T12:12:48.901 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-14T12:12:48.902 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.0.0-2023.g71e6beb6.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-3.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-14T12:12:48.903 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-14T12:12:48.904 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:48.904 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-14T12:12:49.414 DEBUG:teuthology.parallel:result is None 2024-03-14T12:12:49.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=71e6beb6de9ad34b7d44be0ccdf38721ae92202d 2024-03-14T12:12:49.559 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-14T12:12:49.580 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-2023.g71e6beb6.el9 2024-03-14T12:12:49.581 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2023.g71e6beb6.el9 2024-03-14T12:12:49.581 INFO:teuthology.task.install:The correct ceph version 19.0.0-2023.g71e6beb6 is installed. 2024-03-14T12:12:49.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=71e6beb6de9ad34b7d44be0ccdf38721ae92202d 2024-03-14T12:12:49.717 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-14T12:12:49.743 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-2023.g71e6beb6.el9 2024-03-14T12:12:49.743 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2023.g71e6beb6.el9 2024-03-14T12:12:49.743 INFO:teuthology.task.install:The correct ceph version 19.0.0-2023.g71e6beb6 is installed. 2024-03-14T12:12:49.745 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-14T12:12:49.745 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:49.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-14T12:12:49.775 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:49.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-14T12:12:49.818 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-14T12:12:49.818 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:49.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-03-14T12:12:49.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-14T12:12:49.912 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:49.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-03-14T12:12:49.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-14T12:12:50.010 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-14T12:12:50.011 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:50.011 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-14T12:12:50.037 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-14T12:12:50.104 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:50.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-14T12:12:50.132 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-14T12:12:50.200 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-14T12:12:50.200 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:50.200 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-03-14T12:12:50.226 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-14T12:12:50.293 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:50.293 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-03-14T12:12:50.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-14T12:12:50.390 INFO:teuthology.run_tasks:Running task ceph... 2024-03-14T12:12:50.470 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-14T12:12:50.470 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /var/log/ceph 2024-03-14T12:12:50.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-03-14T12:12:50.500 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-14T12:12:50.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-14T12:12:50.540 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-14T12:12:50.567 INFO:tasks.ceph:Creating extra log directories... 2024-03-14T12:12:50.567 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-14T12:12:50.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-14T12:12:50.640 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-14T12:12:50.640 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-14T12:12:50.640 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-14T12:12:50.640 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:50.640 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-14T12:12:50.685 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-14T12:12:50.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-14T12:12:50.830 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:50.830 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-14T12:12:50.845 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-14T12:12:50.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-14T12:12:50.995 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-14T12:12:50.996 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'missing required features'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-14T12:12:50.996 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019', 'branch': 'main', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}', 'email': 'jcollin@redhat.com', 'first_in_suite': False, 'job_id': '7600019', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'missing required features'], 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'origin-jcollin-testing', 'sha1': 'ab5e0cba092761c4e4548739d8748ec40a6ffd68'}}, 'owner': 'scheduled_jcollin@teuthology', 'priority': 10, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8468, 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'origin-jcollin-testing', 'suite_path': '/home/teuthworker/src/github.com_joscollin_ceph_ab5e0cba092761c4e4548739d8748ec40a6ffd68/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/joscollin/ceph.git', 'suite_sha1': 'ab5e0cba092761c4e4548739d8748ec40a6ffd68', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi082.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtnm40UtVIa2GFsWgd9HtRYh6qFTSDq+2CKW1kvAiwvrH0JAREZrFSK3nki8uLh601My7RmfbPU0NlKowuAyZo=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFD15MB9iYeb6MrVg/H9dAD9sptnJFZ1tfewH5VGxfYwEJX0gSLWLl1rzNrxUSikF25Rg8xYpZjDfqYrDMW2F5Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'missing required features'], 'sha1': '71e6beb6de9ad34b7d44be0ccdf38721ae92202d', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_admin.TestLabeledPerfCounters']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-14_11:45:06', 'tube': 'smithi', 'user': 'jcollin', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-14T12:12:50.996 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-14T12:12:50.999 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-14T12:12:51.045 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/run/ceph 2024-03-14T12:12:51.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-03-14T12:12:51.113 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:51.113 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-03-14T12:12:51.134 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-14T12:12:51.134 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-14 12:12:41.412089293 +0000 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-14 12:10:23.564786490 +0000 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-14 12:10:23.564786490 +0000 2024-03-14T12:12:51.188 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-14 12:10:23.564786490 +0000 2024-03-14T12:12:51.189 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-14T12:12:51.253 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-14T12:12:51.253 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-14T12:12:51.253 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000304188 s, 1.7 MB/s 2024-03-14T12:12:51.254 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-14T12:12:51.310 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-03-14T12:12:51.364 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-14T12:12:51.364 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:51.364 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-14T12:12:51.364 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:51.364 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:51.364 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-14 12:12:41.412089293 +0000 2024-03-14T12:12:51.364 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-14 12:10:23.905780156 +0000 2024-03-14T12:12:51.365 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-14 12:10:23.905780156 +0000 2024-03-14T12:12:51.365 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-14 12:10:23.905780156 +0000 2024-03-14T12:12:51.365 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-14T12:12:51.430 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-14T12:12:51.430 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-14T12:12:51.430 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000199562 s, 2.6 MB/s 2024-03-14T12:12:51.431 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-14T12:12:51.487 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-03-14T12:12:51.541 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-14T12:12:51.541 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:51.541 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-14T12:12:51.541 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:51.541 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:51.541 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-14 12:12:41.412089293 +0000 2024-03-14T12:12:51.542 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-14 12:10:24.244773858 +0000 2024-03-14T12:12:51.542 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-14 12:10:24.244773858 +0000 2024-03-14T12:12:51.542 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-14 12:10:24.244773858 +0000 2024-03-14T12:12:51.542 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-14T12:12:51.606 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-14T12:12:51.606 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-14T12:12:51.607 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000172005 s, 3.0 MB/s 2024-03-14T12:12:51.608 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-14T12:12:51.663 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-14 12:12:41.413089274 +0000 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-14 12:10:24.598767282 +0000 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-14 12:10:24.598767282 +0000 2024-03-14T12:12:51.718 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-14 12:10:24.598767282 +0000 2024-03-14T12:12:51.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-14T12:12:51.783 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-14T12:12:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-14T12:12:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000155586 s, 3.3 MB/s 2024-03-14T12:12:51.785 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-14T12:12:51.840 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T12:12:51.840 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:51.840 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-03-14T12:12:51.856 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-14T12:12:51.857 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:12:20.457921987 +0000 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:10:23.561306727 +0000 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:10:23.561306727 +0000 2024-03-14T12:12:51.913 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:10:23.561306727 +0000 2024-03-14T12:12:51.913 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-14T12:12:51.980 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-14T12:12:51.980 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-14T12:12:51.980 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000284182 s, 1.8 MB/s 2024-03-14T12:12:51.981 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-14T12:12:52.038 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-03-14T12:12:52.094 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-14T12:12:52.094 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:52.095 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-14T12:12:52.095 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:52.095 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:52.095 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:12:20.457921987 +0000 2024-03-14T12:12:52.095 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:10:23.889301078 +0000 2024-03-14T12:12:52.095 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:10:23.889301078 +0000 2024-03-14T12:12:52.095 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:10:23.889301078 +0000 2024-03-14T12:12:52.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-14T12:12:52.161 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-14T12:12:52.161 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-14T12:12:52.161 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000211966 s, 2.4 MB/s 2024-03-14T12:12:52.162 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-14T12:12:52.218 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-03-14T12:12:52.274 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-14T12:12:52.274 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:52.274 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-14T12:12:52.275 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:52.275 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:52.275 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:12:20.457921987 +0000 2024-03-14T12:12:52.275 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:10:24.223295324 +0000 2024-03-14T12:12:52.275 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:10:24.223295324 +0000 2024-03-14T12:12:52.275 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:10:24.223295324 +0000 2024-03-14T12:12:52.275 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-14T12:12:52.340 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-14T12:12:52.340 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-14T12:12:52.341 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000287959 s, 1.8 MB/s 2024-03-14T12:12:52.342 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-14T12:12:52.397 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:12:20.458921970 +0000 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:10:24.558289554 +0000 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:10:24.558289554 +0000 2024-03-14T12:12:52.452 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:10:24.558289554 +0000 2024-03-14T12:12:52.453 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-14T12:12:52.516 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-14T12:12:52.517 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-14T12:12:52.517 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00020318 s, 2.5 MB/s 2024-03-14T12:12:52.518 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-14T12:12:52.573 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T12:12:52.573 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-14T12:12:52.573 INFO:tasks.ceph:Generating config... 2024-03-14T12:12:52.575 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-14T12:12:52.575 INFO:tasks.ceph:[client] debug client = 20 2024-03-14T12:12:52.575 INFO:tasks.ceph:[client] debug ms = 1 2024-03-14T12:12:52.575 INFO:tasks.ceph:[client] pid file = 2024-03-14T12:12:52.575 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-14T12:12:52.575 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-14T12:12:52.575 INFO:tasks.ceph:[global] lockdep = True 2024-03-14T12:12:52.575 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-14T12:12:52.575 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-14T12:12:52.575 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-14T12:12:52.576 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-14T12:12:52.577 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-14T12:12:52.578 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-14T12:12:52.578 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-14T12:12:52.578 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-14T12:12:52.578 INFO:tasks.ceph:Setting up mon.a... 2024-03-14T12:12:52.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-14T12:12:52.626 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.keyring 2024-03-14T12:12:52.629 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-14T12:12:52.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T12:12:52.700 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.82'), ('mon.b', '172.21.15.164'), ('mon.c', '[v2:172.21.15.164:3301,v1:172.21.15.164:6790]')] 2024-03-14T12:12:52.700 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.82,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]', 'lockdep': True, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-14T12:12:52.701 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:52.701 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-14T12:12:52.754 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.82 --add b 172.21.15.164 --addv c '[v2:172.21.15.164:3301,v1:172.21.15.164:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: generated fsid 30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = quincy 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 0 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:last_changed 2024-03-14T12:12:52.840868+0000 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-03-14T12:12:52.840868+0000 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:min_mon_release 17 (quincy) 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:election_strategy: 1 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:2: [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] mon.c 2024-03-14T12:12:52.839 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-14T12:12:52.840 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-14T12:12:52.854 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2... 2024-03-14T12:12:52.855 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-14T12:12:52.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-14T12:12:52.944 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-03-14T12:12:52.944 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-03-14T12:12:52.944 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-14T12:12:52.944 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.945 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-03-14T12:12:52.946 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.947 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.82,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]" 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: lockdep = True 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-03-14T12:12:52.948 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-03-14T12:12:52.949 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: bdev async discard = True 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: bdev enable discard = True 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore allocator = bitmap 2024-03-14T12:12:52.950 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 1/20 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 1/20 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 4/10 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-03-14T12:12:52.951 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-03-14T12:12:52.952 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-14T12:12:52.953 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:12:52.954 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-03-14T12:12:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-03-14T12:12:52.956 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-03-14T12:12:52.957 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-03-14T12:12:52.958 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-03-14T12:12:52.959 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-03-14T12:12:52.960 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.82,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]" 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: lockdep = True 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on skipped message = False 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.961 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.962 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: bdev async discard = True 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: bdev enable discard = True 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore allocator = bitmap 2024-03-14T12:12:52.963 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-03-14T12:12:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-14T12:12:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-03-14T12:12:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-03-14T12:12:52.967 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = "" 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-03-14T12:12:52.968 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds balancer = 20 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2024-03-14T12:12:52.969 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2024-03-14T12:12:52.970 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-03-14T12:12:52.970 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-03-14T12:12:52.970 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-03-14T12:12:52.971 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-14T12:12:52.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-14T12:12:53.067 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-14T12:12:53.068 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:53.068 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-14T12:12:53.083 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:53.083 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-14T12:12:53.137 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:12:53.137 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:53.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-14T12:12:53.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T12:12:53.215 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:12:53.215 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:12:53.270 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:12:53.270 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:53.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-14T12:12:53.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T12:12:53.310 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:12:53.310 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:12:53.364 INFO:tasks.ceph:Setting up mon nodes... 2024-03-14T12:12:53.364 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-14T12:12:53.365 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-14T12:12:53.418 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-14T12:12:53.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-14T12:12:53.486 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-14T12:12:53.488 INFO:tasks.ceph:Setting up mds nodes... 2024-03-14T12:12:53.488 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-14T12:12:53.544 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-14T12:12:53.546 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-14T12:12:53.573 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-14T12:12:53.666 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-14T12:12:53.669 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-14T12:12:53.695 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-14T12:12:53.749 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-14T12:12:53.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-14T12:12:53.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-14T12:12:53.873 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-14T12:12:53.876 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-14T12:12:53.902 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-14T12:12:53.902 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-14T12:12:53.944 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-14T12:12:53.959 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-14T12:12:54.041 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-14T12:12:54.056 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-14T12:12:54.098 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-14T12:12:54.113 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-14T12:12:54.196 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-14T12:12:54.211 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-14T12:12:54.211 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-14T12:12:54.211 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-14T12:12:54.239 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T12:12:54.239 INFO:tasks.ceph:role: osd.0 2024-03-14T12:12:54.239 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:12:54.239 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-14T12:12:54.312 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:12:54.312 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:12:54.312 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:12:54.312 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:12:54.313 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:12:54.313 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:12:54.313 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:12:54.313 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:12:54.313 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:12:54.313 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:12:54.826 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-14T12:12:54.827 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-14T12:12:54.828 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-14T12:12:54.963 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-14T12:12:54.998 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-14T12:12:55.065 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T12:12:55.066 INFO:tasks.ceph:role: osd.1 2024-03-14T12:12:55.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:12:55.066 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-14T12:12:55.139 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:12:55.139 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:12:55.140 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:12:55.602 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-14T12:12:55.604 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-14T12:12:55.604 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-14T12:12:55.698 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-14T12:12:55.729 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-14T12:12:55.797 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T12:12:55.797 INFO:tasks.ceph:role: osd.2 2024-03-14T12:12:55.797 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:12:55.797 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-14T12:12:55.870 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:12:55.870 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:12:55.870 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:12:55.871 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:12:55.871 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:12:55.871 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:12:55.871 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:12:55.871 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:12:55.871 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:12:55.871 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:12:56.364 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-14T12:12:56.366 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-14T12:12:56.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-14T12:12:56.524 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-14T12:12:56.554 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-14T12:12:56.622 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T12:12:56.622 INFO:tasks.ceph:role: osd.3 2024-03-14T12:12:56.623 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:12:56.623 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-14T12:12:56.697 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:12:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:12:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:12:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:12:56.698 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:12:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:12:56.698 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:12:56.698 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:12:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:12:56.698 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:12:57.210 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-14T12:12:57.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-14T12:12:57.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-14T12:12:57.339 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-14T12:12:57.375 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:12:57.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:12:57.472+0000 7f727a80f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-14T12:12:57.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:12:57.472+0000 7f727a80f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-14T12:12:57.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:12:57.472+0000 7f727a80f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T12:12:57.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:12:57.472+0000 7f727a80f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T12:12:57.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:12:57.472+0000 7f727a80f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T12:12:57.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:12:57.473+0000 7f727a80f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-14T12:13:00.324 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-14T12:13:00.354 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:00.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:00.446+0000 7f44ef68b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-14T12:13:00.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:00.446+0000 7f44ef68b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-14T12:13:00.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:00.446+0000 7f44ef68b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T12:13:00.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:00.446+0000 7f44ef68b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T12:13:00.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:00.446+0000 7f44ef68b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T12:13:00.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:00.447+0000 7f44ef68b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-14T12:13:03.297 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-14T12:13:03.328 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:03.419+0000 7f659320d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-14T12:13:03.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:03.419+0000 7f659320d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-14T12:13:03.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:03.419+0000 7f659320d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T12:13:03.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:03.419+0000 7f659320d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T12:13:03.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:03.419+0000 7f659320d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T12:13:03.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:03.420+0000 7f659320d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-14T12:13:06.281 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-14T12:13:06.312 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:06.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:06.405+0000 7f276180f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-14T12:13:06.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:06.406+0000 7f276180f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-14T12:13:06.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:06.406+0000 7f276180f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T12:13:06.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:06.406+0000 7f276180f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T12:13:06.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:06.406+0000 7f276180f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T12:13:06.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:06.406+0000 7f276180f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-14T12:13:09.263 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-14T12:13:09.294 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-14T12:13:09.326 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T12:13:09.326 INFO:tasks.ceph:role: osd.4 2024-03-14T12:13:09.326 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:13:09.326 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-14T12:13:09.404 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:13:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:13:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:13:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:13:09.404 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:13:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:13:09.404 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:13:09.405 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:13:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:13:09.405 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:13:09.901 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-14T12:13:09.903 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-14T12:13:09.904 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-14T12:13:10.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-14T12:13:10.068 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-14T12:13:10.137 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T12:13:10.137 INFO:tasks.ceph:role: osd.5 2024-03-14T12:13:10.137 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:13:10.137 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:13:10.213 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:13:10.672 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-14T12:13:10.674 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-14T12:13:10.675 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-14T12:13:10.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-14T12:13:10.807 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-14T12:13:10.875 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T12:13:10.875 INFO:tasks.ceph:role: osd.6 2024-03-14T12:13:10.875 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:13:10.875 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:13:10.949 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:13:11.431 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-14T12:13:11.433 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-14T12:13:11.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-14T12:13:11.567 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-14T12:13:11.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-14T12:13:11.665 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T12:13:11.665 INFO:tasks.ceph:role: osd.7 2024-03-14T12:13:11.666 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:13:11.666 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T12:13:11.740 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T12:13:11.741 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T12:13:11.741 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T12:13:12.245 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-14T12:13:12.246 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-14T12:13:12.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-14T12:13:12.391 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-14T12:13:12.423 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:12.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:12.520+0000 7fbc944f9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-14T12:13:12.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:12.520+0000 7fbc944f9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-14T12:13:12.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:12.520+0000 7fbc944f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T12:13:12.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:12.520+0000 7fbc944f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T12:13:12.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:12.520+0000 7fbc944f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T12:13:12.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:12.521+0000 7fbc944f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-14T12:13:15.365 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-14T12:13:15.396 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:15.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:15.488+0000 7f8f2a9fa640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-14T12:13:15.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:15.489+0000 7f8f2a9fa640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-14T12:13:15.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:15.489+0000 7f8f2a9fa640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T12:13:15.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:15.489+0000 7f8f2a9fa640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T12:13:15.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:15.489+0000 7f8f2a9fa640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T12:13:15.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:15.489+0000 7f8f2a9fa640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-14T12:13:18.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-14T12:13:18.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:18.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:18.463+0000 7fbe3d411640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-14T12:13:18.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:18.463+0000 7fbe3d411640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-14T12:13:18.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:18.463+0000 7fbe3d411640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T12:13:18.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:18.463+0000 7fbe3d411640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T12:13:18.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:18.463+0000 7fbe3d411640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T12:13:18.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:18.463+0000 7fbe3d411640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-14T12:13:20.997 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:13:21.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:13:21.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-14T12:13:21.352 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:21.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:21.447+0000 7f96712d5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-14T12:13:21.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:21.447+0000 7f96712d5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-14T12:13:21.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:21.447+0000 7f96712d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T12:13:21.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:21.447+0000 7f96712d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T12:13:21.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:21.447+0000 7f96712d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T12:13:21.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:21.447+0000 7f96712d5640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-14T12:13:24.320 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-14T12:13:24.350 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-14T12:13:24.350 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:24.350 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-14T12:13:24.382 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:24.382 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-14T12:13:24.452 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:24.452 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-14T12:13:24.520 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:24.520 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-14T12:13:24.589 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:24.589 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-14T12:13:24.658 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:24.658 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-14T12:13:24.727 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:24.727 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-14T12:13:24.793 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:24.793 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-14T12:13:24.825 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:24.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-14T12:13:24.895 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:24.896 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-14T12:13:24.965 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:24.965 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-14T12:13:25.035 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:25.035 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-14T12:13:25.103 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:25.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-14T12:13:25.170 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:25.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-14T12:13:25.238 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:25.238 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-14T12:13:25.255 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:25.255 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-14T12:13:25.313 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:25.313 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-14T12:13:25.328 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:25.328 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-14T12:13:25.383 INFO:tasks.ceph:Adding keys to all mons... 2024-03-14T12:13:25.383 DEBUG:teuthology.orchestra.run.smithi082:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-14T12:13:25.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-14T12:13:25.426 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.y] 2024-03-14T12:13:25.426 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlxtwQGRAArcSgLFwgKjmSqjq0sst8Gw== 2024-03-14T12:13:25.426 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-03-14T12:13:25.426 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlAQmRIBAA45IMoIGH+bLaSIro/QUEmQ== 2024-03-14T12:13:25.426 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlt7bdJxAAH34bGUjVrlElC4nA11eNIQ== 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.0] 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDJ6fJlKwo/HBAA/Ma3FBjIdO1LF3RySRD0DQ== 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.1] 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDM6fJlot+1GhAAEQqFqku2rdbqZ3rEBfVO2g== 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.2] 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDP6fJlPRIYGRAAUxqEbSYn6AhIE+3lXEqEOg== 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.3] 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDS6fJlEYhJGBAApIl3qciLHwIJzDq5J2NkqA== 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.x] 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJl5qEWHRAAI8LqGdvUwjHG/5pNgcAB1A== 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJljqfOLBAAulbwLxzEv+lHJOPUy3p4BQ== 2024-03-14T12:13:25.427 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlmeIuNBAAzNKyG5CdCVCURsWZywpTnw== 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.4] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDY6fJlL2cQHxAAeHY/pdO0TvpUciB22xvOUg== 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.5] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDb6fJlrNxAHRAALwT6cIpCh9fE4NMoqV6PWA== 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.6] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDe6fJlPLi2GxAAeZ/Nk/340P6Y0ZJYojfNMw== 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.7] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDh6fJlLhW0GhAAtvxrvm93pbGKTJbzzpIRiQ== 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg== 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:[client.2] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDG6fJlDQeZAhAAJW1K+HBJuwgzu4nVr8Mq8g== 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:[client.1] 2024-03-14T12:13:25.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDG6fJlUlkBBhAA9QGPbDVg8uWX5WTR2W936g== 2024-03-14T12:13:25.429 INFO:teuthology.orchestra.run.smithi082.stdout:[client.3] 2024-03-14T12:13:25.429 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDG6fJldFHSCxAAci5DVZh9406uu1BZIAZXiw== 2024-03-14T12:13:25.448 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-03-14T12:13:25.448 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDF6fJlxtwQGRAArcSgLFwgKjmSqjq0sst8Gw== 2024-03-14T12:13:25.448 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-03-14T12:13:25.448 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDF6fJlAQmRIBAA45IMoIGH+bLaSIro/QUEmQ== 2024-03-14T12:13:25.448 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-03-14T12:13:25.448 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDF6fJlt7bdJxAAH34bGUjVrlElC4nA11eNIQ== 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDJ6fJlKwo/HBAA/Ma3FBjIdO1LF3RySRD0DQ== 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDM6fJlot+1GhAAEQqFqku2rdbqZ3rEBfVO2g== 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDP6fJlPRIYGRAAUxqEbSYn6AhIE+3lXEqEOg== 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDS6fJlEYhJGBAApIl3qciLHwIJzDq5J2NkqA== 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDF6fJl5qEWHRAAI8LqGdvUwjHG/5pNgcAB1A== 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDF6fJljqfOLBAAulbwLxzEv+lHJOPUy3p4BQ== 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-03-14T12:13:25.449 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDF6fJlmeIuNBAAzNKyG5CdCVCURsWZywpTnw== 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDY6fJlL2cQHxAAeHY/pdO0TvpUciB22xvOUg== 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDb6fJlrNxAHRAALwT6cIpCh9fE4NMoqV6PWA== 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDe6fJlPLi2GxAAeZ/Nk/340P6Y0ZJYojfNMw== 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDh6fJlLhW0GhAAtvxrvm93pbGKTJbzzpIRiQ== 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg== 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:[client.2] 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDG6fJlDQeZAhAAJW1K+HBJuwgzu4nVr8Mq8g== 2024-03-14T12:13:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2024-03-14T12:13:25.451 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDG6fJlUlkBBhAA9QGPbDVg8uWX5WTR2W936g== 2024-03-14T12:13:25.451 INFO:teuthology.orchestra.run.smithi164.stdout:[client.3] 2024-03-14T12:13:25.451 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDG6fJldFHSCxAAci5DVZh9406uu1BZIAZXiw== 2024-03-14T12:13:25.452 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T12:13:25.472 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T12:13:25.544 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:25.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:25.601 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:25.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:25.658 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.712 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.714 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.766 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.768 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.823 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:25.875 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T12:13:25.877 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T12:13:25.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:25.931 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:25.985 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:25.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T12:13:26.042 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.096 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.149 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.151 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.203 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.205 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T12:13:26.257 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.259 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.314 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.370 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.373 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.427 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T12:13:26.484 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-14T12:13:26.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-14T12:13:26.512 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T12:13:26.609 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-14T12:13:26.636 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-14T12:13:26.663 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T12:13:26.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-14T12:13:26.785 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-14T12:13:26.851 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T12:13:26.948 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-14T12:13:26.975 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:26.977 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-14T12:13:27.029 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-14T12:13:27.029 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-14T12:13:27.029 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-14T12:13:27.032 INFO:tasks.ceph.mon.a:Started 2024-03-14T12:13:27.032 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-14T12:13:27.032 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-14T12:13:27.071 INFO:tasks.ceph.mon.b:Started 2024-03-14T12:13:27.071 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-14T12:13:27.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-14T12:13:27.074 INFO:tasks.ceph.mon.c:Started 2024-03-14T12:13:27.074 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-14T12:13:27.074 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-14T12:13:27.074 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-14T12:13:27.076 INFO:tasks.ceph.mgr.y:Started 2024-03-14T12:13:27.076 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-14T12:13:27.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-14T12:13:27.078 INFO:tasks.ceph.mgr.x:Started 2024-03-14T12:13:27.078 INFO:tasks.ceph:Setting crush tunables to default 2024-03-14T12:13:27.078 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd crush tunables default 2024-03-14T12:13:27.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.180+0000 7f1a0f844640 1 Processor -- start 2024-03-14T12:13:27.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0f844640 1 -- start start 2024-03-14T12:13:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0f844640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 0x7f1a080af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0f844640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 0x7f1a080afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0f844640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0815e500 con 0x7f1a08158290 2024-03-14T12:13:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0f844640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a080b03a0 con 0x7f1a080d1510 2024-03-14T12:13:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0f844640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0815e7a0 con 0x7f1a08158670 2024-03-14T12:13:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0cddf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 0x7f1a080af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0cddf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 0x7f1a080af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34800/0 (socket says 172.21.15.82:34800) 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.181+0000 7f1a0cddf640 1 -- 172.21.15.82:0/1679954392 learned_addr learned my addr 172.21.15.82:0/1679954392 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0cddf640 1 -- 172.21.15.82:0/1679954392 >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 msgr2=0x7f1a080af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0cddf640 1 -- 172.21.15.82:0/1679954392 >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 msgr2=0x7f1a080af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0cddf640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 0x7f1a080af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0cddf640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 0x7f1a080af920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0dde1640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 0x7f1a080afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0dde1640 1 -- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 msgr2=0x7f1a080afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0dde1640 1 -- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 msgr2=0x7f1a080afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0dde1640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 0x7f1a080afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-14T12:13:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.182+0000 7f1a0dde1640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 0x7f1a080afe60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-14T12:13:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.383+0000 7f1a0cddf640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.82:3300/0 conn(0x7f1a080d1510 0x7f1a080af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.383+0000 7f1a0dde1640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 0x7f1a080afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.383+0000 7f1a0cddf640 1 -- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 msgr2=0x7f1a080afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.383+0000 7f1a0cddf640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 0x7f1a080afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.383+0000 7f1a0cddf640 1 -- 172.21.15.82:0/1679954392 >> v1:172.21.15.164:6789/0 conn(0x7f1a08158290 legacy=0x7f1a080d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.383+0000 7f1a0cddf640 1 -- 172.21.15.82:0/1679954392 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0815ea80 con 0x7f1a080d1510 2024-03-14T12:13:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.384+0000 7f1a0cddf640 1 --2- 172.21.15.82:0/1679954392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a080d1510 0x7f1a080af920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1a080b0580 tx=0x7f1a0402bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7d15f6b268e10c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.384+0000 7f19fffff640 1 -- 172.21.15.82:0/1679954392 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04039070 con 0x7f1a080d1510 2024-03-14T12:13:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.384+0000 7f19fffff640 1 -- 172.21.15.82:0/1679954392 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0402ca20 con 0x7f1a080d1510 2024-03-14T12:13:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.384+0000 7f19fffff640 1 -- 172.21.15.82:0/1679954392 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400f040 con 0x7f1a080d1510 2024-03-14T12:13:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 -- 172.21.15.82:0/1679954392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a080d1510 msgr2=0x7f1a080af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 --2- 172.21.15.82:0/1679954392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a080d1510 0x7f1a080af920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1a080b0580 tx=0x7f1a0402bc70 comp rx=0 tx=0).stop 2024-03-14T12:13:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 -- 172.21.15.82:0/1679954392 shutdown_connections 2024-03-14T12:13:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 --2- 172.21.15.82:0/1679954392 >> v2:172.21.15.164:3300/0 conn(0x7f1a08158670 0x7f1a080afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 --2- 172.21.15.82:0/1679954392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a080d1510 0x7f1a080af920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 -- 172.21.15.82:0/1679954392 >> 172.21.15.82:0/1679954392 conn(0x7f1a0806c000 msgr2=0x7f1a0806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 -- 172.21.15.82:0/1679954392 shutdown_connections 2024-03-14T12:13:27.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.385+0000 7f1a0f844640 1 -- 172.21.15.82:0/1679954392 wait complete. 2024-03-14T12:13:27.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 Processor -- start 2024-03-14T12:13:27.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 -- start start 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a080d1510 0x7f1a080cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a08158290 0x7f1a080ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a08158670 0x7f1a081444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0806d710 con 0x7f1a08158670 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0815f040 con 0x7f1a08158290 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.387+0000 7f1a0f844640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0806d890 con 0x7f1a080d1510 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a080d1510 0x7f1a080cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0dde1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a08158670 0x7f1a081444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0cddf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a08158290 0x7f1a080ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a080d1510 0x7f1a080cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:58900/0 (socket says 172.21.15.82:58900) 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 -- 172.21.15.82:0/276833209 learned_addr learned my addr 172.21.15.82:0/276833209 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 -- 172.21.15.82:0/276833209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a08158290 msgr2=0x7f1a080ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a08158290 0x7f1a080ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 -- 172.21.15.82:0/276833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a08158670 msgr2=0x7f1a081444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a08158670 0x7f1a081444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 -- 172.21.15.82:0/276833209 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08145010 con 0x7f1a080d1510 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0dde1640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a08158670 0x7f1a081444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.388+0000 7f1a0d5e0640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a080d1510 0x7f1a080cd110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c400 tx=0x7f19f800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.389+0000 7f19fdffb640 1 -- 172.21.15.82:0/276833209 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80052d0 con 0x7f1a080d1510 2024-03-14T12:13:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.389+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a081452f0 con 0x7f1a080d1510 2024-03-14T12:13:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.389+0000 7f19fdffb640 1 -- 172.21.15.82:0/276833209 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f8005490 con 0x7f1a080d1510 2024-03-14T12:13:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.389+0000 7f19fdffb640 1 -- 172.21.15.82:0/276833209 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8007260 con 0x7f1a080d1510 2024-03-14T12:13:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.390+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a081458b0 con 0x7f1a080d1510 2024-03-14T12:13:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.391+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0804f050 con 0x7f1a080d1510 2024-03-14T12:13:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.395+0000 7f19fdffb640 1 -- 172.21.15.82:0/276833209 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f19f8007400 con 0x7f1a080d1510 2024-03-14T12:13:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.395+0000 7f19fdffb640 1 -- 172.21.15.82:0/276833209 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f19f8003b50 con 0x7f1a080d1510 2024-03-14T12:13:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.398+0000 7f19fdffb640 1 -- 172.21.15.82:0/276833209 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f19f80035f0 con 0x7f1a080d1510 2024-03-14T12:13:27.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.465+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1a080cf280 con 0x7f1a080d1510 2024-03-14T12:13:27.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.468+0000 7f19fdffb640 1 -- 172.21.15.82:0/276833209 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f19f8003d50 con 0x7f1a080d1510 2024-03-14T12:13:27.467 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.469+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a080d1510 msgr2=0x7f1a080cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.469+0000 7f1a0f844640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a080d1510 0x7f1a080cd110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c400 tx=0x7f19f800c8c0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.469+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 shutdown_connections 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.469+0000 7f1a0f844640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a08158670 0x7f1a081444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.469+0000 7f1a0f844640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a08158290 0x7f1a080ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.469+0000 7f1a0f844640 1 --2- 172.21.15.82:0/276833209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a080d1510 0x7f1a080cd110 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.469+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 >> 172.21.15.82:0/276833209 conn(0x7f1a0806c000 msgr2=0x7f1a080b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.470+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 shutdown_connections 2024-03-14T12:13:27.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:27.470+0000 7f1a0f844640 1 -- 172.21.15.82:0/276833209 wait complete. 2024-03-14T12:13:27.479 INFO:tasks.ceph:check_enable_crimson: False 2024-03-14T12:13:27.480 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-14T12:13:27.480 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:27.480 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-14T12:13:27.506 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:27.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-14T12:13:27.573 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:27.573 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-14T12:13:27.640 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:13:27.640 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-14T12:13:27.707 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:27.707 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-14T12:13:27.735 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:27.735 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-14T12:13:27.803 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:27.803 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-14T12:13:27.840 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:27.841+0000 7f1c6263a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-14T12:13:27.871 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:13:27.871 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-14T12:13:27.880 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:27.881+0000 7f1cd4e49040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-14T12:13:27.893 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:27.894+0000 7f1c6263a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-14T12:13:27.906 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new cbe5962a-eef4-46b1-8a89-9d92797b47f7 0 2024-03-14T12:13:27.937 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:27.938+0000 7f1cd4e49040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-14T12:13:28.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.039+0000 7f65e064b640 1 Processor -- start 2024-03-14T12:13:28.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.039+0000 7f65e064b640 1 -- start start 2024-03-14T12:13:28.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65e064b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d806ad10 0x7f65d806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65e064b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65d815cb20 0x7f65d80cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65e064b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d806b630 con 0x7f65d8072530 2024-03-14T12:13:28.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65e064b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d815d140 con 0x7f65d815cb20 2024-03-14T12:13:28.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65e064b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8071a50 con 0x7f65d806ad10 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65de3e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d806ad10 0x7f65d806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65de3e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d806ad10 0x7f65d806b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54298/0 (socket says 172.21.15.164:54298) 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65ddbe6640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f65d8072530 0x7f65d815c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42730/0 (socket says 172.21.15.164:42730) 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.040+0000 7f65de3e7640 1 -- 172.21.15.164:0/959865512 learned_addr learned my addr 172.21.15.164:0/959865512 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.041+0000 7f65debe8640 1 --2- 172.21.15.164:0/959865512 >> v2:172.21.15.82:3300/0 conn(0x7f65d815cb20 0x7f65d80cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.041+0000 7f65de3e7640 1 -- 172.21.15.164:0/959865512 >> v2:172.21.15.82:3300/0 conn(0x7f65d815cb20 msgr2=0x7f65d80cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.041+0000 7f65de3e7640 1 --2- 172.21.15.164:0/959865512 >> v2:172.21.15.82:3300/0 conn(0x7f65d815cb20 0x7f65d80cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.041+0000 7f65de3e7640 1 -- 172.21.15.164:0/959865512 >> v1:172.21.15.164:6789/0 conn(0x7f65d8072530 legacy=0x7f65d815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.041+0000 7f65de3e7640 1 -- 172.21.15.164:0/959865512 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8071d30 con 0x7f65d806ad10 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.041+0000 7f65dd3e5640 1 -- 172.21.15.164:0/959865512 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484517636 0 0) 0x7f65d806b630 con 0x7f65d8072530 2024-03-14T12:13:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.041+0000 7f65debe8640 1 --2- 172.21.15.164:0/959865512 >> v2:172.21.15.82:3300/0 conn(0x7f65d815cb20 0x7f65d80cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:28.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.042+0000 7f65de3e7640 1 --2- 172.21.15.164:0/959865512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d806ad10 0x7f65d806b0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009870 tx=0x7f65c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32464ce981ae6edd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.042+0000 7f65dd3e5640 1 -- 172.21.15.164:0/959865512 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8034070 con 0x7f65d806ad10 2024-03-14T12:13:28.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.042+0000 7f65dd3e5640 1 -- 172.21.15.164:0/959865512 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8030840 con 0x7f65d806ad10 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.042+0000 7f65dd3e5640 1 -- 172.21.15.164:0/959865512 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8030c10 con 0x7f65d806ad10 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 -- 172.21.15.164:0/959865512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d806ad10 msgr2=0x7f65d806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 --2- 172.21.15.164:0/959865512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d806ad10 0x7f65d806b0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009870 tx=0x7f65c802b980 comp rx=0 tx=0).stop 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 -- 172.21.15.164:0/959865512 shutdown_connections 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 --2- 172.21.15.164:0/959865512 >> v2:172.21.15.82:3300/0 conn(0x7f65d815cb20 0x7f65d80cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 --2- 172.21.15.164:0/959865512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d806ad10 0x7f65d806b0f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 -- 172.21.15.164:0/959865512 >> 172.21.15.164:0/959865512 conn(0x7f65d80d0b40 msgr2=0x7f65d80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 -- 172.21.15.164:0/959865512 shutdown_connections 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.043+0000 7f65e064b640 1 -- 172.21.15.164:0/959865512 wait complete. 2024-03-14T12:13:28.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.044+0000 7f65e064b640 1 Processor -- start 2024-03-14T12:13:28.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.044+0000 7f65e064b640 1 -- start start 2024-03-14T12:13:28.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65e064b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d806ad10 0x7f65d81e5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65e064b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65d8072530 0x7f65d806c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65e064b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d815cb20 0x7f65d813b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65e064b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d813f260 con 0x7f65d806ad10 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65e064b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d813f0e0 con 0x7f65d8072530 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65e064b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80cf7a0 con 0x7f65d815cb20 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d815cb20 0x7f65d813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65ddbe6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65d8072530 0x7f65d806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d815cb20 0x7f65d813b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54306/0 (socket says 172.21.15.164:54306) 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 -- 172.21.15.164:0/1986483800 learned_addr learned my addr 172.21.15.164:0/1986483800 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65de3e7640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d806ad10 0x7f65d81e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 -- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65d8072530 msgr2=0x7f65d806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65d8072530 0x7f65d806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 -- 172.21.15.164:0/1986483800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d806ad10 msgr2=0x7f65d81e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d806ad10 0x7f65d81e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 -- 172.21.15.164:0/1986483800 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8140a50 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65ddbe6640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65d8072530 0x7f65d806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65debe8640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d815cb20 0x7f65d813b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65d0007df0 tx=0x7f65d000d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.045+0000 7f65ceffd640 1 -- 172.21.15.164:0/1986483800 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d001a070 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.046+0000 7f65ceffd640 1 -- 172.21.15.164:0/1986483800 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0003990 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.046+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d8140d30 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.046+0000 7f65ceffd640 1 -- 172.21.15.164:0/1986483800 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0003d80 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.046+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d81412f0 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.046+0000 7f65ceffd640 1 -- 172.21.15.164:0/1986483800 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f65d001d070 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.046+0000 7f65ceffd640 1 -- 172.21.15.164:0/1986483800 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f65d001fbc0 con 0x7f65d815cb20 2024-03-14T12:13:28.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.046+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d81417e0 con 0x7f65d815cb20 2024-03-14T12:13:28.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.049+0000 7f65ceffd640 1 -- 172.21.15.164:0/1986483800 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f65d0024020 con 0x7f65d815cb20 2024-03-14T12:13:28.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.121+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cbe5962a-eef4-46b1-8a89-9d92797b47f7", "id": 0} v 0) v1 -- 0x7f65d804f050 con 0x7f65d815cb20 2024-03-14T12:13:28.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.124+0000 7f65ceffd640 1 -- 172.21.15.164:0/1986483800 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbe5962a-eef4-46b1-8a89-9d92797b47f7", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f65d0010070 con 0x7f65d815cb20 2024-03-14T12:13:28.125 INFO:teuthology.orchestra.run.smithi164.stdout:0 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d815cb20 msgr2=0x7f65d813b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d815cb20 0x7f65d813b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65d0007df0 tx=0x7f65d000d4a0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 shutdown_connections 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65d815cb20 0x7f65d813b530 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65d8072530 0x7f65d806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 --2- 172.21.15.164:0/1986483800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d806ad10 0x7f65d81e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 >> 172.21.15.164:0/1986483800 conn(0x7f65d80d0b40 msgr2=0x7f65d806eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 shutdown_connections 2024-03-14T12:13:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.125+0000 7f65e064b640 1 -- 172.21.15.164:0/1986483800 wait complete. 2024-03-14T12:13:28.135 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 8856df27-763b-47d7-853b-29410857ca0e 1 2024-03-14T12:13:28.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.236+0000 7f37f5636640 1 Processor -- start 2024-03-14T12:13:28.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.236+0000 7f37f5636640 1 -- start start 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37f5636640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f0072530 0x7f37f015c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37f5636640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37f015c950 0x7f37f00cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37f5636640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f015d140 con 0x7f37f006ad10 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37f5636640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f0071a50 con 0x7f37f015c950 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37f5636640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f0071d30 con 0x7f37f0072530 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f0072530 0x7f37f015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f0072530 0x7f37f015c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54314/0 (socket says 172.21.15.164:54314) 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37e7fff640 1 -- 172.21.15.164:0/2735530792 learned_addr learned my addr 172.21.15.164:0/2735530792 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.237+0000 7f37ef7fe640 1 --2- 172.21.15.164:0/2735530792 >> v2:172.21.15.82:3300/0 conn(0x7f37f015c950 0x7f37f00cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.238+0000 7f37e7fff640 1 -- 172.21.15.164:0/2735530792 >> v2:172.21.15.82:3300/0 conn(0x7f37f015c950 msgr2=0x7f37f00cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.238+0000 7f37e7fff640 1 --2- 172.21.15.164:0/2735530792 >> v2:172.21.15.82:3300/0 conn(0x7f37f015c950 0x7f37f00cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.238+0000 7f37e7fff640 1 -- 172.21.15.164:0/2735530792 >> v1:172.21.15.82:6789/0 conn(0x7f37f006ad10 legacy=0x7f37f006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.238+0000 7f37e7fff640 1 -- 172.21.15.164:0/2735530792 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f0072010 con 0x7f37f0072530 2024-03-14T12:13:28.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.238+0000 7f37e7fff640 1 --2- 172.21.15.164:0/2735530792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f0072530 0x7f37f015c410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009870 tx=0x7f37d802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2448e0c24814e698 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37ee7fc640 1 -- 172.21.15.164:0/2735530792 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8034070 con 0x7f37f0072530 2024-03-14T12:13:28.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37ee7fc640 1 -- 172.21.15.164:0/2735530792 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d8030840 con 0x7f37f0072530 2024-03-14T12:13:28.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37ee7fc640 1 -- 172.21.15.164:0/2735530792 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8030c10 con 0x7f37f0072530 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 -- 172.21.15.164:0/2735530792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f0072530 msgr2=0x7f37f015c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 --2- 172.21.15.164:0/2735530792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f0072530 0x7f37f015c410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009870 tx=0x7f37d802b980 comp rx=0 tx=0).stop 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 -- 172.21.15.164:0/2735530792 shutdown_connections 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 --2- 172.21.15.164:0/2735530792 >> v2:172.21.15.82:3300/0 conn(0x7f37f015c950 0x7f37f00cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 --2- 172.21.15.164:0/2735530792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f0072530 0x7f37f015c410 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 -- 172.21.15.164:0/2735530792 >> 172.21.15.164:0/2735530792 conn(0x7f37f00d0b40 msgr2=0x7f37f00d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 -- 172.21.15.164:0/2735530792 shutdown_connections 2024-03-14T12:13:28.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.239+0000 7f37f5636640 1 -- 172.21.15.164:0/2735530792 wait complete. 2024-03-14T12:13:28.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.240+0000 7f37f5636640 1 Processor -- start 2024-03-14T12:13:28.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37f5636640 1 -- start start 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37f5636640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f006ad10 0x7f37f015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37f5636640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f0072530 0x7f37f0151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37f5636640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f015c950 0x7f37f015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37f5636640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f013f260 con 0x7f37f006ad10 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37f5636640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f013f0e0 con 0x7f37f0072530 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37f5636640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00cf7a0 con 0x7f37f015c950 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f0072530 0x7f37f0151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37ef7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f015c950 0x7f37f015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37ef7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f015c950 0x7f37f015dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54316/0 (socket says 172.21.15.164:54316) 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f0072530 0x7f37f0151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50282/0 (socket says 172.21.15.164:50282) 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37ef7fe640 1 -- 172.21.15.164:0/4119420691 learned_addr learned my addr 172.21.15.164:0/4119420691 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.241+0000 7f37eeffd640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f006ad10 0x7f37f015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e7fff640 1 -- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f015c950 msgr2=0x7f37f015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e7fff640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f015c950 0x7f37f015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e7fff640 1 -- 172.21.15.164:0/4119420691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f006ad10 msgr2=0x7f37f015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e7fff640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f006ad10 0x7f37f015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e7fff640 1 -- 172.21.15.164:0/4119420691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00b5390 con 0x7f37f0072530 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37ef7fe640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f015c950 0x7f37f015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e7fff640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f0072530 0x7f37f0151a00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f37d8033000 tx=0x7f37d8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e77fe640 1 -- 172.21.15.164:0/4119420691 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8048070 con 0x7f37f0072530 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e77fe640 1 -- 172.21.15.164:0/4119420691 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d8030640 con 0x7f37f0072530 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e77fe640 1 -- 172.21.15.164:0/4119420691 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8034040 con 0x7f37f0072530 2024-03-14T12:13:28.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f015ab00 con 0x7f37f0072530 2024-03-14T12:13:28.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f015b5e0 con 0x7f37f0072530 2024-03-14T12:13:28.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e77fe640 1 -- 172.21.15.164:0/4119420691 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f37d804b070 con 0x7f37f0072530 2024-03-14T12:13:28.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.242+0000 7f37e77fe640 1 -- 172.21.15.164:0/4119420691 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f37d8043410 con 0x7f37f0072530 2024-03-14T12:13:28.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.243+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f004f050 con 0x7f37f0072530 2024-03-14T12:13:28.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.245+0000 7f37e77fe640 1 -- 172.21.15.164:0/4119420691 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f37d8052070 con 0x7f37f0072530 2024-03-14T12:13:28.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.317+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8856df27-763b-47d7-853b-29410857ca0e", "id": 1} v 0) v1 -- 0x7f37f00d1630 con 0x7f37f0072530 2024-03-14T12:13:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.322+0000 7f37e77fe640 1 -- 172.21.15.164:0/4119420691 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8856df27-763b-47d7-853b-29410857ca0e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f37d8030810 con 0x7f37f0072530 2024-03-14T12:13:28.322 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f0072530 msgr2=0x7f37f0151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f0072530 0x7f37f0151a00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f37d8033000 tx=0x7f37d8004730 comp rx=0 tx=0).stop 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 shutdown_connections 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37f015c950 0x7f37f015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37f0072530 0x7f37f0151a00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 --2- 172.21.15.164:0/4119420691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f006ad10 0x7f37f015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 >> 172.21.15.164:0/4119420691 conn(0x7f37f00d0b40 msgr2=0x7f37f00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 shutdown_connections 2024-03-14T12:13:28.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.323+0000 7f37f5636640 1 -- 172.21.15.164:0/4119420691 wait complete. 2024-03-14T12:13:28.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57 2 2024-03-14T12:13:28.428 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.429+0000 7f1c6263a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-14T12:13:28.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.435+0000 7fdec52b2640 1 Processor -- start 2024-03-14T12:13:28.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.435+0000 7fdec52b2640 1 -- start start 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdec52b2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdec01447c0 0x7fdec006f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdec52b2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdec00b3b30 0x7fdec006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdec52b2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec006f6d0 con 0x7fdec00b4bc0 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdec52b2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec006e0f0 con 0x7fdec00b3b30 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdec52b2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec006e3d0 con 0x7fdec01447c0 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdebeffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdec01447c0 0x7fdec006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdebf7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdec00b4bc0 0x7fdec00ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42744/0 (socket says 172.21.15.164:42744) 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdebf7fe640 1 -- 172.21.15.164:0/2788355343 learned_addr learned my addr 172.21.15.164:0/2788355343 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-14T12:13:28.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.436+0000 7fdeb7fff640 1 --2- 172.21.15.164:0/2788355343 >> v2:172.21.15.82:3300/0 conn(0x7fdec00b3b30 0x7fdec006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebeffd640 1 -- 172.21.15.164:0/2788355343 >> v2:172.21.15.82:3300/0 conn(0x7fdec00b3b30 msgr2=0x7fdec006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:28.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebeffd640 1 --2- 172.21.15.164:0/2788355343 >> v2:172.21.15.82:3300/0 conn(0x7fdec00b3b30 0x7fdec006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebeffd640 1 -- 172.21.15.164:0/2788355343 >> v1:172.21.15.164:6789/0 conn(0x7fdec00b4bc0 legacy=0x7fdec00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebeffd640 1 -- 172.21.15.164:0/2788355343 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec006e6b0 con 0x7fdec01447c0 2024-03-14T12:13:28.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebe7fc640 1 -- 172.21.15.164:0/2788355343 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376557801 0 0) 0x7fdec006f6d0 con 0x7fdec00b4bc0 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebeffd640 1 --2- 172.21.15.164:0/2788355343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec006f190 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdec0073110 tx=0x7fdeb002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6318bf6052f9da server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebe7fc640 1 -- 172.21.15.164:0/2788355343 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0035020 con 0x7fdec01447c0 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebe7fc640 1 -- 172.21.15.164:0/2788355343 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb0031830 con 0x7fdec01447c0 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.437+0000 7fdebe7fc640 1 -- 172.21.15.164:0/2788355343 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0031c00 con 0x7fdec01447c0 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 -- 172.21.15.164:0/2788355343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 msgr2=0x7fdec006f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 --2- 172.21.15.164:0/2788355343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec006f190 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdec0073110 tx=0x7fdeb002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 -- 172.21.15.164:0/2788355343 shutdown_connections 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 --2- 172.21.15.164:0/2788355343 >> v2:172.21.15.82:3300/0 conn(0x7fdec00b3b30 0x7fdec006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 --2- 172.21.15.164:0/2788355343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec006f190 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 -- 172.21.15.164:0/2788355343 >> 172.21.15.164:0/2788355343 conn(0x7fdec00d0b40 msgr2=0x7fdec00d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 -- 172.21.15.164:0/2788355343 shutdown_connections 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.438+0000 7fdec52b2640 1 -- 172.21.15.164:0/2788355343 wait complete. 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.439+0000 7fdec52b2640 1 Processor -- start 2024-03-14T12:13:28.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.439+0000 7fdec52b2640 1 -- start start 2024-03-14T12:13:28.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdec52b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00b3b30 0x7fdec00ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdec52b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b4bc0 0x7fdec01e0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdec52b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec013e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdec52b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec006dde0 con 0x7fdec00b4bc0 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdec52b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec006ec70 con 0x7fdec01447c0 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdec52b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00b5050 con 0x7fdec00b3b30 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00b3b30 0x7fdec00ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec013e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00b3b30 0x7fdec00ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54322/0 (socket says 172.21.15.164:54322) 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 -- 172.21.15.164:0/891785168 learned_addr learned my addr 172.21.15.164:0/891785168 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdeb7fff640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b4bc0 0x7fdec01e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 -- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 msgr2=0x7fdec013e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec013e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 -- 172.21.15.164:0/891785168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b4bc0 msgr2=0x7fdec01e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b4bc0 0x7fdec01e0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebeffd640 1 -- 172.21.15.164:0/891785168 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00ae160 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.440+0000 7fdebf7fe640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec013e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdebeffd640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00b3b30 0x7fdec00ada60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0000c00 tx=0x7fdeb00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdeb77fe640 1 -- 172.21.15.164:0/891785168 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb003a8b0 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdeb77fe640 1 -- 172.21.15.164:0/891785168 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb003aa70 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdeb77fe640 1 -- 172.21.15.164:0/891785168 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb003ae60 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec00ae440 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec00aea00 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdeb77fe640 1 -- 172.21.15.164:0/891785168 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdeb004b060 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.441+0000 7fdeb77fe640 1 -- 172.21.15.164:0/891785168 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fdeb0044900 con 0x7fdec00b3b30 2024-03-14T12:13:28.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.442+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec00c5aa0 con 0x7fdec00b3b30 2024-03-14T12:13:28.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.444+0000 7fdeb77fe640 1 -- 172.21.15.164:0/891785168 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdeb003f070 con 0x7fdec00b3b30 2024-03-14T12:13:28.482 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.483+0000 7f1c6263a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-14T12:13:28.515 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.516+0000 7f1cd4e49040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-14T12:13:28.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.517+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57", "id": 2} v 0) v1 -- 0x7fdec004f050 con 0x7fdec00b3b30 2024-03-14T12:13:28.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.520+0000 7fdeb77fe640 1 -- 172.21.15.164:0/891785168 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdeb0044be0 con 0x7fdec00b3b30 2024-03-14T12:13:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-03-14T12:13:28.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.520+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00b3b30 msgr2=0x7fdec00ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.520+0000 7fdec52b2640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00b3b30 0x7fdec00ada60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0000c00 tx=0x7fdeb00046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.521+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 shutdown_connections 2024-03-14T12:13:28.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.521+0000 7fdec52b2640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01447c0 0x7fdec013e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.521+0000 7fdec52b2640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b4bc0 0x7fdec01e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.521+0000 7fdec52b2640 1 --2- 172.21.15.164:0/891785168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00b3b30 0x7fdec00ada60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.521+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 >> 172.21.15.164:0/891785168 conn(0x7fdec00d0b40 msgr2=0x7fdec00ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.521+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 shutdown_connections 2024-03-14T12:13:28.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.521+0000 7fdec52b2640 1 -- 172.21.15.164:0/891785168 wait complete. 2024-03-14T12:13:28.527 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.528+0000 7f1c6263a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-14T12:13:28.530 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new e8b7f18c-6a7f-48b9-a7e4-481dc021cec8 3 2024-03-14T12:13:28.574 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.575+0000 7f1cd4e49040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-14T12:13:28.580 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.581+0000 7f1c6263a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-14T12:13:28.621 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.622+0000 7f1cd4e49040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-14T12:13:28.622 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.623+0000 7f1c6263a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-14T12:13:28.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.634+0000 7f20df012640 1 Processor -- start 2024-03-14T12:13:28.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.634+0000 7f20df012640 1 -- start start 2024-03-14T12:13:28.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20df012640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20d806b420 0x7f20d80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20df012640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f20d81477c0 0x7f20d806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20df012640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d8162a30 con 0x7f20d806f390 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20df012640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d806f770 con 0x7f20d806b420 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20df012640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d81470a0 con 0x7f20d81477c0 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20dd5af640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f20d81477c0 0x7f20d806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20dcdae640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f20d806f390 0x7f20d806ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42752/0 (socket says 172.21.15.164:42752) 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20dcdae640 1 -- 172.21.15.164:0/724119821 learned_addr learned my addr 172.21.15.164:0/724119821 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20d7fff640 1 --2- 172.21.15.164:0/724119821 >> v2:172.21.15.82:3300/0 conn(0x7f20d806b420 0x7f20d80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20dd5af640 1 -- 172.21.15.164:0/724119821 >> v2:172.21.15.82:3300/0 conn(0x7f20d806b420 msgr2=0x7f20d80b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20dd5af640 1 --2- 172.21.15.164:0/724119821 >> v2:172.21.15.82:3300/0 conn(0x7f20d806b420 0x7f20d80b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20dd5af640 1 -- 172.21.15.164:0/724119821 >> v1:172.21.15.164:6789/0 conn(0x7f20d806f390 legacy=0x7f20d806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20dd5af640 1 -- 172.21.15.164:0/724119821 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8161c70 con 0x7f20d81477c0 2024-03-14T12:13:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.635+0000 7f20d77fe640 1 -- 172.21.15.164:0/724119821 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653997039 0 0) 0x7f20d8162a30 con 0x7f20d806f390 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.636+0000 7f20dd5af640 1 --2- 172.21.15.164:0/724119821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d81477c0 0x7f20d806d8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f20d8074e30 tx=0x7f20d002be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d369c494bc2e7b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.636+0000 7f20d77fe640 1 -- 172.21.15.164:0/724119821 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0039070 con 0x7f20d81477c0 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.636+0000 7f20d77fe640 1 -- 172.21.15.164:0/724119821 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d002cbf0 con 0x7f20d81477c0 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.636+0000 7f20d77fe640 1 -- 172.21.15.164:0/724119821 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0034440 con 0x7f20d81477c0 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 -- 172.21.15.164:0/724119821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d81477c0 msgr2=0x7f20d806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 --2- 172.21.15.164:0/724119821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d81477c0 0x7f20d806d8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f20d8074e30 tx=0x7f20d002be40 comp rx=0 tx=0).stop 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 -- 172.21.15.164:0/724119821 shutdown_connections 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 --2- 172.21.15.164:0/724119821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d81477c0 0x7f20d806d8a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 --2- 172.21.15.164:0/724119821 >> v2:172.21.15.82:3300/0 conn(0x7f20d806b420 0x7f20d80b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 -- 172.21.15.164:0/724119821 >> 172.21.15.164:0/724119821 conn(0x7f20d80d0b40 msgr2=0x7f20d80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 -- 172.21.15.164:0/724119821 shutdown_connections 2024-03-14T12:13:28.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.637+0000 7f20df012640 1 -- 172.21.15.164:0/724119821 wait complete. 2024-03-14T12:13:28.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.638+0000 7f20df012640 1 Processor -- start 2024-03-14T12:13:28.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.638+0000 7f20df012640 1 -- start start 2024-03-14T12:13:28.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20df012640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20d806b420 0x7f20d80ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20df012640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d806f390 0x7f20d80add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20df012640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d81477c0 0x7f20d80ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20df012640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8162230 con 0x7f20d81477c0 2024-03-14T12:13:28.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20df012640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81475e0 con 0x7f20d806f390 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20df012640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8162510 con 0x7f20d806b420 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20d806b420 0x7f20d80ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d806f390 0x7f20d80add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20d806b420 0x7f20d80ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54324/0 (socket says 172.21.15.164:54324) 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 -- 172.21.15.164:0/2310778902 learned_addr learned my addr 172.21.15.164:0/2310778902 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dd5af640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d81477c0 0x7f20d80ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 -- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d806f390 msgr2=0x7f20d80add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d806f390 0x7f20d80add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 -- 172.21.15.164:0/2310778902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d81477c0 msgr2=0x7f20d80ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d81477c0 0x7f20d80ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 -- 172.21.15.164:0/2310778902 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d815d970 con 0x7f20d806b420 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20d7fff640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d806f390 0x7f20d80add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.639+0000 7f20dcdae640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20d806b420 0x7f20d80ad850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00c4d0 tx=0x7f20cc00c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.640+0000 7f20d57fa640 1 -- 172.21.15.164:0/2310778902 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc018070 con 0x7f20d806b420 2024-03-14T12:13:28.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.640+0000 7f20d57fa640 1 -- 172.21.15.164:0/2310778902 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc007500 con 0x7f20d806b420 2024-03-14T12:13:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.640+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d81e7900 con 0x7f20d806b420 2024-03-14T12:13:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.640+0000 7f20d57fa640 1 -- 172.21.15.164:0/2310778902 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc004a20 con 0x7f20d806b420 2024-03-14T12:13:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.640+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d81e7ec0 con 0x7f20d806b420 2024-03-14T12:13:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.640+0000 7f20d57fa640 1 -- 172.21.15.164:0/2310778902 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f20cc01b070 con 0x7f20d806b420 2024-03-14T12:13:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.640+0000 7f20d57fa640 1 -- 172.21.15.164:0/2310778902 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f20cc012890 con 0x7f20d806b420 2024-03-14T12:13:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.641+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d804f050 con 0x7f20d806b420 2024-03-14T12:13:28.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.643+0000 7f20d57fa640 1 -- 172.21.15.164:0/2310778902 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f20cc012b70 con 0x7f20d806b420 2024-03-14T12:13:28.680 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.681+0000 7f1cd4e49040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-14T12:13:28.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.714+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e8b7f18c-6a7f-48b9-a7e4-481dc021cec8", "id": 3} v 0) v1 -- 0x7f20d8072300 con 0x7f20d806b420 2024-03-14T12:13:28.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.717+0000 7f20d57fa640 1 -- 172.21.15.164:0/2310778902 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8b7f18c-6a7f-48b9-a7e4-481dc021cec8", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f20cc004bc0 con 0x7f20d806b420 2024-03-14T12:13:28.717 INFO:teuthology.orchestra.run.smithi164.stdout:3 2024-03-14T12:13:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20d806b420 msgr2=0x7f20d80ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20d806b420 0x7f20d80ad850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00c4d0 tx=0x7f20cc00c990 comp rx=0 tx=0).stop 2024-03-14T12:13:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 shutdown_connections 2024-03-14T12:13:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d81477c0 0x7f20d80ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d806f390 0x7f20d80add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 --2- 172.21.15.164:0/2310778902 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20d806b420 0x7f20d80ad850 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 >> 172.21.15.164:0/2310778902 conn(0x7f20d80d0b40 msgr2=0x7f20d806ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 shutdown_connections 2024-03-14T12:13:28.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.718+0000 7f20df012640 1 -- 172.21.15.164:0/2310778902 wait complete. 2024-03-14T12:13:28.724 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.725+0000 7f1cd4e49040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-14T12:13:28.728 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 00d8cc9a-7044-475d-a609-45a893e543c7 4 2024-03-14T12:13:28.739 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.740+0000 7f1c6263a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-14T12:13:28.807 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.808+0000 7f1c6263a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-14T12:13:28.846 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.847+0000 7f1cd4e49040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-14T12:13:28.853 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.854+0000 7f1c6263a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-14T12:13:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.871+0000 7f0f3b2e9640 1 Processor -- start 2024-03-14T12:13:28.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.872+0000 7f0f3b2e9640 1 -- start start 2024-03-14T12:13:28.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.872+0000 7f0f3b2e9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0f340b1e70 0x7f0f340af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.872+0000 7f0f3b2e9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0f3415e280 0x7f0f340b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.872+0000 7f0f3b2e9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f340b1560 con 0x7f0f3415f5b0 2024-03-14T12:13:28.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.872+0000 7f0f3b2e9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f340b62e0 con 0x7f0f3415e280 2024-03-14T12:13:28.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.872+0000 7f0f3b2e9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f340b17d0 con 0x7f0f340b1e70 2024-03-14T12:13:28.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f39085640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0f340b1e70 0x7f0f340af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f39886640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0f3415f5b0 0x7f0f340b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42762/0 (socket says 172.21.15.164:42762) 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f39886640 1 -- 172.21.15.164:0/1923770958 learned_addr learned my addr 172.21.15.164:0/1923770958 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f38884640 1 --2- 172.21.15.164:0/1923770958 >> v2:172.21.15.82:3300/0 conn(0x7f0f3415e280 0x7f0f340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f39085640 1 -- 172.21.15.164:0/1923770958 >> v2:172.21.15.82:3300/0 conn(0x7f0f3415e280 msgr2=0x7f0f340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f39085640 1 --2- 172.21.15.164:0/1923770958 >> v2:172.21.15.82:3300/0 conn(0x7f0f3415e280 0x7f0f340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f39085640 1 -- 172.21.15.164:0/1923770958 >> v1:172.21.15.164:6789/0 conn(0x7f0f3415f5b0 legacy=0x7f0f340b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f2bfff640 1 -- 172.21.15.164:0/1923770958 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010334867 0 0) 0x7f0f340b1560 con 0x7f0f3415f5b0 2024-03-14T12:13:28.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.873+0000 7f0f39085640 1 -- 172.21.15.164:0/1923770958 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f340b1ab0 con 0x7f0f340b1e70 2024-03-14T12:13:28.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f39085640 1 --2- 172.21.15.164:0/1923770958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f340b1e70 0x7f0f340af810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f340d1210 tx=0x7f0f3002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68caa05efaa4ac32 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f2bfff640 1 -- 172.21.15.164:0/1923770958 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30035020 con 0x7f0f340b1e70 2024-03-14T12:13:28.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f2bfff640 1 -- 172.21.15.164:0/1923770958 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f30031830 con 0x7f0f340b1e70 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f2bfff640 1 -- 172.21.15.164:0/1923770958 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30031c00 con 0x7f0f340b1e70 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/1923770958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f340b1e70 msgr2=0x7f0f340af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f3b2e9640 1 --2- 172.21.15.164:0/1923770958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f340b1e70 0x7f0f340af810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f340d1210 tx=0x7f0f3002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/1923770958 shutdown_connections 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f3b2e9640 1 --2- 172.21.15.164:0/1923770958 >> v2:172.21.15.82:3300/0 conn(0x7f0f3415e280 0x7f0f340b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f3b2e9640 1 --2- 172.21.15.164:0/1923770958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f340b1e70 0x7f0f340af810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/1923770958 >> 172.21.15.164:0/1923770958 conn(0x7f0f340b4fd0 msgr2=0x7f0f340b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.874+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/1923770958 shutdown_connections 2024-03-14T12:13:28.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.875+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/1923770958 wait complete. 2024-03-14T12:13:28.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 Processor -- start 2024-03-14T12:13:28.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 -- start start 2024-03-14T12:13:28.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f340b1e70 0x7f0f34151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3415e280 0x7f0f3409d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3415f5b0 0x7f0f340c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340aff70 con 0x7f0f340b1e70 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340afdf0 con 0x7f0f3415f5b0 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f3b2e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3406ad10 con 0x7f0f3415e280 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f39886640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3415f5b0 0x7f0f340c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.876+0000 7f0f38884640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3415e280 0x7f0f3409d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3415e280 0x7f0f3409d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54336/0 (socket says 172.21.15.164:54336) 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 -- 172.21.15.164:0/81233059 learned_addr learned my addr 172.21.15.164:0/81233059 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f39886640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3415f5b0 0x7f0f340c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50334/0 (socket says 172.21.15.164:50334) 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f39085640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f340b1e70 0x7f0f34151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 -- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3415f5b0 msgr2=0x7f0f340c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3415f5b0 0x7f0f340c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 -- 172.21.15.164:0/81233059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f340b1e70 msgr2=0x7f0f34151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f340b1e70 0x7f0f34151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 -- 172.21.15.164:0/81233059 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3409a400 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f39886640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3415f5b0 0x7f0f340c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f38884640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3415e280 0x7f0f3409d540 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400e9e0 tx=0x7f0f2400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f29ffb640 1 -- 172.21.15.164:0/81233059 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24017070 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f29ffb640 1 -- 172.21.15.164:0/81233059 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2400c9d0 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3409dd90 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f29ffb640 1 -- 172.21.15.164:0/81233059 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400cdc0 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.877+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3409a6a0 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.878+0000 7f0f29ffb640 1 -- 172.21.15.164:0/81233059 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0f2401a070 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.878+0000 7f0f29ffb640 1 -- 172.21.15.164:0/81233059 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f0f24006660 con 0x7f0f3415e280 2024-03-14T12:13:28.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.879+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3404f050 con 0x7f0f3415e280 2024-03-14T12:13:28.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.881+0000 7f0f29ffb640 1 -- 172.21.15.164:0/81233059 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0f24006020 con 0x7f0f3415e280 2024-03-14T12:13:28.905 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.906+0000 7f1c6263a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-14T12:13:28.920 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.921+0000 7f1cd4e49040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-14T12:13:28.951 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:28.952+0000 7f1c6263a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-14T12:13:28.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.953+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00d8cc9a-7044-475d-a609-45a893e543c7", "id": 4} v 0) v1 -- 0x7f0f3409e2b0 con 0x7f0f3415e280 2024-03-14T12:13:28.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.958+0000 7f0f29ffb640 1 -- 172.21.15.164:0/81233059 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00d8cc9a-7044-475d-a609-45a893e543c7", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0f24005990 con 0x7f0f3415e280 2024-03-14T12:13:28.958 INFO:teuthology.orchestra.run.smithi164.stdout:4 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.959+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3415e280 msgr2=0x7f0f3409d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.959+0000 7f0f3b2e9640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3415e280 0x7f0f3409d540 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400e9e0 tx=0x7f0f2400eea0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.959+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 shutdown_connections 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.959+0000 7f0f3b2e9640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f3415f5b0 0x7f0f340c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.959+0000 7f0f3b2e9640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3415e280 0x7f0f3409d540 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.959+0000 7f0f3b2e9640 1 --2- 172.21.15.164:0/81233059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f340b1e70 0x7f0f34151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.959+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 >> 172.21.15.164:0/81233059 conn(0x7f0f340b4fd0 msgr2=0x7f0f340ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.960+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 shutdown_connections 2024-03-14T12:13:28.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:28.960+0000 7f0f3b2e9640 1 -- 172.21.15.164:0/81233059 wait complete. 2024-03-14T12:13:28.968 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:28.969+0000 7f1cd4e49040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-14T12:13:28.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new f28ae07f-38af-4f2a-81ad-69db2b9f57a7 5 2024-03-14T12:13:29.024 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:29.025+0000 7f1cd4e49040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-14T12:13:29.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.070+0000 7fbf04a85640 1 Processor -- start 2024-03-14T12:13:29.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.070+0000 7fbf04a85640 1 -- start start 2024-03-14T12:13:29.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbf04a85640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbf000d00e0 0x7fbf000b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbf04a85640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbf000af1c0 0x7fbf000d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbf04a85640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000aff70 con 0x7fbf000d0a00 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbf04a85640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0015e5a0 con 0x7fbf000af1c0 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbf04a85640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000b62e0 con 0x7fbf000d00e0 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbefe59c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbf000d00e0 0x7fbf000b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbefed9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbf000d0a00 0x7fbf000ad250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42778/0 (socket says 172.21.15.164:42778) 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.071+0000 7fbefed9d640 1 -- 172.21.15.164:0/1602120399 learned_addr learned my addr 172.21.15.164:0/1602120399 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.072+0000 7fbefdd9b640 1 --2- 172.21.15.164:0/1602120399 >> v2:172.21.15.82:3300/0 conn(0x7fbf000af1c0 0x7fbf000d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.072+0000 7fbefe59c640 1 -- 172.21.15.164:0/1602120399 >> v2:172.21.15.82:3300/0 conn(0x7fbf000af1c0 msgr2=0x7fbf000d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.072+0000 7fbefe59c640 1 --2- 172.21.15.164:0/1602120399 >> v2:172.21.15.82:3300/0 conn(0x7fbf000af1c0 0x7fbf000d04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.072+0000 7fbefe59c640 1 -- 172.21.15.164:0/1602120399 >> v1:172.21.15.164:6789/0 conn(0x7fbf000d0a00 legacy=0x7fbf000ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.072+0000 7fbefd59a640 1 -- 172.21.15.164:0/1602120399 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864506380 0 0) 0x7fbf000aff70 con 0x7fbf000d0a00 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.072+0000 7fbefe59c640 1 -- 172.21.15.164:0/1602120399 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000b1f70 con 0x7fbf000d00e0 2024-03-14T12:13:29.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbefe59c640 1 --2- 172.21.15.164:0/1602120399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d00e0 0x7fbf000b5da0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbf00074b00 tx=0x7fbef002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa04a88ca7b6159 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbefd59a640 1 -- 172.21.15.164:0/1602120399 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0035020 con 0x7fbf000d00e0 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbefd59a640 1 -- 172.21.15.164:0/1602120399 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef0031830 con 0x7fbf000d00e0 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbefd59a640 1 -- 172.21.15.164:0/1602120399 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0031c00 con 0x7fbf000d00e0 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbf04a85640 1 -- 172.21.15.164:0/1602120399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d00e0 msgr2=0x7fbf000b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbf04a85640 1 --2- 172.21.15.164:0/1602120399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d00e0 0x7fbf000b5da0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbf00074b00 tx=0x7fbef002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbf04a85640 1 -- 172.21.15.164:0/1602120399 shutdown_connections 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbf04a85640 1 --2- 172.21.15.164:0/1602120399 >> v2:172.21.15.82:3300/0 conn(0x7fbf000af1c0 0x7fbf000d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.073+0000 7fbf04a85640 1 --2- 172.21.15.164:0/1602120399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d00e0 0x7fbf000b5da0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.074+0000 7fbf04a85640 1 -- 172.21.15.164:0/1602120399 >> 172.21.15.164:0/1602120399 conn(0x7fbf000b4fd0 msgr2=0x7fbf000b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.074+0000 7fbf04a85640 1 -- 172.21.15.164:0/1602120399 shutdown_connections 2024-03-14T12:13:29.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.074+0000 7fbf04a85640 1 -- 172.21.15.164:0/1602120399 wait complete. 2024-03-14T12:13:29.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.075+0000 7fbf04a85640 1 Processor -- start 2024-03-14T12:13:29.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.075+0000 7fbf04a85640 1 -- start start 2024-03-14T12:13:29.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000af1c0 0x7fbf000cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000d00e0 0x7fbf000c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf000d0a00 0x7fbf0009d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000b2810 con 0x7fbf000d0a00 2024-03-14T12:13:29.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000b2530 con 0x7fbf000af1c0 2024-03-14T12:13:29.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000b2af0 con 0x7fbf000d00e0 2024-03-14T12:13:29.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000af1c0 0x7fbf000cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000af1c0 0x7fbf000cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50346/0 (socket says 172.21.15.164:50346) 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 -- 172.21.15.164:0/3801947026 learned_addr learned my addr 172.21.15.164:0/3801947026 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefdd9b640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000d00e0 0x7fbf000c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefed9d640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf000d0a00 0x7fbf0009d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 -- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000d00e0 msgr2=0x7fbf000c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000d00e0 0x7fbf000c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 -- 172.21.15.164:0/3801947026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf000d0a00 msgr2=0x7fbf0009d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf000d0a00 0x7fbf0009d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 -- 172.21.15.164:0/3801947026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00152200 con 0x7fbf000af1c0 2024-03-14T12:13:29.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefed9d640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf000d0a00 0x7fbf0009d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:29.077 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:29.076+0000 7f1cd4e49040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-14T12:13:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbefe59c640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000af1c0 0x7fbf000cf3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009890 tx=0x7fbef00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbeeeffd640 1 -- 172.21.15.164:0/3801947026 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0041070 con 0x7fbf000af1c0 2024-03-14T12:13:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbeeeffd640 1 -- 172.21.15.164:0/3801947026 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef003b920 con 0x7fbf000af1c0 2024-03-14T12:13:29.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0009dc40 con 0x7fbf000af1c0 2024-03-14T12:13:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbeeeffd640 1 -- 172.21.15.164:0/3801947026 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0035040 con 0x7fbf000af1c0 2024-03-14T12:13:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.076+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf000c9620 con 0x7fbf000af1c0 2024-03-14T12:13:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.077+0000 7fbeeeffd640 1 -- 172.21.15.164:0/3801947026 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbef0038020 con 0x7fbf000af1c0 2024-03-14T12:13:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.077+0000 7fbeeeffd640 1 -- 172.21.15.164:0/3801947026 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fbef003bd60 con 0x7fbf000af1c0 2024-03-14T12:13:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.078+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0004f050 con 0x7fbf000af1c0 2024-03-14T12:13:29.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.080+0000 7fbeeeffd640 1 -- 172.21.15.164:0/3801947026 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbef004e910 con 0x7fbf000af1c0 2024-03-14T12:13:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.152+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f28ae07f-38af-4f2a-81ad-69db2b9f57a7", "id": 5} v 0) v1 -- 0x7fbf00160d30 con 0x7fbf000af1c0 2024-03-14T12:13:29.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.157+0000 7fbeeeffd640 1 -- 172.21.15.164:0/3801947026 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f28ae07f-38af-4f2a-81ad-69db2b9f57a7", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbef004eaf0 con 0x7fbf000af1c0 2024-03-14T12:13:29.157 INFO:teuthology.orchestra.run.smithi164.stdout:5 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000af1c0 msgr2=0x7fbf000cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000af1c0 0x7fbf000cf3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009890 tx=0x7fbef00046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 shutdown_connections 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf000d0a00 0x7fbf0009d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000d00e0 0x7fbf000c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 --2- 172.21.15.164:0/3801947026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000af1c0 0x7fbf000cf3f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 >> 172.21.15.164:0/3801947026 conn(0x7fbf000b4fd0 msgr2=0x7fbf0015d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 shutdown_connections 2024-03-14T12:13:29.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.158+0000 7fbf04a85640 1 -- 172.21.15.164:0/3801947026 wait complete. 2024-03-14T12:13:29.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new d4030c0e-7f55-42bf-8f3c-a25e05fd6218 6 2024-03-14T12:13:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.269+0000 7f2979212640 1 Processor -- start 2024-03-14T12:13:29.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.269+0000 7f2979212640 1 -- start start 2024-03-14T12:13:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2979212640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2974158760 0x7f297415e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2979212640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f29740b5470 0x7f297406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2979212640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2974158b40 con 0x7f297406ccc0 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2979212640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297409b700 con 0x7f29740b5470 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2979212640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29740b6840 con 0x7f2974158760 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2974158760 0x7f297415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f297259c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f29740b5470 0x7f297406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f297259c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f29740b5470 0x7f297406c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50354/0 (socket says 172.21.15.164:50354) 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2974158760 0x7f297415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54356/0 (socket says 172.21.15.164:54356) 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f297259c640 1 -- 172.21.15.164:0/867345789 learned_addr learned my addr 172.21.15.164:0/867345789 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f297259c640 1 -- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2974158760 msgr2=0x7f297415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f297259c640 1 --2- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2974158760 0x7f297415e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f297259c640 1 -- 172.21.15.164:0/867345789 >> v1:172.21.15.82:6789/0 conn(0x7f297406ccc0 legacy=0x7f297415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f297259c640 1 -- 172.21.15.164:0/867345789 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740b5c30 con 0x7f29740b5470 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.270+0000 7f2972d9d640 1 --2- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2974158760 0x7f297415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.271+0000 7f297259c640 1 --2- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f297406c780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2964009870 tx=0x7f296402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21cec2a89361242f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.271+0000 7f2971d9b640 1 -- 172.21.15.164:0/867345789 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964034070 con 0x7f29740b5470 2024-03-14T12:13:29.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.271+0000 7f2971d9b640 1 -- 172.21.15.164:0/867345789 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2964030840 con 0x7f29740b5470 2024-03-14T12:13:29.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.271+0000 7f2971d9b640 1 -- 172.21.15.164:0/867345789 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964030c10 con 0x7f29740b5470 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 -- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 msgr2=0x7f297406c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 --2- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f297406c780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2964009870 tx=0x7f296402b980 comp rx=0 tx=0).stop 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 -- 172.21.15.164:0/867345789 shutdown_connections 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 --2- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f297406c780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 --2- 172.21.15.164:0/867345789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2974158760 0x7f297415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 -- 172.21.15.164:0/867345789 >> 172.21.15.164:0/867345789 conn(0x7f297409a9e0 msgr2=0x7f297409adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 -- 172.21.15.164:0/867345789 shutdown_connections 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.272+0000 7f2979212640 1 -- 172.21.15.164:0/867345789 wait complete. 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.273+0000 7f2979212640 1 Processor -- start 2024-03-14T12:13:29.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.273+0000 7f2979212640 1 -- start start 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2979212640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f297406ccc0 0x7f29740afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2979212640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f29740b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2979212640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974158760 0x7f2974153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2979212640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29740b64d0 con 0x7f2974158760 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2979212640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29740b61f0 con 0x7f29740b5470 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2979212640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29740b58f0 con 0x7f297406ccc0 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f297406ccc0 0x7f29740afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f297259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f29740b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f297406ccc0 0x7f29740afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54368/0 (socket says 172.21.15.164:54368) 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f297259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f29740b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50360/0 (socket says 172.21.15.164:50360) 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f2972d9d640 1 -- 172.21.15.164:0/1708961227 learned_addr learned my addr 172.21.15.164:0/1708961227 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.274+0000 7f297359e640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974158760 0x7f2974153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2972d9d640 1 -- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 msgr2=0x7f29740b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2972d9d640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f29740b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2972d9d640 1 -- 172.21.15.164:0/1708961227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974158760 msgr2=0x7f2974153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2972d9d640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974158760 0x7f2974153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2972d9d640 1 -- 172.21.15.164:0/1708961227 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740b05e0 con 0x7f297406ccc0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f297259c640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f29740b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2972d9d640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f297406ccc0 0x7f29740afc90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f296800e960 tx=0x7f296800ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f295b7fe640 1 -- 172.21.15.164:0/1708961227 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968017070 con 0x7f297406ccc0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f295b7fe640 1 -- 172.21.15.164:0/1708961227 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800ca00 con 0x7f297406ccc0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29740b08c0 con 0x7f297406ccc0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f295b7fe640 1 -- 172.21.15.164:0/1708961227 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296800cdd0 con 0x7f297406ccc0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29740b0e80 con 0x7f297406ccc0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f295b7fe640 1 -- 172.21.15.164:0/1708961227 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f296801a070 con 0x7f297406ccc0 2024-03-14T12:13:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.275+0000 7f295b7fe640 1 -- 172.21.15.164:0/1708961227 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f29680064d0 con 0x7f297406ccc0 2024-03-14T12:13:29.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.276+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297404f050 con 0x7f297406ccc0 2024-03-14T12:13:29.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.278+0000 7f295b7fe640 1 -- 172.21.15.164:0/1708961227 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f29680067b0 con 0x7f297406ccc0 2024-03-14T12:13:29.328 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:29.329+0000 7f1c6263a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-14T12:13:29.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.351+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d4030c0e-7f55-42bf-8f3c-a25e05fd6218", "id": 6} v 0) v1 -- 0x7f29740b1280 con 0x7f297406ccc0 2024-03-14T12:13:29.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.356+0000 7f295b7fe640 1 -- 172.21.15.164:0/1708961227 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4030c0e-7f55-42bf-8f3c-a25e05fd6218", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2968005990 con 0x7f297406ccc0 2024-03-14T12:13:29.356 INFO:teuthology.orchestra.run.smithi164.stdout:6 2024-03-14T12:13:29.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f297406ccc0 msgr2=0x7f29740afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f297406ccc0 0x7f29740afc90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f296800e960 tx=0x7f296800ee20 comp rx=0 tx=0).stop 2024-03-14T12:13:29.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 shutdown_connections 2024-03-14T12:13:29.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974158760 0x7f2974153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29740b5470 0x7f29740b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 --2- 172.21.15.164:0/1708961227 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f297406ccc0 0x7f29740afc90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 >> 172.21.15.164:0/1708961227 conn(0x7f297409a9e0 msgr2=0x7f2974157a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 shutdown_connections 2024-03-14T12:13:29.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.358+0000 7f2979212640 1 -- 172.21.15.164:0/1708961227 wait complete. 2024-03-14T12:13:29.368 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new ad7ab054-0c02-4739-9e9d-8c711dd9fc8d 7 2024-03-14T12:13:29.463 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:29.464+0000 7f1c6263a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-14T12:13:29.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.469+0000 7f799e06c640 1 Processor -- start 2024-03-14T12:13:29.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.469+0000 7f799e06c640 1 -- start start 2024-03-14T12:13:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f799e06c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f79981447c0 0x7f799806f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f799e06c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79980b4bc0 0x7f79980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f799e06c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799806f6d0 con 0x7f79980b3b30 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f799e06c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799806e0f0 con 0x7f79980b4bc0 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f799e06c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799806e3d0 con 0x7f79981447c0 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f79977fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f79981447c0 0x7f799806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f798effd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f79980b3b30 0x7f799806d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42786/0 (socket says 172.21.15.164:42786) 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f798effd640 1 -- 172.21.15.164:0/382985989 learned_addr learned my addr 172.21.15.164:0/382985989 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f7997fff640 1 --2- 172.21.15.164:0/382985989 >> v2:172.21.15.82:3300/0 conn(0x7f79980b4bc0 0x7f79980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f79977fe640 1 -- 172.21.15.164:0/382985989 >> v2:172.21.15.82:3300/0 conn(0x7f79980b4bc0 msgr2=0x7f79980b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f79977fe640 1 --2- 172.21.15.164:0/382985989 >> v2:172.21.15.82:3300/0 conn(0x7f79980b4bc0 0x7f79980b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f79977fe640 1 -- 172.21.15.164:0/382985989 >> v1:172.21.15.164:6789/0 conn(0x7f79980b3b30 legacy=0x7f799806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f79977fe640 1 -- 172.21.15.164:0/382985989 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799806e6b0 con 0x7f79981447c0 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f7996ffd640 1 -- 172.21.15.164:0/382985989 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986279387 0 0) 0x7f799806f6d0 con 0x7f79980b3b30 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.470+0000 7f7997fff640 1 --2- 172.21.15.164:0/382985989 >> v2:172.21.15.82:3300/0 conn(0x7f79980b4bc0 0x7f79980b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.471+0000 7f79977fe640 1 --2- 172.21.15.164:0/382985989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79981447c0 0x7f799806f190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7998073110 tx=0x7f798802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad72c8c25fec4ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.471+0000 7f7996ffd640 1 -- 172.21.15.164:0/382985989 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988035020 con 0x7f79981447c0 2024-03-14T12:13:29.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.471+0000 7f7996ffd640 1 -- 172.21.15.164:0/382985989 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7988031830 con 0x7f79981447c0 2024-03-14T12:13:29.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.471+0000 7f7996ffd640 1 -- 172.21.15.164:0/382985989 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988031c00 con 0x7f79981447c0 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 -- 172.21.15.164:0/382985989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79981447c0 msgr2=0x7f799806f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 --2- 172.21.15.164:0/382985989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79981447c0 0x7f799806f190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7998073110 tx=0x7f798802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 -- 172.21.15.164:0/382985989 shutdown_connections 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 --2- 172.21.15.164:0/382985989 >> v2:172.21.15.82:3300/0 conn(0x7f79980b4bc0 0x7f79980b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 --2- 172.21.15.164:0/382985989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79981447c0 0x7f799806f190 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 -- 172.21.15.164:0/382985989 >> 172.21.15.164:0/382985989 conn(0x7f79980d0b40 msgr2=0x7f79980d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 -- 172.21.15.164:0/382985989 shutdown_connections 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.472+0000 7f799e06c640 1 -- 172.21.15.164:0/382985989 wait complete. 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.473+0000 7f799e06c640 1 Processor -- start 2024-03-14T12:13:29.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.473+0000 7f799e06c640 1 -- start start 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f799e06c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79980b3b30 0x7f799814d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f799e06c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79980b4bc0 0x7f799813e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f799e06c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79981447c0 0x7f799814bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f799e06c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79980ce0d0 con 0x7f79981447c0 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f799e06c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799806ec70 con 0x7f79980b4bc0 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f799e06c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79980ce250 con 0x7f79980b3b30 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f798effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79980b4bc0 0x7f799813e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79980b3b30 0x7f799814d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79980b3b30 0x7f799814d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:54370/0 (socket says 172.21.15.164:54370) 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 -- 172.21.15.164:0/2846041181 learned_addr learned my addr 172.21.15.164:0/2846041181 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f7997fff640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79981447c0 0x7f799814bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 -- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79980b4bc0 msgr2=0x7f799813e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79980b4bc0 0x7f799813e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 -- 172.21.15.164:0/2846041181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79981447c0 msgr2=0x7f799814bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79981447c0 0x7f799814bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f79977fe640 1 -- 172.21.15.164:0/2846041181 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799814c6f0 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.474+0000 7f798effd640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79980b4bc0 0x7f799813e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f79977fe640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79980b3b30 0x7f799814d790 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7988009890 tx=0x7f79880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f7994ff9640 1 -- 172.21.15.164:0/2846041181 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798803a9b0 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f7994ff9640 1 -- 172.21.15.164:0/2846041181 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798803ab70 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799814c9d0 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f7994ff9640 1 -- 172.21.15.164:0/2846041181 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988045450 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799814cf90 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f7994ff9640 1 -- 172.21.15.164:0/2846041181 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7988038020 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.475+0000 7f7994ff9640 1 -- 172.21.15.164:0/2846041181 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f798804ec80 con 0x7f79980b3b30 2024-03-14T12:13:29.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.476+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799804f050 con 0x7f79980b3b30 2024-03-14T12:13:29.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.478+0000 7f7994ff9640 1 -- 172.21.15.164:0/2846041181 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7988053020 con 0x7f79980b3b30 2024-03-14T12:13:29.480 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:29.481+0000 7f1cd4e49040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-14T12:13:29.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.564+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ad7ab054-0c02-4739-9e9d-8c711dd9fc8d", "id": 7} v 0) v1 -- 0x7f7998072520 con 0x7f79980b3b30 2024-03-14T12:13:29.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.570+0000 7f7994ff9640 1 -- 172.21.15.164:0/2846041181 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad7ab054-0c02-4739-9e9d-8c711dd9fc8d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f79880317d0 con 0x7f79980b3b30 2024-03-14T12:13:29.569 INFO:teuthology.orchestra.run.smithi164.stdout:7 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79980b3b30 msgr2=0x7f799814d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79980b3b30 0x7f799814d790 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7988009890 tx=0x7f79880046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 shutdown_connections 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79981447c0 0x7f799814bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79980b4bc0 0x7f799813e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 --2- 172.21.15.164:0/2846041181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79980b3b30 0x7f799814d790 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 >> 172.21.15.164:0/2846041181 conn(0x7f79980d0b40 msgr2=0x7f79980ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 shutdown_connections 2024-03-14T12:13:29.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:13:29.571+0000 7f799e06c640 1 -- 172.21.15.164:0/2846041181 wait complete. 2024-03-14T12:13:29.572 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:29.572+0000 7f1c6263a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-14T12:13:29.581 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-14T12:13:29.581 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-14T12:13:29.583 INFO:tasks.ceph.osd.0:Started 2024-03-14T12:13:29.583 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-14T12:13:29.583 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-14T12:13:29.585 INFO:tasks.ceph.osd.1:Started 2024-03-14T12:13:29.586 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-14T12:13:29.586 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-14T12:13:29.587 INFO:tasks.ceph.osd.2:Started 2024-03-14T12:13:29.588 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-14T12:13:29.588 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-14T12:13:29.590 INFO:tasks.ceph.osd.3:Started 2024-03-14T12:13:29.590 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-14T12:13:29.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-14T12:13:29.592 INFO:tasks.ceph.osd.4:Started 2024-03-14T12:13:29.592 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-14T12:13:29.592 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-14T12:13:29.594 INFO:tasks.ceph.osd.5:Started 2024-03-14T12:13:29.594 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-14T12:13:29.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-14T12:13:29.596 INFO:tasks.ceph.osd.6:Started 2024-03-14T12:13:29.596 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-14T12:13:29.596 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-14T12:13:29.600 INFO:tasks.ceph.osd.7:Started 2024-03-14T12:13:29.600 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:29.630 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:29.632+0000 7f1c6263a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-14T12:13:29.638 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:29.639+0000 7f1cd4e49040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-14T12:13:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.719+0000 7ffbb2497640 1 Processor -- start 2024-03-14T12:13:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.719+0000 7ffbb2497640 1 -- start start 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbb2497640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0d0760 0x7ffbac0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbb2497640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffbac0cf370 0x7ffbac0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbb2497640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbac0d1080 con 0x7ffbac0d1da0 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbb2497640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac0cf8c0 con 0x7ffbac0cf370 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbb2497640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0cfba0 con 0x7ffbac0d0760 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbab7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffbac0cf370 0x7ffbac0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbb0a34640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffbac0d1da0 0x7ffbac0d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58022/0 (socket says 172.21.15.82:58022) 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbb0a34640 1 -- 172.21.15.82:0/1879268152 learned_addr learned my addr 172.21.15.82:0/1879268152 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.720+0000 7ffbabfff640 1 --2- 172.21.15.82:0/1879268152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0d0760 0x7ffbac0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.721+0000 7ffbab7fe640 1 -- 172.21.15.82:0/1879268152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0d0760 msgr2=0x7ffbac0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.721+0000 7ffbab7fe640 1 --2- 172.21.15.82:0/1879268152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0d0760 0x7ffbac0d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.721+0000 7ffbab7fe640 1 -- 172.21.15.82:0/1879268152 >> v1:172.21.15.82:6789/0 conn(0x7ffbac0d1da0 legacy=0x7ffbac0d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.721+0000 7ffbaaffd640 1 -- 172.21.15.82:0/1879268152 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275028840 0 0) 0x7ffbac0d1080 con 0x7ffbac0d1da0 2024-03-14T12:13:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.721+0000 7ffbab7fe640 1 -- 172.21.15.82:0/1879268152 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0b6c40 con 0x7ffbac0cf370 2024-03-14T12:13:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.721+0000 7ffbab7fe640 1 --2- 172.21.15.82:0/1879268152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0cf370 0x7ffbac0d4660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0097a0 tx=0x7ffb9c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb550513082ff405 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbaaffd640 1 -- 172.21.15.82:0/1879268152 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c034070 con 0x7ffbac0cf370 2024-03-14T12:13:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbaaffd640 1 -- 172.21.15.82:0/1879268152 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c030610 con 0x7ffbac0cf370 2024-03-14T12:13:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbaaffd640 1 -- 172.21.15.82:0/1879268152 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0309e0 con 0x7ffbac0cf370 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 -- 172.21.15.82:0/1879268152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0cf370 msgr2=0x7ffbac0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 --2- 172.21.15.82:0/1879268152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0cf370 0x7ffbac0d4660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0097a0 tx=0x7ffb9c02b960 comp rx=0 tx=0).stop 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 -- 172.21.15.82:0/1879268152 shutdown_connections 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 --2- 172.21.15.82:0/1879268152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0cf370 0x7ffbac0d4660 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 --2- 172.21.15.82:0/1879268152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0d0760 0x7ffbac0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 -- 172.21.15.82:0/1879268152 >> 172.21.15.82:0/1879268152 conn(0x7ffbac09d030 msgr2=0x7ffbac0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 -- 172.21.15.82:0/1879268152 shutdown_connections 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.722+0000 7ffbb2497640 1 -- 172.21.15.82:0/1879268152 wait complete. 2024-03-14T12:13:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.723+0000 7ffbb2497640 1 Processor -- start 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbb2497640 1 -- start start 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbb2497640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0cf370 0x7ffbac1f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbb2497640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0d0760 0x7ffbac154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbb2497640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbac0d1da0 0x7ffbac06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbb2497640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0d4ba0 con 0x7ffbac0d0760 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbb2497640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0b7200 con 0x7ffbac0d1da0 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbb2497640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1617a0 con 0x7ffbac0cf370 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.724+0000 7ffbab7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0d0760 0x7ffbac154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0d0760 0x7ffbac154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34902/0 (socket says 172.21.15.82:34902) 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 -- 172.21.15.82:0/4234008907 learned_addr learned my addr 172.21.15.82:0/4234008907 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbabfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0cf370 0x7ffbac1f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbb0a34640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbac0d1da0 0x7ffbac06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 -- 172.21.15.82:0/4234008907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0cf370 msgr2=0x7ffbac1f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0cf370 0x7ffbac1f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 -- 172.21.15.82:0/4234008907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbac0d1da0 msgr2=0x7ffbac06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbac0d1da0 0x7ffbac06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 -- 172.21.15.82:0/4234008907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac1550c0 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbab7fe640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0d0760 0x7ffbac154b80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c02bf70 tx=0x7ffb9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffba8ff9640 1 -- 172.21.15.82:0/4234008907 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c034040 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffba8ff9640 1 -- 172.21.15.82:0/4234008907 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c039990 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac0cb1e0 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffba8ff9640 1 -- 172.21.15.82:0/4234008907 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c039da0 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac0cb7a0 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffba8ff9640 1 -- 172.21.15.82:0/4234008907 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ffb9c037070 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.725+0000 7ffba8ff9640 1 -- 172.21.15.82:0/4234008907 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ffb9c04bd90 con 0x7ffbac0d0760 2024-03-14T12:13:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.726+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac04f050 con 0x7ffbac0d0760 2024-03-14T12:13:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.728+0000 7ffba8ff9640 1 -- 172.21.15.82:0/4234008907 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ffb9c0304d0 con 0x7ffbac0d0760 2024-03-14T12:13:29.768 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:29.769+0000 7f1cd4e49040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-14T12:13:29.770 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:29.771+0000 7f1c6263a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-14T12:13:29.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.789+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbac0d0540 con 0x7ffbac0d0760 2024-03-14T12:13:29.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.790+0000 7ffba8ff9640 1 -- 172.21.15.82:0/4234008907 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7ffb9c04b400 con 0x7ffbac0d0760 2024-03-14T12:13:29.789 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:29.789 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:29.566529+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0d0760 msgr2=0x7ffbac154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0d0760 0x7ffbac154b80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c02bf70 tx=0x7ffb9c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 shutdown_connections 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbac0d1da0 0x7ffbac06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac0d0760 0x7ffbac154b80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 --2- 172.21.15.82:0/4234008907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbac0cf370 0x7ffbac1f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 >> 172.21.15.82:0/4234008907 conn(0x7ffbac09d030 msgr2=0x7ffbac0b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:29.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 shutdown_connections 2024-03-14T12:13:29.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:29.791+0000 7ffbb2497640 1 -- 172.21.15.82:0/4234008907 wait complete. 2024-03-14T12:13:29.800 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-14T12:13:29.801 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-14T12:13:29.812 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:29.813+0000 7f1cd4e49040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-14T12:13:29.868 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:29.869+0000 7f1c6263a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-14T12:13:29.923 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:29.924+0000 7f1cd4e49040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-14T12:13:30.021 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:30.022+0000 7f1cd4e49040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-14T12:13:30.023 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:30.024+0000 7f1c6263a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-14T12:13:30.064 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:30.065+0000 7f1c6263a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-14T12:13:30.174 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:30.174+0000 7f1cd4e49040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-14T12:13:30.186 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-03-14T12:13:30.187+0000 7f8a5340d640 -1 Falling back to public interface 2024-03-14T12:13:30.195 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-03-14T12:13:30.196+0000 7fdfa3324640 -1 Falling back to public interface 2024-03-14T12:13:30.196 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-14T12:13:30.197+0000 7f9a2a811640 -1 Falling back to public interface 2024-03-14T12:13:30.203 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-14T12:13:30.204+0000 7f7672411640 -1 Falling back to public interface 2024-03-14T12:13:30.205 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-14T12:13:30.206+0000 7fbd49624640 -1 Falling back to public interface 2024-03-14T12:13:30.218 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:30.219+0000 7f1cd4e49040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-14T12:13:30.220 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:30.221+0000 7f1c6263a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-14T12:13:30.223 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-14T12:13:30.224+0000 7f739b3ff640 -1 Falling back to public interface 2024-03-14T12:13:30.224 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-03-14T12:13:30.225+0000 7f33d1706640 -1 Falling back to public interface 2024-03-14T12:13:30.225 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-03-14T12:13:30.225+0000 7f9ef1f06640 -1 Falling back to public interface 2024-03-14T12:13:30.225 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-03-14T12:13:30.226+0000 7f9ef0eb5640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-03-14T12:13:30.376 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:30.376+0000 7f1cd4e49040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-14T12:13:30.605 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:30.696 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.697+0000 7fe9f6cb2640 1 Processor -- start 2024-03-14T12:13:30.696 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.697+0000 7fe9f6cb2640 1 -- start start 2024-03-14T12:13:30.698 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9f6cb2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f015dfc0 0x7fe9f00d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:30.698 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9f6cb2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9f006fbc0 0x7fe9f00b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:30.698 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9f6cb2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f0070150 con 0x7fe9f00b59f0 2024-03-14T12:13:30.698 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9f6cb2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f006e320 con 0x7fe9f006fbc0 2024-03-14T12:13:30.698 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9f6cb2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f006d880 con 0x7fe9f015dfc0 2024-03-14T12:13:30.698 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9effff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9f006fbc0 0x7fe9f00b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9effff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9f006fbc0 0x7fe9f00b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34906/0 (socket says 172.21.15.82:34906) 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9effff640 1 -- 172.21.15.82:0/885162261 learned_addr learned my addr 172.21.15.82:0/885162261 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.698+0000 7fe9f4a4e640 1 --2- 172.21.15.82:0/885162261 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f015dfc0 0x7fe9f00d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9effff640 1 -- 172.21.15.82:0/885162261 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f015dfc0 msgr2=0x7fe9f00d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9effff640 1 --2- 172.21.15.82:0/885162261 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f015dfc0 0x7fe9f00d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9effff640 1 -- 172.21.15.82:0/885162261 >> v1:172.21.15.164:6789/0 conn(0x7fe9f00b59f0 legacy=0x7fe9f00b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9effff640 1 -- 172.21.15.82:0/885162261 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f006db60 con 0x7fe9f006fbc0 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9effff640 1 --2- 172.21.15.82:0/885162261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f00b54b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009870 tx=0x7fe9e402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a24e2e68ec73f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9ef7fe640 1 -- 172.21.15.82:0/885162261 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40317c0 con 0x7fe9f006fbc0 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9ef7fe640 1 -- 172.21.15.82:0/885162261 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4037070 con 0x7fe9f006fbc0 2024-03-14T12:13:30.699 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.699+0000 7fe9ef7fe640 1 -- 172.21.15.82:0/885162261 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4031b80 con 0x7fe9f006fbc0 2024-03-14T12:13:30.700 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/885162261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 msgr2=0x7fe9f00b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:30.700 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 --2- 172.21.15.82:0/885162261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f00b54b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009870 tx=0x7fe9e402b860 comp rx=0 tx=0).stop 2024-03-14T12:13:30.700 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/885162261 shutdown_connections 2024-03-14T12:13:30.700 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 --2- 172.21.15.82:0/885162261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f00b54b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.701 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 --2- 172.21.15.82:0/885162261 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f015dfc0 0x7fe9f00d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.701 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/885162261 >> 172.21.15.82:0/885162261 conn(0x7fe9f00d0a80 msgr2=0x7fe9f00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:30.701 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/885162261 shutdown_connections 2024-03-14T12:13:30.701 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.700+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/885162261 wait complete. 2024-03-14T12:13:30.701 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.701+0000 7fe9f6cb2640 1 Processor -- start 2024-03-14T12:13:30.701 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.701+0000 7fe9f6cb2640 1 -- start start 2024-03-14T12:13:30.703 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f6cb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f0143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:30.703 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f6cb2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f00b59f0 0x7fe9f00cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:30.703 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f6cb2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f015dfc0 0x7fe9f0158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:30.703 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f6cb2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f006e780 con 0x7fe9f006fbc0 2024-03-14T12:13:30.703 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f6cb2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f006e600 con 0x7fe9f015dfc0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f6cb2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00b4080 con 0x7fe9f00b59f0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f0143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f0143440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34916/0 (socket says 172.21.15.82:34916) 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 -- 172.21.15.82:0/3986111448 learned_addr learned my addr 172.21.15.82:0/3986111448 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9effff640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f00b59f0 0x7fe9f00cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f524f640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f015dfc0 0x7fe9f0158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 -- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f00b59f0 msgr2=0x7fe9f00cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f00b59f0 0x7fe9f00cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 -- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f015dfc0 msgr2=0x7fe9f0158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f015dfc0 0x7fe9f0158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 -- 172.21.15.82:0/3986111448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00ce740 con 0x7fe9f006fbc0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9f4a4e640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f0143440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000e8a0 tx=0x7fe9e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.702+0000 7fe9ed7fa640 1 -- 172.21.15.82:0/3986111448 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0017070 con 0x7fe9f006fbc0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.703+0000 7fe9ed7fa640 1 -- 172.21.15.82:0/3986111448 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e000c940 con 0x7fe9f006fbc0 2024-03-14T12:13:30.704 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.703+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f00cea20 con 0x7fe9f006fbc0 2024-03-14T12:13:30.705 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.703+0000 7fe9ed7fa640 1 -- 172.21.15.82:0/3986111448 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000cd30 con 0x7fe9f006fbc0 2024-03-14T12:13:30.705 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.703+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f00cefe0 con 0x7fe9f006fbc0 2024-03-14T12:13:30.705 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.703+0000 7fe9ed7fa640 1 -- 172.21.15.82:0/3986111448 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe9e001a070 con 0x7fe9f006fbc0 2024-03-14T12:13:30.705 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.703+0000 7fe9ed7fa640 1 -- 172.21.15.82:0/3986111448 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe9e001cb20 con 0x7fe9f006fbc0 2024-03-14T12:13:30.705 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.703+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f004f050 con 0x7fe9f006fbc0 2024-03-14T12:13:30.705 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.706+0000 7fe9ed7fa640 1 -- 172.21.15.82:0/3986111448 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe9e001c450 con 0x7fe9f006fbc0 2024-03-14T12:13:30.765 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.767+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9f0072900 con 0x7fe9f006fbc0 2024-03-14T12:13:30.766 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.767+0000 7fe9ed7fa640 1 -- 172.21.15.82:0/3986111448 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fe9e001c630 con 0x7fe9f006fbc0 2024-03-14T12:13:30.766 INFO:teuthology.misc.health.smithi082.stdout: 2024-03-14T12:13:30.766 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":10,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:29.566529+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:30.767 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 msgr2=0x7fe9f0143440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:30.767 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f0143440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000e8a0 tx=0x7fe9e000ed60 comp rx=0 tx=0).stop 2024-03-14T12:13:30.768 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 shutdown_connections 2024-03-14T12:13:30.768 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9f015dfc0 0x7fe9f0158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.768 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9f00b59f0 0x7fe9f00cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.768 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 --2- 172.21.15.82:0/3986111448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f006fbc0 0x7fe9f0143440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:30.768 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 >> 172.21.15.82:0/3986111448 conn(0x7fe9f00d0a80 msgr2=0x7fe9f00af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:30.768 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 shutdown_connections 2024-03-14T12:13:30.768 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:30.768+0000 7fe9f6cb2640 1 -- 172.21.15.82:0/3986111448 wait complete. 2024-03-14T12:13:30.777 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-14T12:13:31.027 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:31.028+0000 7f1c6263a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-14T12:13:31.195 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:31.195+0000 7f1cd4e49040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-14T12:13:31.304 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:31.305+0000 7f1c6263a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-14T12:13:31.355 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:31.357+0000 7f1c6263a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-14T12:13:31.471 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:31.472+0000 7f1c6263a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-14T12:13:31.473 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:31.474+0000 7f1cd4e49040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-14T12:13:31.527 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:31.528+0000 7f1cd4e49040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-14T12:13:31.645 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-03-14T12:13:31.646+0000 7f1cd4e49040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-14T12:13:32.497 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:32.498+0000 7f1c03391640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-14T12:13:32.497 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-14T12:13:32.498+0000 7f1c03391640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-14T12:13:33.669 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-14T12:13:33.670+0000 7f9a2a811640 -1 osd.0 0 log_to_monitors true 2024-03-14T12:13:33.672 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-03-14T12:13:33.673+0000 7fdfa3324640 -1 osd.6 0 log_to_monitors true 2024-03-14T12:13:33.674 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-03-14T12:13:33.675+0000 7f8a5340d640 -1 osd.4 0 log_to_monitors true 2024-03-14T12:13:33.719 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-03-14T12:13:33.720+0000 7f33d1706640 -1 osd.7 0 log_to_monitors true 2024-03-14T12:13:33.721 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-14T12:13:33.722+0000 7fbd49624640 -1 osd.1 0 log_to_monitors true 2024-03-14T12:13:33.743 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-03-14T12:13:33.744+0000 7f9ef1f06640 -1 osd.5 0 log_to_monitors true 2024-03-14T12:13:33.761 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-14T12:13:33.762+0000 7f739b3ff640 -1 osd.3 0 log_to_monitors true 2024-03-14T12:13:33.766 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-14T12:13:33.767+0000 7f7672411640 -1 osd.2 0 log_to_monitors true 2024-03-14T12:13:37.422 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-14T12:13:37.422+0000 7f9a27124640 -1 osd.0 0 waiting for initial osdmap 2024-03-14T12:13:37.441 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-14T12:13:37.442+0000 7f9a1df1e640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.489 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-14T12:13:37.490+0000 7f7396b72640 -1 osd.3 0 waiting for initial osdmap 2024-03-14T12:13:37.506 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-14T12:13:37.507+0000 7f738f1a4640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.550 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-03-14T12:13:37.551+0000 7f9eeeeb1640 -1 osd.5 0 waiting for initial osdmap 2024-03-14T12:13:37.565 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-03-14T12:13:37.566+0000 7f9ee5cab640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.570 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-03-14T12:13:37.571+0000 7fdf9ea97640 -1 osd.6 0 waiting for initial osdmap 2024-03-14T12:13:37.580 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-14T12:13:37.581+0000 7fbd44d97640 -1 osd.1 0 waiting for initial osdmap 2024-03-14T12:13:37.583 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:37.583 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-03-14T12:13:37.584+0000 7fdf970c9640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.593 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-03-14T12:13:37.594+0000 7f8a503b8640 -1 osd.4 0 waiting for initial osdmap 2024-03-14T12:13:37.593 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-03-14T12:13:37.594+0000 7f33cce79640 -1 osd.7 0 waiting for initial osdmap 2024-03-14T12:13:37.597 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-14T12:13:37.598+0000 7fbd3d3c9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.605 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-03-14T12:13:37.606+0000 7f8a471b2640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.608 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-03-14T12:13:37.609+0000 7f33c54ab640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.646 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-14T12:13:37.647+0000 7f766d604640 -1 osd.2 0 waiting for initial osdmap 2024-03-14T12:13:37.662 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-14T12:13:37.664+0000 7f7665c36640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T12:13:37.676 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.677+0000 7f6c42e68640 1 Processor -- start 2024-03-14T12:13:37.676 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.677+0000 7f6c42e68640 1 -- start start 2024-03-14T12:13:37.677 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.677+0000 7f6c42e68640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6c3c06ae10 0x7f6c3c071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:37.677 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c42e68640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c072160 con 0x7f6c3c06db50 2024-03-14T12:13:37.677 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c42e68640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c072440 con 0x7f6c3c071cc0 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c42e68640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c15c700 con 0x7f6c3c06ae10 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c40c04640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6c3c06db50 0x7f6c3c15cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47428/0 (socket says 172.21.15.82:47428) 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c40c04640 1 -- 172.21.15.82:0/1349108078 learned_addr learned my addr 172.21.15.82:0/1349108078 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3bfff640 1 --2- 172.21.15.82:0/1349108078 >> v2:172.21.15.164:3300/0 conn(0x7f6c3c06ae10 0x7f6c3c071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3b7fe640 1 -- 172.21.15.82:0/1349108078 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059297631 0 0) 0x7f6c3c072160 con 0x7f6c3c06db50 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3b7fe640 1 -- 172.21.15.82:0/1349108078 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c2c003520 con 0x7f6c3c06db50 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3b7fe640 1 -- 172.21.15.82:0/1349108078 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697488045 0 0) 0x7f6c3c072440 con 0x7f6c3c071cc0 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3b7fe640 1 -- 172.21.15.82:0/1349108078 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c072160 con 0x7f6c3c071cc0 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3bfff640 1 -- 172.21.15.82:0/1349108078 >> v1:172.21.15.164:6789/0 conn(0x7f6c3c071cc0 legacy=0x7f6c3c06f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3bfff640 1 -- 172.21.15.82:0/1349108078 >> v1:172.21.15.82:6789/0 conn(0x7f6c3c06db50 legacy=0x7f6c3c15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:37.678 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.678+0000 7f6c3bfff640 1 -- 172.21.15.82:0/1349108078 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c15c390 con 0x7f6c3c06ae10 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.679+0000 7f6c3bfff640 1 --2- 172.21.15.82:0/1349108078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c071780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c15c880 tx=0x7f6c2802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c543c728b6cfa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.679+0000 7f6c3b7fe640 1 -- 172.21.15.82:0/1349108078 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28035020 con 0x7f6c3c06ae10 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.679+0000 7f6c3b7fe640 1 -- 172.21.15.82:0/1349108078 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28031830 con 0x7f6c3c06ae10 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.679+0000 7f6c3b7fe640 1 -- 172.21.15.82:0/1349108078 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28031be0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.680+0000 7f6c42e68640 1 -- 172.21.15.82:0/1349108078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c06ae10 msgr2=0x7f6c3c071780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.680+0000 7f6c42e68640 1 --2- 172.21.15.82:0/1349108078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c071780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c15c880 tx=0x7f6c2802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.680+0000 7f6c42e68640 1 -- 172.21.15.82:0/1349108078 shutdown_connections 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.680+0000 7f6c42e68640 1 --2- 172.21.15.82:0/1349108078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c071780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.680+0000 7f6c42e68640 1 -- 172.21.15.82:0/1349108078 >> 172.21.15.82:0/1349108078 conn(0x7f6c3c0d0a80 msgr2=0x7f6c3c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.680+0000 7f6c42e68640 1 -- 172.21.15.82:0/1349108078 shutdown_connections 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.680+0000 7f6c42e68640 1 -- 172.21.15.82:0/1349108078 wait complete. 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.681+0000 7f6c42e68640 1 Processor -- start 2024-03-14T12:13:37.680 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.681+0000 7f6c42e68640 1 -- start start 2024-03-14T12:13:37.682 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c42e68640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c42e68640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c3c06db50 0x7f6c3c138530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c42e68640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c071cc0 0x7f6c3c15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c42e68640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c06fdf0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c42e68640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c06e200 con 0x7f6c3c071cc0 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c42e68640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c06ff70 con 0x7f6c3c06db50 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38086/0 (socket says 172.21.15.82:38086) 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 -- 172.21.15.82:0/3000160929 learned_addr learned my addr 172.21.15.82:0/3000160929 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c3bfff640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c3c06db50 0x7f6c3c138530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c41405640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c071cc0 0x7f6c3c15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 -- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c3c06db50 msgr2=0x7f6c3c138530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c3c06db50 0x7f6c3c138530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 -- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c071cc0 msgr2=0x7f6c3c15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:37.683 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c071cc0 0x7f6c3c15e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 -- 172.21.15.82:0/3000160929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c138ff0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c41405640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c071cc0 0x7f6c3c15e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c40c04640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c155700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c530 tx=0x7f6c3000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c397fa640 1 -- 172.21.15.82:0/3000160929 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30017070 con 0x7f6c3c06ae10 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c397fa640 1 -- 172.21.15.82:0/3000160929 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30005ce0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c130f20 con 0x7f6c3c06ae10 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.682+0000 7f6c397fa640 1 -- 172.21.15.82:0/3000160929 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30007270 con 0x7f6c3c06ae10 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.683+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c1314e0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.684+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c04f050 con 0x7f6c3c06ae10 2024-03-14T12:13:37.684 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.684+0000 7f6c397fa640 1 -- 172.21.15.82:0/3000160929 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7f6c3001a070 con 0x7f6c3c06ae10 2024-03-14T12:13:37.685 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.684+0000 7f6c397fa640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c2006bb40 0x7f6c2006e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:37.685 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.684+0000 7f6c397fa640 1 -- 172.21.15.82:0/3000160929 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f6c30088da0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.685 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.684+0000 7f6c3bfff640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c2006bb40 0x7f6c2006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:37.685 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.685+0000 7f6c3bfff640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c2006bb40 0x7f6c2006e000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6c28036000 tx=0x7f6c2803c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:37.688 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.689+0000 7f6c397fa640 1 -- 172.21.15.82:0/3000160929 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6c30058140 con 0x7f6c3c06ae10 2024-03-14T12:13:37.864 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.865+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c3c0d17b0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.865 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.866+0000 7f6c397fa640 1 -- 172.21.15.82:0/3000160929 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f6c3005c1e0 con 0x7f6c3c06ae10 2024-03-14T12:13:37.865 INFO:teuthology.misc.health.smithi082.stdout: 2024-03-14T12:13:37.865 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":12,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:35.499381+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c2006bb40 msgr2=0x7f6c2006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c2006bb40 0x7f6c2006e000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6c28036000 tx=0x7f6c2803c040 comp rx=0 tx=0).stop 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c3c06ae10 msgr2=0x7f6c3c155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c155700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c530 tx=0x7f6c3000c9f0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 shutdown_connections 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c3c071cc0 0x7f6c3c15e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c2006bb40 0x7f6c2006e000 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.867 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c3c06db50 0x7f6c3c138530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.868 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 --2- 172.21.15.82:0/3000160929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c3c06ae10 0x7f6c3c155700 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:37.868 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 >> 172.21.15.82:0/3000160929 conn(0x7f6c3c0d0a80 msgr2=0x7f6c3c06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:37.868 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 shutdown_connections 2024-03-14T12:13:37.868 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:37.868+0000 7f6c42e68640 1 -- 172.21.15.82:0/3000160929 wait complete. 2024-03-14T12:13:37.877 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-14T12:13:44.684 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:44.784 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.784+0000 7fe8b69b3640 1 Processor -- start 2024-03-14T12:13:44.784 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.784+0000 7fe8b69b3640 1 -- start start 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8b69b3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00cf520 0x7fe8b00b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8b69b3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00cfa00 con 0x7fe8b00b3690 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8b69b3640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00cfce0 con 0x7fe8b00b42e0 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8b69b3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d1be0 con 0x7fe8b00cf520 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8b4f50640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe8b00b3690 0x7fe8b00d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59052/0 (socket says 172.21.15.82:59052) 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8b4f50640 1 -- 172.21.15.82:0/999159360 learned_addr learned my addr 172.21.15.82:0/999159360 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00cf520 0x7fe8b00b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.788 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332065685 0 0) 0x7fe8b00cfa00 con 0x7fe8b00b3690 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.785+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe898003520 con 0x7fe8b00b3690 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3936602421 0 0) 0x7fe898003520 con 0x7fe8b00b3690 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00cf520 msgr2=0x7fe8b00b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00cf520 0x7fe8b00b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 >> v1:172.21.15.164:6789/0 conn(0x7fe8b00b42e0 legacy=0x7fe8b00d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00d1ec0 con 0x7fe8b00b3690 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fe8a8001a10 con 0x7fe8b00b3690 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 >> v1:172.21.15.82:6789/0 conn(0x7fe8b00b3690 legacy=0x7fe8b00d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8980039e0 0x7fe898003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe898004390 0x7fe898004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.789 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe898005060 0x7fe89800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d1be0 con 0x7fe898004390 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89800bf30 con 0x7fe8980039e0 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89800c130 con 0x7fe898005060 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00cf520 0x7fe8b00b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8b4f50640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe898005060 0x7fe89800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe898004390 0x7fe898004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8affff640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8980039e0 0x7fe898003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8af7fe640 1 -- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe898005060 msgr2=0x7fe89800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe898005060 0x7fe89800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8af7fe640 1 -- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8980039e0 msgr2=0x7fe898003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8980039e0 0x7fe898003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.786+0000 7fe8af7fe640 1 -- 172.21.15.82:0/999159360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898003520 con 0x7fe898004390 2024-03-14T12:13:44.790 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.787+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe898004390 0x7fe898004860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00a1d20 tx=0x7fe898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe89847372b170c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.787+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00097d0 con 0x7fe898004390 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.787+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0004030 con 0x7fe898004390 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.787+0000 7fe8aeffd640 1 -- 172.21.15.82:0/999159360 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0004400 con 0x7fe898004390 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.788+0000 7fe8b69b3640 1 -- 172.21.15.82:0/999159360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe898004390 msgr2=0x7fe898004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.788+0000 7fe8b69b3640 1 --2- 172.21.15.82:0/999159360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe898004390 0x7fe898004860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00a1d20 tx=0x7fe898005830 comp rx=0 tx=0).stop 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.788+0000 7fe8b4f50640 1 -- 172.21.15.82:0/999159360 reap_dead start 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.788+0000 7fe8b69b3640 1 -- 172.21.15.82:0/999159360 shutdown_connections 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.788+0000 7fe8b69b3640 1 -- 172.21.15.82:0/999159360 >> 172.21.15.82:0/999159360 conn(0x7fe8b009d070 msgr2=0x7fe8b00d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.788+0000 7fe8b69b3640 1 -- 172.21.15.82:0/999159360 shutdown_connections 2024-03-14T12:13:44.791 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.788+0000 7fe8b69b3640 1 -- 172.21.15.82:0/999159360 wait complete. 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.789+0000 7fe8b69b3640 1 Processor -- start 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.789+0000 7fe8b69b3640 1 -- start start 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b69b3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe898004390 0x7fe8b0158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b69b3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00c9e90 0x7fe8b00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b69b3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b00d2230 0x7fe8b00d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b69b3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b3b00 con 0x7fe8b00d2230 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b69b3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d10f0 con 0x7fe898004390 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b69b3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b3c80 con 0x7fe8b00c9e90 2024-03-14T12:13:44.794 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b00d2230 0x7fe8b00d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b00d2230 0x7fe8b00d26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33872/0 (socket says 172.21.15.82:33872) 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 -- 172.21.15.82:0/1772371192 learned_addr learned my addr 172.21.15.82:0/1772371192 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00c9e90 0x7fe8b00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8affff640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe898004390 0x7fe8b0158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 -- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00c9e90 msgr2=0x7fe8b00c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00c9e90 0x7fe8b00c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 -- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe898004390 msgr2=0x7fe8b0158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe898004390 0x7fe8b0158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 -- 172.21.15.82:0/1772371192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00ca620 con 0x7fe8b00d2230 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8b4f50640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b00d2230 0x7fe8b00d26b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8b008d680 tx=0x7fe8a8007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.790+0000 7fe8acff9640 1 -- 172.21.15.82:0/1772371192 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800ac30 con 0x7fe8b00d2230 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.791+0000 7fe8acff9640 1 -- 172.21.15.82:0/1772371192 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8019070 con 0x7fe8b00d2230 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.791+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b00ca900 con 0x7fe8b00d2230 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.791+0000 7fe8acff9640 1 -- 172.21.15.82:0/1772371192 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8013630 con 0x7fe8b00d2230 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.791+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b00b7c40 con 0x7fe8b00d2230 2024-03-14T12:13:44.795 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.792+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b004f050 con 0x7fe8b00d2230 2024-03-14T12:13:44.796 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.792+0000 7fe8acff9640 1 -- 172.21.15.82:0/1772371192 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe8a80137d0 con 0x7fe8b00d2230 2024-03-14T12:13:44.796 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.792+0000 7fe8acff9640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bc00 0x7fe89c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:44.796 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.792+0000 7fe8affff640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bc00 0x7fe89c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:44.796 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.793+0000 7fe8acff9640 1 -- 172.21.15.82:0/1772371192 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe8a808ada0 con 0x7fe8b00d2230 2024-03-14T12:13:44.796 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.793+0000 7fe8affff640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bc00 0x7fe89c06e0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000c00 tx=0x7fe8a4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:44.797 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.797+0000 7fe8acff9640 1 -- 172.21.15.82:0/1772371192 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe8a8059be0 con 0x7fe8b00d2230 2024-03-14T12:13:44.971 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.971+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8b00d1480 con 0x7fe8b00d2230 2024-03-14T12:13:44.972 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.972+0000 7fe8acff9640 1 -- 172.21.15.82:0/1772371192 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12872 (secure 0 0 0) 0x7fe8a80594e0 con 0x7fe8b00d2230 2024-03-14T12:13:44.972 INFO:teuthology.misc.health.smithi082.stdout: 2024-03-14T12:13:44.972 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":17,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:42.520066+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:44.974 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bc00 msgr2=0x7fe89c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bc00 0x7fe89c06e0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000c00 tx=0x7fe8a4009040 comp rx=0 tx=0).stop 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b00d2230 msgr2=0x7fe8b00d26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b00d2230 0x7fe8b00d26b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8b008d680 tx=0x7fe8a8007bd0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 shutdown_connections 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b00d2230 0x7fe8b00d26b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bc00 0x7fe89c06e0c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b00c9e90 0x7fe8b00c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 --2- 172.21.15.82:0/1772371192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe898004390 0x7fe8b0158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 >> 172.21.15.82:0/1772371192 conn(0x7fe8b009d070 msgr2=0x7fe8b006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 shutdown_connections 2024-03-14T12:13:44.975 INFO:teuthology.misc.health.smithi082.stderr:2024-03-14T12:13:44.974+0000 7fe8b69b3640 1 -- 172.21.15.82:0/1772371192 wait complete. 2024-03-14T12:13:44.984 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-14T12:13:44.984 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-14T12:13:44.984 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T12:13:44.985 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T12:13:44.987 INFO:tasks.ceph.mds.a:Started 2024-03-14T12:13:44.987 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T12:13:44.987 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T12:13:44.989 INFO:tasks.ceph.mds.c:Started 2024-03-14T12:13:44.989 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T12:13:44.989 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T12:13:44.992 INFO:tasks.ceph.mds.b:Started 2024-03-14T12:13:44.992 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T12:13:44.992 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T12:13:44.994 INFO:tasks.ceph.mds.d:Started 2024-03-14T12:13:44.994 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-14T12:13:44.995 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:45.062 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-14T12:13:45.062 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:13:45.063 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-03-14T12:13:45.063+0000 7fc9ebe3e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-14T12:13:45.077 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-03-14T12:13:45.077 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-03-14T12:13:45.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.099+0000 7fca8735b640 1 Processor -- start 2024-03-14T12:13:45.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.099+0000 7fca8735b640 1 -- start start 2024-03-14T12:13:45.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca8735b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fca800cf370 0x7fca800d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca8735b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca800b7720 0x7fca800b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca8735b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca800cf8c0 con 0x7fca800d0760 2024-03-14T12:13:45.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca8735b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800cfba0 con 0x7fca800b7720 2024-03-14T12:13:45.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca8735b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800b6c40 con 0x7fca800cf370 2024-03-14T12:13:45.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca858f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca800b7720 0x7fca800b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca858f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca800b7720 0x7fca800b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33894/0 (socket says 172.21.15.82:33894) 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca858f8640 1 -- 172.21.15.82:0/3211108053 learned_addr learned my addr 172.21.15.82:0/3211108053 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.100+0000 7fca848f6640 1 --2- 172.21.15.82:0/3211108053 >> v2:172.21.15.164:3300/0 conn(0x7fca800cf370 0x7fca800d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca858f8640 1 -- 172.21.15.82:0/3211108053 >> v2:172.21.15.164:3300/0 conn(0x7fca800cf370 msgr2=0x7fca800d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca858f8640 1 --2- 172.21.15.82:0/3211108053 >> v2:172.21.15.164:3300/0 conn(0x7fca800cf370 0x7fca800d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca858f8640 1 -- 172.21.15.82:0/3211108053 >> v1:172.21.15.82:6789/0 conn(0x7fca800d0760 legacy=0x7fca800d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca858f8640 1 -- 172.21.15.82:0/3211108053 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800b6f20 con 0x7fca800b7720 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca848f6640 1 --2- 172.21.15.82:0/3211108053 >> v2:172.21.15.164:3300/0 conn(0x7fca800cf370 0x7fca800d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca858f8640 1 --2- 172.21.15.82:0/3211108053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800b7720 0x7fca800b7b70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca78009e50 tx=0x7fca7802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1034278f5f836cd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca77fff640 1 -- 172.21.15.82:0/3211108053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7802cbb0 con 0x7fca800b7720 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca77fff640 1 -- 172.21.15.82:0/3211108053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7802cd50 con 0x7fca800b7720 2024-03-14T12:13:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.101+0000 7fca77fff640 1 -- 172.21.15.82:0/3211108053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78034610 con 0x7fca800b7720 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 -- 172.21.15.82:0/3211108053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800b7720 msgr2=0x7fca800b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 --2- 172.21.15.82:0/3211108053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800b7720 0x7fca800b7b70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca78009e50 tx=0x7fca7802c010 comp rx=0 tx=0).stop 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 -- 172.21.15.82:0/3211108053 shutdown_connections 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 --2- 172.21.15.82:0/3211108053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800b7720 0x7fca800b7b70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 --2- 172.21.15.82:0/3211108053 >> v2:172.21.15.164:3300/0 conn(0x7fca800cf370 0x7fca800d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 -- 172.21.15.82:0/3211108053 >> 172.21.15.82:0/3211108053 conn(0x7fca8009d030 msgr2=0x7fca800d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 -- 172.21.15.82:0/3211108053 shutdown_connections 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.102+0000 7fca8735b640 1 -- 172.21.15.82:0/3211108053 wait complete. 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.103+0000 7fca8735b640 1 Processor -- start 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.103+0000 7fca8735b640 1 -- start start 2024-03-14T12:13:45.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca8735b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca800b7720 0x7fca800b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca8735b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800cf370 0x7fca801f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca8735b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca800d0760 0x7fca800c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca8735b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca801617a0 con 0x7fca800cf370 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca8735b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800d4ba0 con 0x7fca800b7720 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca8735b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca80161920 con 0x7fca800d0760 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800cf370 0x7fca801f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800cf370 0x7fca801f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33904/0 (socket says 172.21.15.82:33904) 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 -- 172.21.15.82:0/2083141050 learned_addr learned my addr 172.21.15.82:0/2083141050 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca850f7640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca800b7720 0x7fca800b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca858f8640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca800d0760 0x7fca800c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 -- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca800d0760 msgr2=0x7fca800c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca800d0760 0x7fca800c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 -- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca800b7720 msgr2=0x7fca800b8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca800b7720 0x7fca800b8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 -- 172.21.15.82:0/2083141050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800c8fb0 con 0x7fca800cf370 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca858f8640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca800d0760 0x7fca800c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.104+0000 7fca848f6640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800cf370 0x7fca801f3c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca7000c7f0 tx=0x7fca7000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.105+0000 7fca75ffb640 1 -- 172.21.15.82:0/2083141050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70015070 con 0x7fca800cf370 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.105+0000 7fca75ffb640 1 -- 172.21.15.82:0/2083141050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca70007a30 con 0x7fca800cf370 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.105+0000 7fca75ffb640 1 -- 172.21.15.82:0/2083141050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70007e00 con 0x7fca800cf370 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.105+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca800c9290 con 0x7fca800cf370 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.105+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca800cafe0 con 0x7fca800cf370 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.106+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8004f050 con 0x7fca800cf370 2024-03-14T12:13:45.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.106+0000 7fca75ffb640 1 -- 172.21.15.82:0/2083141050 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fca70018070 con 0x7fca800cf370 2024-03-14T12:13:45.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.106+0000 7fca75ffb640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca6406bbd0 0x7fca6406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.107+0000 7fca75ffb640 1 -- 172.21.15.82:0/2083141050 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fca7008a640 con 0x7fca800cf370 2024-03-14T12:13:45.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.107+0000 7fca850f7640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca6406bbd0 0x7fca6406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.107+0000 7fca850f7640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca6406bbd0 0x7fca6406e090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0016e0 tx=0x7fca7c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.111+0000 7fca75ffb640 1 -- 172.21.15.82:0/2083141050 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fca70058e30 con 0x7fca800cf370 2024-03-14T12:13:45.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.284+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca800d0540 con 0x7fca800cf370 2024-03-14T12:13:45.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.285+0000 7fca75ffb640 1 -- 172.21.15.82:0/2083141050 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12872 (secure 0 0 0) 0x7fca7005ced0 con 0x7fca800cf370 2024-03-14T12:13:45.285 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:45.285 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:42.520066+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:45.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca6406bbd0 msgr2=0x7fca6406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca6406bbd0 0x7fca6406e090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0016e0 tx=0x7fca7c009040 comp rx=0 tx=0).stop 2024-03-14T12:13:45.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800cf370 msgr2=0x7fca801f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800cf370 0x7fca801f3c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca7000c7f0 tx=0x7fca7000cbf0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 shutdown_connections 2024-03-14T12:13:45.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca6406bbd0 0x7fca6406e090 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca800d0760 0x7fca800c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca800cf370 0x7fca801f3c10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 --2- 172.21.15.82:0/2083141050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca800b7720 0x7fca800b8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 >> 172.21.15.82:0/2083141050 conn(0x7fca8009d030 msgr2=0x7fca8015e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:45.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 shutdown_connections 2024-03-14T12:13:45.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.287+0000 7fca8735b640 1 -- 172.21.15.82:0/2083141050 wait complete. 2024-03-14T12:13:45.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:13:45.298 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:13:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.402+0000 7ffbcb709640 1 Processor -- start 2024-03-14T12:13:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.402+0000 7ffbcb709640 1 -- start start 2024-03-14T12:13:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbcb709640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc40d08c0 0x7ffbc40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbcb709640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ffbc40b0bc0 0x7ffbc40ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbcb709640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc40cf440 con 0x7ffbc415e0c0 2024-03-14T12:13:45.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbcb709640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40cf1e0 con 0x7ffbc40b0bc0 2024-03-14T12:13:45.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbcb709640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40ccb60 con 0x7ffbc40d08c0 2024-03-14T12:13:45.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc9ca6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ffbc40b0bc0 0x7ffbc40ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc94a5640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ffbc415e0c0 0x7ffbc40d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:46690/0 (socket says 172.21.15.82:46690) 2024-03-14T12:13:45.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc8ca4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc40d08c0 0x7ffbc40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc94a5640 1 -- 172.21.15.82:0/2276851374 learned_addr learned my addr 172.21.15.82:0/2276851374 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbbbfff640 1 -- 172.21.15.82:0/2276851374 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553938047 0 0) 0x7ffbc40cf440 con 0x7ffbc415e0c0 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbbbfff640 1 -- 172.21.15.82:0/2276851374 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb0003520 con 0x7ffbc415e0c0 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc9ca6640 1 -- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc40d08c0 msgr2=0x7ffbc40cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc9ca6640 1 --2- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc40d08c0 0x7ffbc40cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc9ca6640 1 -- 172.21.15.82:0/2276851374 >> v1:172.21.15.164:6789/0 conn(0x7ffbc415e0c0 legacy=0x7ffbc40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc9ca6640 1 -- 172.21.15.82:0/2276851374 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40cce40 con 0x7ffbc40b0bc0 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.403+0000 7ffbc8ca4640 1 --2- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc40d08c0 0x7ffbc40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.404+0000 7ffbc9ca6640 1 --2- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 0x7ffbc40ceca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009d90 tx=0x7ffbbc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f871a7b979b23919 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.404+0000 7ffbbbfff640 1 -- 172.21.15.82:0/2276851374 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc039070 con 0x7ffbc40b0bc0 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.404+0000 7ffbbbfff640 1 -- 172.21.15.82:0/2276851374 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc02cc50 con 0x7ffbc40b0bc0 2024-03-14T12:13:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.404+0000 7ffbbbfff640 1 -- 172.21.15.82:0/2276851374 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc034440 con 0x7ffbc40b0bc0 2024-03-14T12:13:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 -- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 msgr2=0x7ffbc40ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 --2- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 0x7ffbc40ceca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009d90 tx=0x7ffbbc02bea0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 -- 172.21.15.82:0/2276851374 shutdown_connections 2024-03-14T12:13:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 --2- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 0x7ffbc40ceca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 --2- 172.21.15.82:0/2276851374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc40d08c0 0x7ffbc40cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 -- 172.21.15.82:0/2276851374 >> 172.21.15.82:0/2276851374 conn(0x7ffbc40d1bd0 msgr2=0x7ffbc40d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 -- 172.21.15.82:0/2276851374 shutdown_connections 2024-03-14T12:13:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.405+0000 7ffbcb709640 1 -- 172.21.15.82:0/2276851374 wait complete. 2024-03-14T12:13:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.406+0000 7ffbcb709640 1 Processor -- start 2024-03-14T12:13:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.406+0000 7ffbcb709640 1 -- start start 2024-03-14T12:13:45.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbcb709640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 0x7ffbc41570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbcb709640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc40d08c0 0x7ffbc40b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbcb709640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc415e0c0 0x7ffbc40b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbcb709640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc415e4a0 con 0x7ffbc40d08c0 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbcb709640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40cd400 con 0x7ffbc40b0bc0 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbcb709640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc406bbc0 con 0x7ffbc415e0c0 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc40d08c0 0x7ffbc40b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc40d08c0 0x7ffbc40b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33912/0 (socket says 172.21.15.82:33912) 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 -- 172.21.15.82:0/3132206304 learned_addr learned my addr 172.21.15.82:0/3132206304 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc94a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 0x7ffbc41570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc9ca6640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc415e0c0 0x7ffbc40b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 -- 172.21.15.82:0/3132206304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc415e0c0 msgr2=0x7ffbc40b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc415e0c0 0x7ffbc40b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 -- 172.21.15.82:0/3132206304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 msgr2=0x7ffbc41570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 0x7ffbc41570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.407+0000 7ffbc8ca4640 1 -- 172.21.15.82:0/3132206304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4157600 con 0x7ffbc40d08c0 2024-03-14T12:13:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.408+0000 7ffbc8ca4640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc40d08c0 0x7ffbc40b5d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400ebe0 tx=0x7ffbb400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.408+0000 7ffbb9ffb640 1 -- 172.21.15.82:0/3132206304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4017070 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.408+0000 7ffbb9ffb640 1 -- 172.21.15.82:0/3132206304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb400cc30 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.408+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc409f1a0 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.408+0000 7ffbb9ffb640 1 -- 172.21.15.82:0/3132206304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4013470 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.408+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc409f760 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.409+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc404f050 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.409+0000 7ffbb9ffb640 1 -- 172.21.15.82:0/3132206304 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ffbb401a070 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.409+0000 7ffbb9ffb640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffba806bbd0 0x7ffba806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.410+0000 7ffbb9ffb640 1 -- 172.21.15.82:0/3132206304 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffbb408a600 con 0x7ffbc40d08c0 2024-03-14T12:13:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.410+0000 7ffbc94a5640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffba806bbd0 0x7ffba806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.410+0000 7ffbc94a5640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffba806bbd0 0x7ffba806e090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0005880 tx=0x7ffbc0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.414+0000 7ffbb9ffb640 1 -- 172.21.15.82:0/3132206304 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ffbc4157600 con 0x7ffbc40d08c0 2024-03-14T12:13:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.588+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbc40b7b50 con 0x7ffbc40d08c0 2024-03-14T12:13:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.588+0000 7ffbb9ffb640 1 -- 172.21.15.82:0/3132206304 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbb4058e00 con 0x7ffbc40d08c0 2024-03-14T12:13:45.589 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:13:45.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffba806bbd0 msgr2=0x7ffba806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffba806bbd0 0x7ffba806e090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0005880 tx=0x7ffbc0009040 comp rx=0 tx=0).stop 2024-03-14T12:13:45.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc40d08c0 msgr2=0x7ffbc40b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc40d08c0 0x7ffbc40b5d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400ebe0 tx=0x7ffbb400c520 comp rx=0 tx=0).stop 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 shutdown_connections 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffba806bbd0 0x7ffba806e090 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffbc415e0c0 0x7ffbc40b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc40d08c0 0x7ffbc40b5d60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 --2- 172.21.15.82:0/3132206304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc40b0bc0 0x7ffbc41570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 >> 172.21.15.82:0/3132206304 conn(0x7ffbc40d1bd0 msgr2=0x7ffbc406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 shutdown_connections 2024-03-14T12:13:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.591+0000 7ffbcb709640 1 -- 172.21.15.82:0/3132206304 wait complete. 2024-03-14T12:13:45.602 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:13:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.702+0000 7f6fa8880640 1 Processor -- start 2024-03-14T12:13:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.702+0000 7f6fa8880640 1 -- start start 2024-03-14T12:13:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.703+0000 7f6fa8880640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 0x7f6fa4071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.703+0000 7f6fa8880640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa40b3050 con 0x7f6fa415e670 2024-03-14T12:13:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.703+0000 7f6fa8880640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa4072060 con 0x7f6fa4143f60 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.703+0000 7f6fa8880640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa415ebe0 con 0x7f6fa4071740 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa1d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6fa415e670 0x7f6fa40b2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59090/0 (socket says 172.21.15.82:59090) 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa1d9b640 1 -- 172.21.15.82:0/1592433764 learned_addr learned my addr 172.21.15.82:0/1592433764 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa259c640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 0x7f6fa4071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127626318 0 0) 0x7f6fa40b3050 con 0x7f6fa415e670 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f90003520 con 0x7f6fa415e670 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216463294 0 0) 0x7f6f90003520 con 0x7f6fa415e670 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 msgr2=0x7f6fa4071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 0x7f6fa4071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 >> v1:172.21.15.164:6789/0 conn(0x7f6fa4143f60 legacy=0x7f6fa415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa415eec0 con 0x7f6fa415e670 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6f8c001670 con 0x7f6fa415e670 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 >> v1:172.21.15.82:6789/0 conn(0x7f6fa415e670 legacy=0x7f6fa40b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 0x7f6f900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90003f20 0x7f6f90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa415ebe0 con 0x7f6f90004d40 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9000ba20 con 0x7f6f90003f20 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9000de30 con 0x7f6fa4071740 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa2d9d640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa1d9b640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90003f20 0x7f6f90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa259c640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 0x7f6f900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.704+0000 7f6fa2d9d640 1 -- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 msgr2=0x7f6f900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa2d9d640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa4071740 0x7f6f900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa2d9d640 1 -- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90003f20 msgr2=0x7f6f90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa2d9d640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90003f20 0x7f6f90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa2d9d640 1 -- 172.21.15.82:0/1592433764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90003520 con 0x7f6f90004d40 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa2d9d640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f94001f60 tx=0x7f6f90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9368f60754f64b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94001b30 con 0x7f6f90004d40 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9402d710 con 0x7f6f90004d40 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.705+0000 7f6fa159a640 1 -- 172.21.15.82:0/1592433764 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9402db00 con 0x7f6f90004d40 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.706+0000 7f6fa8880640 1 -- 172.21.15.82:0/1592433764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90004d40 msgr2=0x7f6f900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.706+0000 7f6fa8880640 1 --2- 172.21.15.82:0/1592433764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f94001f60 tx=0x7f6f90005830 comp rx=0 tx=0).stop 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.706+0000 7f6fa2d9d640 1 -- 172.21.15.82:0/1592433764 reap_dead start 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.706+0000 7f6fa8880640 1 -- 172.21.15.82:0/1592433764 shutdown_connections 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.706+0000 7f6fa8880640 1 -- 172.21.15.82:0/1592433764 >> 172.21.15.82:0/1592433764 conn(0x7f6fa40b1c70 msgr2=0x7f6fa40cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.706+0000 7f6fa8880640 1 -- 172.21.15.82:0/1592433764 shutdown_connections 2024-03-14T12:13:45.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.706+0000 7f6fa8880640 1 -- 172.21.15.82:0/1592433764 wait complete. 2024-03-14T12:13:45.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.707+0000 7f6fa8880640 1 Processor -- start 2024-03-14T12:13:45.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.707+0000 7f6fa8880640 1 -- start start 2024-03-14T12:13:45.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa8880640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90004d40 0x7f6fa413a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa8880640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa41edce0 0x7f6fa4139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa8880640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40cde70 0x7f6fa41f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa8880640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4143ce0 con 0x7f6fa41edce0 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa8880640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa415f480 con 0x7f6f90004d40 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa8880640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa415b080 con 0x7f6fa40cde70 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa41edce0 0x7f6fa4139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa41edce0 0x7f6fa4139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33922/0 (socket says 172.21.15.82:33922) 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 -- 172.21.15.82:0/3266369036 learned_addr learned my addr 172.21.15.82:0/3266369036 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa259c640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90004d40 0x7f6fa413a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa2d9d640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40cde70 0x7f6fa41f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 -- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40cde70 msgr2=0x7f6fa41f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40cde70 0x7f6fa41f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 -- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90004d40 msgr2=0x7f6fa413a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90004d40 0x7f6fa413a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa1d9b640 1 -- 172.21.15.82:0/3266369036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa40ccdd0 con 0x7f6fa41edce0 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa259c640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90004d40 0x7f6fa413a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:45.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.708+0000 7f6fa2d9d640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40cde70 0x7f6fa41f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.709+0000 7f6fa1d9b640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa41edce0 0x7f6fa4139940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c007340 tx=0x7f6f8c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.709+0000 7f6f9affd640 1 -- 172.21.15.82:0/3266369036 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00f860 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.709+0000 7f6f9affd640 1 -- 172.21.15.82:0/3266369036 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c018070 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.709+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa40cd0b0 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.709+0000 7f6f9affd640 1 -- 172.21.15.82:0/3266369036 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00fc80 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.709+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa40cd670 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.710+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa404f050 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.710+0000 7f6f9affd640 1 -- 172.21.15.82:0/3266369036 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6f8c01d460 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.710+0000 7f6f9affd640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f8406bbd0 0x7f6f8406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.710+0000 7f6f9affd640 1 -- 172.21.15.82:0/3266369036 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f8c08a7b0 con 0x7f6fa41edce0 2024-03-14T12:13:45.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.710+0000 7f6fa259c640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f8406bbd0 0x7f6f8406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:45.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.711+0000 7f6fa259c640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f8406bbd0 0x7f6f8406e090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40b3000 tx=0x7f6f9c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:45.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.715+0000 7f6f9affd640 1 -- 172.21.15.82:0/3266369036 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6f8c059680 con 0x7f6fa41edce0 2024-03-14T12:13:45.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.922+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fa40d07b0 con 0x7f6fa41edce0 2024-03-14T12:13:45.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.922+0000 7f6f9affd640 1 -- 172.21.15.82:0/3266369036 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f6f8c058f80 con 0x7f6fa41edce0 2024-03-14T12:13:45.923 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:45.923 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-14T12:13:45.923 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f8406bbd0 msgr2=0x7f6f8406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f8406bbd0 0x7f6f8406e090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40b3000 tx=0x7f6f9c008040 comp rx=0 tx=0).stop 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa41edce0 msgr2=0x7f6fa4139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa41edce0 0x7f6fa4139940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c007340 tx=0x7f6f8c007800 comp rx=0 tx=0).stop 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 shutdown_connections 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f8406bbd0 0x7f6f8406e090 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40cde70 0x7f6fa41f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa41edce0 0x7f6fa4139940 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 --2- 172.21.15.82:0/3266369036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f90004d40 0x7f6fa413a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 >> 172.21.15.82:0/3266369036 conn(0x7f6fa40b1c70 msgr2=0x7f6fa4141c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 shutdown_connections 2024-03-14T12:13:45.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:45.924+0000 7f6fa8880640 1 -- 172.21.15.82:0/3266369036 wait complete. 2024-03-14T12:13:47.936 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:13:48.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.048+0000 7fe4ec497640 1 Processor -- start 2024-03-14T12:13:48.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.048+0000 7fe4ec497640 1 -- start start 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4ec497640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 0x7fe4e415e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4ec497640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40b3da0 con 0x7fe4e4151230 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4ec497640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40b4080 con 0x7fe4e40b4b70 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4ec497640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e4070d40 con 0x7fe4e415dde0 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4ea233640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe4e4151230 0x7fe4e40cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59096/0 (socket says 172.21.15.82:59096) 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4ea233640 1 -- 172.21.15.82:0/587454612 learned_addr learned my addr 172.21.15.82:0/587454612 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018331868 0 0) 0x7fe4e40b3da0 con 0x7fe4e4151230 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d0003520 con 0x7fe4e4151230 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4eaa34640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 0x7fe4e415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088358935 0 0) 0x7fe4d0003520 con 0x7fe4e4151230 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 msgr2=0x7fe4e415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 0x7fe4e415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 >> v1:172.21.15.164:6789/0 conn(0x7fe4e40b4b70 legacy=0x7fe4e4070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40b4360 con 0x7fe4e4151230 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.049+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fe4e0001670 con 0x7fe4e4151230 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 >> v1:172.21.15.82:6789/0 conn(0x7fe4e4151230 legacy=0x7fe4e40cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9231640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9231640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 0x7fe4d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9231640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4d0004d40 0x7fe4d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e4070d40 con 0x7fe4d00039e0 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d000ba70 con 0x7fe4d0004d40 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d000bc70 con 0x7fe4e415dde0 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4ea233640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4e9a32640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 0x7fe4d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4eaa34640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4d0004d40 0x7fe4d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4ea233640 1 -- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 msgr2=0x7fe4d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4ea233640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e415dde0 0x7fe4d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4ea233640 1 -- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4d0004d40 msgr2=0x7fe4d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4ea233640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4d0004d40 0x7fe4d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4ea233640 1 -- 172.21.15.82:0/587454612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0003520 con 0x7fe4d00039e0 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.050+0000 7fe4eaa34640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4d0004d40 0x7fe4d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4ea233640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4d0003e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0001050 tx=0x7fe4d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2668022853b9d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0007d10 con 0x7fe4d00039e0 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0030070 con 0x7fe4d00039e0 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4e9231640 1 -- 172.21.15.82:0/587454612 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0005d30 con 0x7fe4d00039e0 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4ec497640 1 -- 172.21.15.82:0/587454612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 msgr2=0x7fe4d0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4ec497640 1 --2- 172.21.15.82:0/587454612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4d0003e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0001050 tx=0x7fe4d0005830 comp rx=0 tx=0).stop 2024-03-14T12:13:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4eaa34640 1 -- 172.21.15.82:0/587454612 reap_dead start 2024-03-14T12:13:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4ec497640 1 -- 172.21.15.82:0/587454612 shutdown_connections 2024-03-14T12:13:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4ec497640 1 -- 172.21.15.82:0/587454612 >> 172.21.15.82:0/587454612 conn(0x7fe4e40b1c70 msgr2=0x7fe4e40cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4ec497640 1 -- 172.21.15.82:0/587454612 shutdown_connections 2024-03-14T12:13:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.051+0000 7fe4ec497640 1 -- 172.21.15.82:0/587454612 wait complete. 2024-03-14T12:13:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 Processor -- start 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 -- start start 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4e41586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e4144d40 0x7fe4e41f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e4140bd0 0x7fe4e4153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e406c3b0 con 0x7fe4d00039e0 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e415e790 con 0x7fe4e4140bd0 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ec497640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e406c690 con 0x7fe4e4144d40 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ea233640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4e41586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ea233640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4e41586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33942/0 (socket says 172.21.15.82:33942) 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.053+0000 7fe4ea233640 1 -- 172.21.15.82:0/1234309893 learned_addr learned my addr 172.21.15.82:0/1234309893 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4eaa34640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e4140bd0 0x7fe4e4153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ea233640 1 -- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e4144d40 msgr2=0x7fe4e41f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4e9a32640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e4144d40 0x7fe4e41f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ea233640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e4144d40 0x7fe4e41f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ea233640 1 -- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e4140bd0 msgr2=0x7fe4e4153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ea233640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e4140bd0 0x7fe4e4153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ea233640 1 -- 172.21.15.82:0/1234309893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e41415f0 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4eaa34640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e4140bd0 0x7fe4e4153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4e9a32640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e4144d40 0x7fe4e41f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ea233640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4e41586b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e4154db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4daffd640 1 -- 172.21.15.82:0/1234309893 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0007d10 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4daffd640 1 -- 172.21.15.82:0/1234309893 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0004bf0 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e41418d0 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4daffd640 1 -- 172.21.15.82:0/1234309893 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0001c70 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.054+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e41f51a0 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.055+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e404f050 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.055+0000 7fe4daffd640 1 -- 172.21.15.82:0/1234309893 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe4e0007070 con 0x7fe4d00039e0 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.056+0000 7fe4daffd640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4c806bc00 0x7fe4c806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.056+0000 7fe4e9a32640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4c806bc00 0x7fe4c806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.056+0000 7fe4daffd640 1 -- 172.21.15.82:0/1234309893 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe4e00ab5e0 con 0x7fe4d00039e0 2024-03-14T12:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.056+0000 7fe4e9a32640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4c806bc00 0x7fe4c806e0c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4145120 tx=0x7fe4d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.060+0000 7fe4daffd640 1 -- 172.21.15.82:0/1234309893 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe4e007a4b0 con 0x7fe4d00039e0 2024-03-14T12:13:48.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.267+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4e40cbde0 con 0x7fe4d00039e0 2024-03-14T12:13:48.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.268+0000 7fe4daffd640 1 -- 172.21.15.82:0/1234309893 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7fe4e007ee20 con 0x7fe4d00039e0 2024-03-14T12:13:48.269 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:48.269 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-14T12:13:48.269 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2024-03-14T12:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4c806bc00 msgr2=0x7fe4c806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4c806bc00 0x7fe4c806e0c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4145120 tx=0x7fe4d4009040 comp rx=0 tx=0).stop 2024-03-14T12:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 msgr2=0x7fe4e41586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4e41586b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e4154db0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 shutdown_connections 2024-03-14T12:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e4140bd0 0x7fe4e4153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4c806bc00 0x7fe4c806e0c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e4144d40 0x7fe4e41f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 --2- 172.21.15.82:0/1234309893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d00039e0 0x7fe4e41586b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 >> 172.21.15.82:0/1234309893 conn(0x7fe4e40b1c70 msgr2=0x7fe4e41501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 shutdown_connections 2024-03-14T12:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.270+0000 7fe4ec497640 1 -- 172.21.15.82:0/1234309893 wait complete. 2024-03-14T12:13:48.281 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:48.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.381+0000 7f51e362b640 1 Processor -- start 2024-03-14T12:13:48.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.381+0000 7f51e362b640 1 -- start start 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e362b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51dc0d05e0 0x7f51dc0d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e362b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51dc0d09c0 0x7f51dc0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e362b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc0cf270 con 0x7f51dc0d1890 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e362b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0b08f0 con 0x7f51dc0d05e0 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e362b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0cf4f0 con 0x7f51dc0d09c0 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e13c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51dc0d05e0 0x7f51dc0d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e13c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51dc0d05e0 0x7f51dc0d47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33952/0 (socket says 172.21.15.82:33952) 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e13c7640 1 -- 172.21.15.82:0/1008530408 learned_addr learned my addr 172.21.15.82:0/1008530408 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e13c7640 1 -- 172.21.15.82:0/1008530408 >> v2:172.21.15.164:3300/0 conn(0x7f51dc0d09c0 msgr2=0x7f51dc0d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e0bc6640 1 --2- 172.21.15.82:0/1008530408 >> v2:172.21.15.164:3300/0 conn(0x7f51dc0d09c0 0x7f51dc0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e13c7640 1 --2- 172.21.15.82:0/1008530408 >> v2:172.21.15.164:3300/0 conn(0x7f51dc0d09c0 0x7f51dc0d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e13c7640 1 -- 172.21.15.82:0/1008530408 >> v1:172.21.15.82:6789/0 conn(0x7f51dc0d1890 legacy=0x7f51dc0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e13c7640 1 -- 172.21.15.82:0/1008530408 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0cf7d0 con 0x7f51dc0d05e0 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.382+0000 7f51e0bc6640 1 --2- 172.21.15.82:0/1008530408 >> v2:172.21.15.164:3300/0 conn(0x7f51dc0d09c0 0x7f51dc0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.383+0000 7f51e13c7640 1 --2- 172.21.15.82:0/1008530408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d05e0 0x7f51dc0d47e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0c75f0 tx=0x7f51d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dfb38c818cad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.383+0000 7f51d3fff640 1 -- 172.21.15.82:0/1008530408 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8035020 con 0x7f51dc0d05e0 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.383+0000 7f51d3fff640 1 -- 172.21.15.82:0/1008530408 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8031870 con 0x7f51dc0d05e0 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.383+0000 7f51d3fff640 1 -- 172.21.15.82:0/1008530408 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8031c40 con 0x7f51dc0d05e0 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.383+0000 7f51e362b640 1 -- 172.21.15.82:0/1008530408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d05e0 msgr2=0x7f51dc0d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.383+0000 7f51e362b640 1 --2- 172.21.15.82:0/1008530408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d05e0 0x7f51dc0d47e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0c75f0 tx=0x7f51d802b910 comp rx=0 tx=0).stop 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.384+0000 7f51e362b640 1 -- 172.21.15.82:0/1008530408 shutdown_connections 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.384+0000 7f51e362b640 1 --2- 172.21.15.82:0/1008530408 >> v2:172.21.15.164:3300/0 conn(0x7f51dc0d09c0 0x7f51dc0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.384+0000 7f51e362b640 1 --2- 172.21.15.82:0/1008530408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d05e0 0x7f51dc0d47e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.384+0000 7f51e362b640 1 -- 172.21.15.82:0/1008530408 >> 172.21.15.82:0/1008530408 conn(0x7f51dc0d3cc0 msgr2=0x7f51dc0d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.384+0000 7f51e362b640 1 -- 172.21.15.82:0/1008530408 shutdown_connections 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.384+0000 7f51e362b640 1 -- 172.21.15.82:0/1008530408 wait complete. 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 Processor -- start 2024-03-14T12:13:48.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 -- start start 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0d05e0 0x7f51dc1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d09c0 0x7f51dc1f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51dc0d1890 0x7f51dc154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0d1360 con 0x7f51dc0d09c0 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0d2bc0 con 0x7f51dc0d05e0 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.385+0000 7f51e362b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc161710 con 0x7f51dc0d1890 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d09c0 0x7f51dc1f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d09c0 0x7f51dc1f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33966/0 (socket says 172.21.15.82:33966) 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 -- 172.21.15.82:0/2795709761 learned_addr learned my addr 172.21.15.82:0/2795709761 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 -- 172.21.15.82:0/2795709761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51dc0d1890 msgr2=0x7f51dc154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51dc0d1890 0x7f51dc154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 -- 172.21.15.82:0/2795709761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0d05e0 msgr2=0x7f51dc1f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0d05e0 0x7f51dc1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 -- 172.21.15.82:0/2795709761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0cd6e0 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e0bc6640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d09c0 0x7f51dc1f6d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c820 tx=0x7f51cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51d1ffb640 1 -- 172.21.15.82:0/2795709761 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc015070 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51d1ffb640 1 -- 172.21.15.82:0/2795709761 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc007a30 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc0c82d0 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51d1ffb640 1 -- 172.21.15.82:0/2795709761 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc007e00 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.386+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc0c8890 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.387+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc04f050 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.387+0000 7f51d1ffb640 1 -- 172.21.15.82:0/2795709761 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f51cc018070 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.388+0000 7f51d1ffb640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f51c006bbd0 0x7f51c006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.388+0000 7f51d1ffb640 1 -- 172.21.15.82:0/2795709761 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f51cc08a730 con 0x7f51dc0d09c0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.388+0000 7f51e13c7640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f51c006bbd0 0x7f51c006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.388+0000 7f51e13c7640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f51c006bbd0 0x7f51c006e090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51d8036000 tx=0x7f51d8030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.392+0000 7f51d1ffb640 1 -- 172.21.15.82:0/2795709761 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f51cc059570 con 0x7f51dc0d09c0 2024-03-14T12:13:48.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.567+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51dc0b1060 con 0x7f51dc0d09c0 2024-03-14T12:13:48.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.567+0000 7f51d1ffb640 1 -- 172.21.15.82:0/2795709761 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f51cc058e70 con 0x7f51dc0d09c0 2024-03-14T12:13:48.567 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:48.568 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:42.520066+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:48.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f51c006bbd0 msgr2=0x7f51c006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f51c006bbd0 0x7f51c006e090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51d8036000 tx=0x7f51d8030000 comp rx=0 tx=0).stop 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d09c0 msgr2=0x7f51dc1f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d09c0 0x7f51dc1f6d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c820 tx=0x7f51cc00cc20 comp rx=0 tx=0).stop 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 shutdown_connections 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f51c006bbd0 0x7f51c006e090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51dc0d1890 0x7f51dc154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51dc0d09c0 0x7f51dc1f6d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 --2- 172.21.15.82:0/2795709761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51dc0d05e0 0x7f51dc1f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 >> 172.21.15.82:0/2795709761 conn(0x7f51dc0d3cc0 msgr2=0x7f51dc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 shutdown_connections 2024-03-14T12:13:48.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.570+0000 7f51e362b640 1 -- 172.21.15.82:0/2795709761 wait complete. 2024-03-14T12:13:48.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:13:48.581 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:13:48.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.683+0000 7feae5b3b640 1 Processor -- start 2024-03-14T12:13:48.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.683+0000 7feae5b3b640 1 -- start start 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feae5b3b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feae015f430 0x7feae00d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feae5b3b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feae006ef30 0x7feae00cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feae5b3b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae006e7a0 con 0x7feae00d1070 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feae5b3b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae006eaa0 con 0x7feae006ef30 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feae5b3b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae015bb80 con 0x7feae015f430 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feadf7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feae00d1070 0x7feae015b2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59112/0 (socket says 172.21.15.82:59112) 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feadf7fe640 1 -- 172.21.15.82:0/3546654937 learned_addr learned my addr 172.21.15.82:0/3546654937 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.684+0000 7feadffff640 1 --2- 172.21.15.82:0/3546654937 >> v2:172.21.15.82:3300/0 conn(0x7feae006ef30 0x7feae00cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.685+0000 7fead6ffd640 1 --2- 172.21.15.82:0/3546654937 >> v2:172.21.15.164:3300/0 conn(0x7feae015f430 0x7feae00d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.685+0000 7feadffff640 1 -- 172.21.15.82:0/3546654937 >> v2:172.21.15.164:3300/0 conn(0x7feae015f430 msgr2=0x7feae00d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.685+0000 7feadffff640 1 --2- 172.21.15.82:0/3546654937 >> v2:172.21.15.164:3300/0 conn(0x7feae015f430 0x7feae00d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.685+0000 7feadffff640 1 -- 172.21.15.82:0/3546654937 >> v1:172.21.15.82:6789/0 conn(0x7feae00d1070 legacy=0x7feae015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.685+0000 7feadffff640 1 -- 172.21.15.82:0/3546654937 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae015be60 con 0x7feae006ef30 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.685+0000 7feadffff640 1 --2- 172.21.15.82:0/3546654937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae00cca30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feae006df70 tx=0x7fead802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6df1ca148285814 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feadeffd640 1 -- 172.21.15.82:0/3546654937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8039070 con 0x7feae006ef30 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feadeffd640 1 -- 172.21.15.82:0/3546654937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead802cc80 con 0x7feae006ef30 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feadeffd640 1 -- 172.21.15.82:0/3546654937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8034660 con 0x7feae006ef30 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 -- 172.21.15.82:0/3546654937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 msgr2=0x7feae00cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 --2- 172.21.15.82:0/3546654937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae00cca30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feae006df70 tx=0x7fead802bed0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 -- 172.21.15.82:0/3546654937 shutdown_connections 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 --2- 172.21.15.82:0/3546654937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae00cca30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 --2- 172.21.15.82:0/3546654937 >> v2:172.21.15.164:3300/0 conn(0x7feae015f430 0x7feae00d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 -- 172.21.15.82:0/3546654937 >> 172.21.15.82:0/3546654937 conn(0x7feae006c000 msgr2=0x7feae006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 -- 172.21.15.82:0/3546654937 shutdown_connections 2024-03-14T12:13:48.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.686+0000 7feae5b3b640 1 -- 172.21.15.82:0/3546654937 wait complete. 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.687+0000 7feae5b3b640 1 Processor -- start 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feae5b3b640 1 -- start start 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feae5b3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae013e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feae5b3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d1070 0x7feae014c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feae5b3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae015f430 0x7feae013ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feae5b3b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae006f510 con 0x7feae006ef30 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feae5b3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae006f390 con 0x7feae015f430 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feae5b3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae00ccf70 con 0x7feae00d1070 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feadf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae013e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feadf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae013e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33984/0 (socket says 172.21.15.82:33984) 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.688+0000 7feadf7fe640 1 -- 172.21.15.82:0/1735200648 learned_addr learned my addr 172.21.15.82:0/1735200648 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7fead6ffd640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d1070 0x7feae014c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadffff640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae015f430 0x7feae013ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadf7fe640 1 -- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d1070 msgr2=0x7feae014c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadf7fe640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d1070 0x7feae014c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadf7fe640 1 -- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae015f430 msgr2=0x7feae013ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadf7fe640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae015f430 0x7feae013ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadf7fe640 1 -- 172.21.15.82:0/1735200648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae013ea60 con 0x7feae006ef30 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7fead6ffd640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d1070 0x7feae014c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadffff640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae015f430 0x7feae013ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadf7fe640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae013e520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fead000c400 tx=0x7fead000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadcff9640 1 -- 172.21.15.82:0/1735200648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0017070 con 0x7feae006ef30 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadcff9640 1 -- 172.21.15.82:0/1735200648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead0005ce0 con 0x7feae006ef30 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae014b690 con 0x7feae006ef30 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feadcff9640 1 -- 172.21.15.82:0/1735200648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00072f0 con 0x7feae006ef30 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.689+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae014bc50 con 0x7feae006ef30 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.690+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae004f050 con 0x7feae006ef30 2024-03-14T12:13:48.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.690+0000 7feadcff9640 1 -- 172.21.15.82:0/1735200648 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fead001a070 con 0x7feae006ef30 2024-03-14T12:13:48.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.691+0000 7feadcff9640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feac006bbd0 0x7feac006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.691+0000 7fead6ffd640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feac006bbd0 0x7feac006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.691+0000 7feadcff9640 1 -- 172.21.15.82:0/1735200648 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fead0089a90 con 0x7feae006ef30 2024-03-14T12:13:48.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.691+0000 7fead6ffd640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feac006bbd0 0x7feac006e090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feae00d1650 tx=0x7feac8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.695+0000 7feadcff9640 1 -- 172.21.15.82:0/1735200648 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fead0058960 con 0x7feae006ef30 2024-03-14T12:13:48.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.868+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feae006dfd0 con 0x7feae006ef30 2024-03-14T12:13:48.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.869+0000 7feadcff9640 1 -- 172.21.15.82:0/1735200648 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fead0058260 con 0x7feae006ef30 2024-03-14T12:13:48.869 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:13:48.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feac006bbd0 msgr2=0x7feac006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feac006bbd0 0x7feac006e090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feae00d1650 tx=0x7feac8008040 comp rx=0 tx=0).stop 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 msgr2=0x7feae013e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae013e520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fead000c400 tx=0x7fead000c8c0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 shutdown_connections 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae015f430 0x7feae013ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feac006bbd0 0x7feac006e090 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d1070 0x7feae014c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 --2- 172.21.15.82:0/1735200648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae006ef30 0x7feae013e520 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.870+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 >> 172.21.15.82:0/1735200648 conn(0x7feae006c000 msgr2=0x7feae00b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.871+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 shutdown_connections 2024-03-14T12:13:48.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.871+0000 7feae5b3b640 1 -- 172.21.15.82:0/1735200648 wait complete. 2024-03-14T12:13:48.881 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T12:13:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.981+0000 7f55ba432640 1 Processor -- start 2024-03-14T12:13:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.981+0000 7f55ba432640 1 -- start start 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55ba432640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55b40d4a20 0x7f55b40d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55ba432640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b415e4c0 con 0x7f55b40b3e10 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55ba432640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b40cf2f0 con 0x7f55b415e0e0 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55ba432640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b40d1640 con 0x7f55b40d4a20 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b89cf640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f55b40b3e10 0x7f55b40b4240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59120/0 (socket says 172.21.15.82:59120) 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b89cf640 1 -- 172.21.15.82:0/1663340739 learned_addr learned my addr 172.21.15.82:0/1663340739 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b37fe640 1 --2- 172.21.15.82:0/1663340739 >> v2:172.21.15.164:3300/0 conn(0x7f55b40d4a20 0x7f55b40d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958850340 0 0) 0x7f55b415e4c0 con 0x7f55b40b3e10 2024-03-14T12:13:48.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f559c003520 con 0x7f55b40b3e10 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017901317 0 0) 0x7f559c003520 con 0x7f55b40b3e10 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 >> v2:172.21.15.164:3300/0 conn(0x7f55b40d4a20 msgr2=0x7f55b40d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 --2- 172.21.15.82:0/1663340739 >> v2:172.21.15.164:3300/0 conn(0x7f55b40d4a20 0x7f55b40d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 >> v1:172.21.15.164:6789/0 conn(0x7f55b415e0e0 legacy=0x7f55b40d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b40cf5d0 con 0x7f55b40b3e10 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.982+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f55ac0019f0 con 0x7f55b40b3e10 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 >> v1:172.21.15.82:6789/0 conn(0x7f55b40b3e10 legacy=0x7f55b40b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b2ffd640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0039b0 0x7f559c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b2ffd640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c004800 0x7f559c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b2ffd640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f559c0051c0 0x7f559c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40d1640 con 0x7f559c0051c0 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c00c040 con 0x7f559c0039b0 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c00c240 con 0x7f559c004800 2024-03-14T12:13:48.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b89cf640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f559c0051c0 0x7f559c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b3fff640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0039b0 0x7f559c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b37fe640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c004800 0x7f559c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b89cf640 1 -- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c004800 msgr2=0x7f559c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b89cf640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c004800 0x7f559c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b89cf640 1 -- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0039b0 msgr2=0x7f559c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b89cf640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0039b0 0x7f559c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b89cf640 1 -- 172.21.15.82:0/1663340739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c003520 con 0x7f559c0051c0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.983+0000 7f55b37fe640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c004800 0x7f559c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55b89cf640 1 -- 172.21.15.82:0/1663340739 reap_dead start 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55b89cf640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f559c0051c0 0x7f559c00b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a790 tx=0x7f559c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b107096945dc77c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00a3b0 con 0x7f559c0051c0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac034650 con 0x7f559c0051c0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55b2ffd640 1 -- 172.21.15.82:0/1663340739 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac034a40 con 0x7f559c0051c0 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55ba432640 1 -- 172.21.15.82:0/1663340739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f559c0051c0 msgr2=0x7f559c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55ba432640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f559c0051c0 0x7f559c00b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a790 tx=0x7f559c005830 comp rx=0 tx=0).stop 2024-03-14T12:13:48.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55ba432640 1 -- 172.21.15.82:0/1663340739 shutdown_connections 2024-03-14T12:13:48.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55ba432640 1 --2- 172.21.15.82:0/1663340739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f559c0051c0 0x7f559c00b9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.984+0000 7f55ba432640 1 -- 172.21.15.82:0/1663340739 >> 172.21.15.82:0/1663340739 conn(0x7f55b40d3d40 msgr2=0x7f55b40d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:48.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.985+0000 7f55ba432640 1 -- 172.21.15.82:0/1663340739 shutdown_connections 2024-03-14T12:13:48.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.985+0000 7f55ba432640 1 -- 172.21.15.82:0/1663340739 wait complete. 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.986+0000 7f55ba432640 1 Processor -- start 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.986+0000 7f55ba432640 1 -- start start 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.986+0000 7f55ba432640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0051c0 0x7f55b4152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55ba432640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55b414f6c0 0x7f55b409ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55ba432640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55b414cbb0 0x7f55b415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55ba432640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40b3610 con 0x7f55b414cbb0 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55ba432640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40cfb90 con 0x7f559c0051c0 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55ba432640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40b38f0 con 0x7f55b414f6c0 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55b414cbb0 0x7f55b415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55b414cbb0 0x7f55b415f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34004/0 (socket says 172.21.15.82:34004) 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 -- 172.21.15.82:0/3603777860 learned_addr learned my addr 172.21.15.82:0/3603777860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 -- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55b414f6c0 msgr2=0x7f55b409ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55b414f6c0 0x7f55b409ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 -- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0051c0 msgr2=0x7f55b4152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:48.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b3fff640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0051c0 0x7f55b4152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0051c0 0x7f55b4152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 -- 172.21.15.82:0/3603777860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b415e710 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b3fff640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0051c0 0x7f55b4152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b89cf640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55b414cbb0 0x7f55b415f4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a8b0 tx=0x7f55b40cf4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b0ff9640 1 -- 172.21.15.82:0/3603777860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac039040 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b0ff9640 1 -- 172.21.15.82:0/3603777860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac043070 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b415e9f0 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55b0ff9640 1 -- 172.21.15.82:0/3603777860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac004d80 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.987+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b415efb0 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.988+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b404f050 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.989+0000 7f55b0ff9640 1 -- 172.21.15.82:0/3603777860 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f55ac0344f0 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.989+0000 7f55b0ff9640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f55a006bbd0 0x7f55a006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.989+0000 7f55b0ff9640 1 -- 172.21.15.82:0/3603777860 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55ac0ac470 con 0x7f55b414cbb0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.989+0000 7f55b3fff640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f55a006bbd0 0x7f55a006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:48.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.990+0000 7f55b3fff640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f55a006bbd0 0x7f55a006e090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005610 tx=0x7f55a4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:48.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:48.994+0000 7f55b0ff9640 1 -- 172.21.15.82:0/3603777860 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f55ac07b2b0 con 0x7f55b414cbb0 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.169+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f55b415f180 con 0x7f55b414cbb0 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.170+0000 7f55b0ff9640 1 -- 172.21.15.82:0/3603777860 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f55ac07abb0 con 0x7f55b414cbb0 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:13:49.170 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f55a006bbd0 msgr2=0x7f55a006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f55a006bbd0 0x7f55a006e090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005610 tx=0x7f55a4009040 comp rx=0 tx=0).stop 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55b414cbb0 msgr2=0x7f55b415f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55b414cbb0 0x7f55b415f4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00a8b0 tx=0x7f55b40cf4c0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 shutdown_connections 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55b414cbb0 0x7f55b415f4d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f55a006bbd0 0x7f55a006e090 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55b414f6c0 0x7f55b409ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 --2- 172.21.15.82:0/3603777860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559c0051c0 0x7f55b4152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 >> 172.21.15.82:0/3603777860 conn(0x7f55b40d3d40 msgr2=0x7f55b406cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:49.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 shutdown_connections 2024-03-14T12:13:49.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.172+0000 7f55ba432640 1 -- 172.21.15.82:0/3603777860 wait complete. 2024-03-14T12:13:49.183 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T12:13:49.183 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T12:13:49.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.286+0000 7f3687969640 1 Processor -- start 2024-03-14T12:13:49.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.287+0000 7f3687969640 1 -- start start 2024-03-14T12:13:49.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.287+0000 7f3687969640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f368015e100 0x7f36800ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.287+0000 7f3687969640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800b4430 con 0x7f36800ce860 2024-03-14T12:13:49.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.287+0000 7f3687969640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368006f100 con 0x7f36800d0c40 2024-03-14T12:13:49.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.287+0000 7f3687969640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368015e4e0 con 0x7f368015e100 2024-03-14T12:13:49.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f36800ce860 0x7f36800cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59134/0 (socket says 172.21.15.82:59134) 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 -- 172.21.15.82:0/4138469803 learned_addr learned my addr 172.21.15.82:0/4138469803 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3685705640 1 --2- 172.21.15.82:0/4138469803 >> v2:172.21.15.164:3300/0 conn(0x7f368015e100 0x7f36800ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269370827 0 0) 0x7f36800b4430 con 0x7f36800ce860 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366c003520 con 0x7f36800ce860 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3512260172 0 0) 0x7f366c003520 con 0x7f36800ce860 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 >> v2:172.21.15.164:3300/0 conn(0x7f368015e100 msgr2=0x7f36800ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 --2- 172.21.15.82:0/4138469803 >> v2:172.21.15.164:3300/0 conn(0x7f368015e100 0x7f36800ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 >> v1:172.21.15.164:6789/0 conn(0x7f36800d0c40 legacy=0x7f36800cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800d10b0 con 0x7f36800ce860 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f3670001690 con 0x7f36800ce860 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 >> v1:172.21.15.82:6789/0 conn(0x7f36800ce860 legacy=0x7f36800cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c0039b0 0x7f366c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368015e4e0 con 0x7f366c004800 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00c110 con 0x7f366c0051c0 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00c310 con 0x7f366c0039b0 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3685705640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c0039b0 0x7f366c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3685f06640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 -- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c0039b0 msgr2=0x7f366c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c0039b0 0x7f366c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 -- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c0051c0 msgr2=0x7f366c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.288+0000 7f3684f04640 1 -- 172.21.15.82:0/4138469803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c003520 con 0x7f366c004800 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.289+0000 7f3685f06640 1 -- 172.21.15.82:0/4138469803 reap_dead start 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.289+0000 7f3684f04640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c004800 0x7f366c004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f367000c210 tx=0x7f366c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cd261ea7d1fbb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.289+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670007d60 con 0x7f366c004800 2024-03-14T12:13:49.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.289+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670030070 con 0x7f366c004800 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.289+0000 7f3677fff640 1 -- 172.21.15.82:0/4138469803 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670005d10 con 0x7f366c004800 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.290+0000 7f3687969640 1 -- 172.21.15.82:0/4138469803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c004800 msgr2=0x7f366c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.290+0000 7f3687969640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c004800 0x7f366c004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f367000c210 tx=0x7f366c005830 comp rx=0 tx=0).stop 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.290+0000 7f3687969640 1 -- 172.21.15.82:0/4138469803 shutdown_connections 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.290+0000 7f3687969640 1 --2- 172.21.15.82:0/4138469803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.290+0000 7f3687969640 1 -- 172.21.15.82:0/4138469803 >> 172.21.15.82:0/4138469803 conn(0x7f36800b3610 msgr2=0x7f36800b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.290+0000 7f3687969640 1 -- 172.21.15.82:0/4138469803 shutdown_connections 2024-03-14T12:13:49.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.290+0000 7f3687969640 1 -- 172.21.15.82:0/4138469803 wait complete. 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.291+0000 7f3687969640 1 Processor -- start 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.291+0000 7f3687969640 1 -- start start 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3687969640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c004800 0x7f36800c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3687969640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f368015b3c0 0x7f3680151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3687969640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36800c5f30 0x7f36800c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3687969640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800cc190 con 0x7f36800c5f30 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3687969640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368006ebd0 con 0x7f368015b3c0 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3687969640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800cc470 con 0x7f366c004800 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36800c5f30 0x7f36800c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36800c5f30 0x7f36800c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34032/0 (socket says 172.21.15.82:34032) 2024-03-14T12:13:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 -- 172.21.15.82:0/1399743491 learned_addr learned my addr 172.21.15.82:0/1399743491 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685705640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c004800 0x7f36800c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3684f04640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f368015b3c0 0x7f3680151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 -- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c004800 msgr2=0x7f36800c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c004800 0x7f36800c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 -- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f368015b3c0 msgr2=0x7f3680151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f368015b3c0 0x7f3680151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 -- 172.21.15.82:0/1399743491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36801520c0 con 0x7f36800c5f30 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685705640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c004800 0x7f36800c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3684f04640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f368015b3c0 0x7f3680151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3685f06640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36800c5f30 0x7f36800c63b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3680071a50 tx=0x7f367800f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3675ffb640 1 -- 172.21.15.82:0/1399743491 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678019070 con 0x7f36800c5f30 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.292+0000 7f3675ffb640 1 -- 172.21.15.82:0/1399743491 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3678007300 con 0x7f36800c5f30 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.293+0000 7f3675ffb640 1 -- 172.21.15.82:0/1399743491 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678005030 con 0x7f36800c5f30 2024-03-14T12:13:49.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.293+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36800c74b0 con 0x7f36800c5f30 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.293+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36800c5730 con 0x7f36800c5f30 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.294+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368004f050 con 0x7f36800c5f30 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.294+0000 7f3675ffb640 1 -- 172.21.15.82:0/1399743491 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f367801c070 con 0x7f36800c5f30 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.294+0000 7f3675ffb640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f366406bbd0 0x7f366406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.294+0000 7f3675ffb640 1 -- 172.21.15.82:0/1399743491 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3678089500 con 0x7f36800c5f30 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.294+0000 7f3685705640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f366406bbd0 0x7f366406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.295+0000 7f3685705640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f366406bbd0 0x7f366406e090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f367c007440 tx=0x7f367c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:49.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.298+0000 7f3675ffb640 1 -- 172.21.15.82:0/1399743491 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f36780583d0 con 0x7f36800c5f30 2024-03-14T12:13:49.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.471+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f36800cdef0 con 0x7f36800c5f30 2024-03-14T12:13:49.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.544+0000 7f3675ffb640 1 -- 172.21.15.82:0/1399743491 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3678057cd0 con 0x7f36800c5f30 2024-03-14T12:13:49.544 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f366406bbd0 msgr2=0x7f366406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f366406bbd0 0x7f366406e090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f367c007440 tx=0x7f367c00b040 comp rx=0 tx=0).stop 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36800c5f30 msgr2=0x7f36800c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36800c5f30 0x7f36800c63b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3680071a50 tx=0x7f367800f9b0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 shutdown_connections 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36800c5f30 0x7f36800c63b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f368015b3c0 0x7f3680151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f366406bbd0 0x7f366406e090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 --2- 172.21.15.82:0/1399743491 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f366c004800 0x7f36800c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.546+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 >> 172.21.15.82:0/1399743491 conn(0x7f36800b3610 msgr2=0x7f36800b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.547+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 shutdown_connections 2024-03-14T12:13:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.547+0000 7f3687969640 1 -- 172.21.15.82:0/1399743491 wait complete. 2024-03-14T12:13:49.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T12:13:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.668+0000 7fb5e7885640 1 Processor -- start 2024-03-14T12:13:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.668+0000 7fb5e7885640 1 -- start start 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.668+0000 7fb5e7885640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d05c0 0x7fb5e00d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.668+0000 7fb5e7885640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5e00cf4a0 0x7fb5e00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e7885640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e00d0ee0 con 0x7fb5e00d4860 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e7885640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e00cfd80 con 0x7fb5e00cf4a0 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e7885640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e00d19d0 con 0x7fb5e00d05c0 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e5e22640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5e00cf4a0 0x7fb5e00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e4e20640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb5e00d4860 0x7fb5e00d2790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59138/0 (socket says 172.21.15.82:59138) 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e4e20640 1 -- 172.21.15.82:0/3976868098 learned_addr learned my addr 172.21.15.82:0/3976868098 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e5621640 1 --2- 172.21.15.82:0/3976868098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d05c0 0x7fb5e00d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e5e22640 1 -- 172.21.15.82:0/3976868098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d05c0 msgr2=0x7fb5e00d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e5e22640 1 --2- 172.21.15.82:0/3976868098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d05c0 0x7fb5e00d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e5e22640 1 -- 172.21.15.82:0/3976868098 >> v1:172.21.15.82:6789/0 conn(0x7fb5e00d4860 legacy=0x7fb5e00d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.669+0000 7fb5e5e22640 1 -- 172.21.15.82:0/3976868098 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e00d1cb0 con 0x7fb5e00cf4a0 2024-03-14T12:13:49.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5e5e22640 1 --2- 172.21.15.82:0/3976868098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00b3630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009dc0 tx=0x7fb5d802bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0842bd928a9459 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:49.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5d7fff640 1 -- 172.21.15.82:0/3976868098 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8039070 con 0x7fb5e00cf4a0 2024-03-14T12:13:49.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5d7fff640 1 -- 172.21.15.82:0/3976868098 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d802cb20 con 0x7fb5e00cf4a0 2024-03-14T12:13:49.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5d7fff640 1 -- 172.21.15.82:0/3976868098 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8034440 con 0x7fb5e00cf4a0 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5e7885640 1 -- 172.21.15.82:0/3976868098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00cf4a0 msgr2=0x7fb5e00b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5e7885640 1 --2- 172.21.15.82:0/3976868098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00b3630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009dc0 tx=0x7fb5d802bd70 comp rx=0 tx=0).stop 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5e7885640 1 -- 172.21.15.82:0/3976868098 shutdown_connections 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5e7885640 1 --2- 172.21.15.82:0/3976868098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00b3630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5e7885640 1 --2- 172.21.15.82:0/3976868098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d05c0 0x7fb5e00d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.670+0000 7fb5e7885640 1 -- 172.21.15.82:0/3976868098 >> 172.21.15.82:0/3976868098 conn(0x7fb5e00d3cd0 msgr2=0x7fb5e00d40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.671+0000 7fb5e7885640 1 -- 172.21.15.82:0/3976868098 shutdown_connections 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.671+0000 7fb5e7885640 1 -- 172.21.15.82:0/3976868098 wait complete. 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 Processor -- start 2024-03-14T12:13:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 -- start start 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00d05c0 0x7fb5e01f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d4860 0x7fb5e00c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e015e280 con 0x7fb5e00d05c0 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e00d2270 con 0x7fb5e00cf4a0 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e7885640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e015e400 con 0x7fb5e00d4860 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e4e20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00d05c0 0x7fb5e01f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e4e20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00d05c0 0x7fb5e01f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34036/0 (socket says 172.21.15.82:34036) 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.672+0000 7fb5e4e20640 1 -- 172.21.15.82:0/1060894904 learned_addr learned my addr 172.21.15.82:0/1060894904 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e5e22640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d4860 0x7fb5e00c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e5621640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e4e20640 1 -- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d4860 msgr2=0x7fb5e00c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e4e20640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d4860 0x7fb5e00c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e4e20640 1 -- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5e00cf4a0 msgr2=0x7fb5e00cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e4e20640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e4e20640 1 -- 172.21.15.82:0/1060894904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e00c99d0 con 0x7fb5e00d05c0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e5e22640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d4860 0x7fb5e00c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e5621640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e4e20640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00d05c0 0x7fb5e01f49b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c440 tx=0x7fb5d000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5d5ffb640 1 -- 172.21.15.82:0/1060894904 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0017070 con 0x7fb5e00d05c0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5d5ffb640 1 -- 172.21.15.82:0/1060894904 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d0005ce0 con 0x7fb5e00d05c0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e00c9cb0 con 0x7fb5e00d05c0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5d5ffb640 1 -- 172.21.15.82:0/1060894904 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00072f0 con 0x7fb5e00d05c0 2024-03-14T12:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.673+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e00ca270 con 0x7fb5e00d05c0 2024-03-14T12:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.674+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e004f050 con 0x7fb5e00d05c0 2024-03-14T12:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.674+0000 7fb5d5ffb640 1 -- 172.21.15.82:0/1060894904 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb5d001a070 con 0x7fb5e00d05c0 2024-03-14T12:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.675+0000 7fb5d5ffb640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb5c406bbd0 0x7fb5c406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.675+0000 7fb5d5ffb640 1 -- 172.21.15.82:0/1060894904 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb5d0089bf0 con 0x7fb5e00d05c0 2024-03-14T12:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.675+0000 7fb5e5621640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb5c406bbd0 0x7fb5c406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.675+0000 7fb5e5621640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb5c406bbd0 0x7fb5c406e090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006fd0 tx=0x7fb5dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:49.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.679+0000 7fb5d5ffb640 1 -- 172.21.15.82:0/1060894904 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb5d00588c0 con 0x7fb5e00d05c0 2024-03-14T12:13:49.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:49.855+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb5e00b52f0 con 0x7fb5e00d05c0 2024-03-14T12:13:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.546+0000 7fb5d5ffb640 1 -- 172.21.15.82:0/1060894904 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb5d00581c0 con 0x7fb5e00d05c0 2024-03-14T12:13:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-03-14T12:13:50.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb5c406bbd0 msgr2=0x7fb5c406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:50.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb5c406bbd0 0x7fb5c406e090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006fd0 tx=0x7fb5dc008040 comp rx=0 tx=0).stop 2024-03-14T12:13:50.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00d05c0 msgr2=0x7fb5e01f49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00d05c0 0x7fb5e01f49b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c440 tx=0x7fb5d000c900 comp rx=0 tx=0).stop 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 shutdown_connections 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb5c406bbd0 0x7fb5c406e090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5e00d4860 0x7fb5e00c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e00d05c0 0x7fb5e01f49b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 --2- 172.21.15.82:0/1060894904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5e00cf4a0 0x7fb5e00cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.549+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 >> 172.21.15.82:0/1060894904 conn(0x7fb5e00d3cd0 msgr2=0x7fb5e00b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.550+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 shutdown_connections 2024-03-14T12:13:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.550+0000 7fb5e7885640 1 -- 172.21.15.82:0/1060894904 wait complete. 2024-03-14T12:13:50.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T12:13:50.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.681+0000 7fd306a4b640 1 Processor -- start 2024-03-14T12:13:50.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.681+0000 7fd306a4b640 1 -- start start 2024-03-14T12:13:50.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd306a4b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd300161580 0x7fd3000b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:50.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd306a4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd300161b10 0x7fd300161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:50.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd306a4b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30006ee00 con 0x7fd3000cfcc0 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd306a4b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd300162490 con 0x7fd300161580 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd306a4b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30006f0e0 con 0x7fd300161b10 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2ff7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd300161b10 0x7fd300161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2fffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd300161580 0x7fd3000b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2ff7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd300161b10 0x7fd300161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:34364/0 (socket says 172.21.15.82:34364) 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd304fe8640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd3000cfcc0 0x7fd3000d0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:46716/0 (socket says 172.21.15.82:46716) 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2ff7fe640 1 -- 172.21.15.82:0/1065580788 learned_addr learned my addr 172.21.15.82:0/1065580788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2fffff640 1 -- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd300161b10 msgr2=0x7fd300161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2fffff640 1 --2- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd300161b10 0x7fd300161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2fffff640 1 -- 172.21.15.82:0/1065580788 >> v1:172.21.15.164:6789/0 conn(0x7fd3000cfcc0 legacy=0x7fd3000d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2feffd640 1 -- 172.21.15.82:0/1065580788 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807917765 0 0) 0x7fd30006ee00 con 0x7fd3000cfcc0 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2fffff640 1 -- 172.21.15.82:0/1065580788 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30006be20 con 0x7fd300161580 2024-03-14T12:13:50.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.682+0000 7fd2ff7fe640 1 --2- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd300161b10 0x7fd300161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.683+0000 7fd2fffff640 1 --2- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 0x7fd3000b41a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00097a0 tx=0x7fd2f002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd661f9f81b798aa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.683+0000 7fd2feffd640 1 -- 172.21.15.82:0/1065580788 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0034070 con 0x7fd300161580 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.683+0000 7fd2feffd640 1 -- 172.21.15.82:0/1065580788 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0030610 con 0x7fd300161580 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.683+0000 7fd2feffd640 1 -- 172.21.15.82:0/1065580788 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00309e0 con 0x7fd300161580 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 -- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 msgr2=0x7fd3000b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 --2- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 0x7fd3000b41a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00097a0 tx=0x7fd2f002b960 comp rx=0 tx=0).stop 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 -- 172.21.15.82:0/1065580788 shutdown_connections 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 --2- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd300161b10 0x7fd300161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 --2- 172.21.15.82:0/1065580788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 0x7fd3000b41a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 -- 172.21.15.82:0/1065580788 >> 172.21.15.82:0/1065580788 conn(0x7fd3000b3680 msgr2=0x7fd3000b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 -- 172.21.15.82:0/1065580788 shutdown_connections 2024-03-14T12:13:50.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.684+0000 7fd306a4b640 1 -- 172.21.15.82:0/1065580788 wait complete. 2024-03-14T12:13:50.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.685+0000 7fd306a4b640 1 Processor -- start 2024-03-14T12:13:50.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.685+0000 7fd306a4b640 1 -- start start 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd306a4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3000cfcc0 0x7fd3000c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd306a4b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 0x7fd3000c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd306a4b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd300161b10 0x7fd3000c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd306a4b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30006e9e0 con 0x7fd300161b10 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd306a4b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30006c3e0 con 0x7fd300161580 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd306a4b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30006eb60 con 0x7fd3000cfcc0 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd300161b10 0x7fd3000c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd300161b10 0x7fd3000c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34046/0 (socket says 172.21.15.82:34046) 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 -- 172.21.15.82:0/2331145426 learned_addr learned my addr 172.21.15.82:0/2331145426 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd2fffff640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3000cfcc0 0x7fd3000c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd2ff7fe640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 0x7fd3000c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 -- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3000cfcc0 msgr2=0x7fd3000c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:50.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3000cfcc0 0x7fd3000c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 -- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 msgr2=0x7fd3000c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 0x7fd3000c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.686+0000 7fd304fe8640 1 -- 172.21.15.82:0/2331145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000c7be0 con 0x7fd300161b10 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.687+0000 7fd304fe8640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd300161b10 0x7fd3000c6b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c7a0 tx=0x7fd2f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.687+0000 7fd2fcff9640 1 -- 172.21.15.82:0/2331145426 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8018070 con 0x7fd300161b10 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.687+0000 7fd2fcff9640 1 -- 172.21.15.82:0/2331145426 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f800a640 con 0x7fd300161b10 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.687+0000 7fd2fcff9640 1 -- 172.21.15.82:0/2331145426 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f800aab0 con 0x7fd300161b10 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.687+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3000b6240 con 0x7fd300161b10 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.687+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3001e7bc0 con 0x7fd300161b10 2024-03-14T12:13:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.688+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30004f050 con 0x7fd300161b10 2024-03-14T12:13:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.688+0000 7fd2fcff9640 1 -- 172.21.15.82:0/2331145426 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd2f801b070 con 0x7fd300161b10 2024-03-14T12:13:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.689+0000 7fd2fcff9640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd2ec06bc00 0x7fd2ec06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.689+0000 7fd2fcff9640 1 -- 172.21.15.82:0/2331145426 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd2f808a150 con 0x7fd300161b10 2024-03-14T12:13:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.689+0000 7fd2fffff640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd2ec06bc00 0x7fd2ec06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.689+0000 7fd2fffff640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd2ec06bc00 0x7fd2ec06e0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0036000 tx=0x7fd2f0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:50.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.693+0000 7fd2fcff9640 1 -- 172.21.15.82:0/2331145426 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd2f80587c0 con 0x7fd300161b10 2024-03-14T12:13:50.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:50.900+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd3000d15c0 con 0x7fd300161b10 2024-03-14T12:13:51.360 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:13:51.363 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:13:51.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.569+0000 7fd2fcff9640 1 -- 172.21.15.82:0/2331145426 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd2f805c860 con 0x7fd300161b10 2024-03-14T12:13:51.570 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2024-03-14T12:13:51.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.573+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd2ec06bc00 msgr2=0x7fd2ec06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:51.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.573+0000 7fd306a4b640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd2ec06bc00 0x7fd2ec06e0c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0036000 tx=0x7fd2f0008030 comp rx=0 tx=0).stop 2024-03-14T12:13:51.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.573+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd300161b10 msgr2=0x7fd3000c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:51.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.573+0000 7fd306a4b640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd300161b10 0x7fd3000c6b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c7a0 tx=0x7fd2f800cc60 comp rx=0 tx=0).stop 2024-03-14T12:13:51.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.574+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 shutdown_connections 2024-03-14T12:13:51.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.574+0000 7fd306a4b640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd300161b10 0x7fd3000c6b10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.574+0000 7fd306a4b640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd300161580 0x7fd3000c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.574+0000 7fd306a4b640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd2ec06bc00 0x7fd2ec06e0c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.574+0000 7fd306a4b640 1 --2- 172.21.15.82:0/2331145426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3000cfcc0 0x7fd3000c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.574+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 >> 172.21.15.82:0/2331145426 conn(0x7fd3000b3680 msgr2=0x7fd30006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:51.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.575+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 shutdown_connections 2024-03-14T12:13:51.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.575+0000 7fd306a4b640 1 -- 172.21.15.82:0/2331145426 wait complete. 2024-03-14T12:13:51.595 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:51.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.696+0000 7fa03f020640 1 Processor -- start 2024-03-14T12:13:51.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.696+0000 7fa03f020640 1 -- start start 2024-03-14T12:13:51.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03f020640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0380b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03f020640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa0380b5bb0 0x7fa038161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03f020640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa038161fc0 0x7fa03815e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03f020640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0380b6d00 con 0x7fa038161fc0 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03f020640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0380b60f0 con 0x7fa0380b5bb0 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03f020640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0380b63d0 con 0x7fa0381551d0 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa038161fc0 0x7fa03815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa038161fc0 0x7fa03815e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48424/0 (socket says 172.21.15.82:48424) 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 -- 172.21.15.82:0/3911471578 learned_addr learned my addr 172.21.15.82:0/3911471578 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:51.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa037fff640 1 --2- 172.21.15.82:0/3911471578 >> v2:172.21.15.164:3300/0 conn(0x7fa0380b5bb0 0x7fa038161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03cdbc640 1 --2- 172.21.15.82:0/3911471578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0380b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 -- 172.21.15.82:0/3911471578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 msgr2=0x7fa0380b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 --2- 172.21.15.82:0/3911471578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0380b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 -- 172.21.15.82:0/3911471578 >> v2:172.21.15.164:3300/0 conn(0x7fa0380b5bb0 msgr2=0x7fa038161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 --2- 172.21.15.82:0/3911471578 >> v2:172.21.15.164:3300/0 conn(0x7fa0380b5bb0 0x7fa038161a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03d5bd640 1 -- 172.21.15.82:0/3911471578 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0380b66b0 con 0x7fa038161fc0 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa037fff640 1 --2- 172.21.15.82:0/3911471578 >> v2:172.21.15.164:3300/0 conn(0x7fa0380b5bb0 0x7fa038161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.697+0000 7fa03cdbc640 1 --2- 172.21.15.82:0/3911471578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0380b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:51.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.698+0000 7fa03d5bd640 1 --2- 172.21.15.82:0/3911471578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa038161fc0 0x7fa03815e490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0380721a0 tx=0x7fa03002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd763710e26328c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.698+0000 7fa0377fe640 1 -- 172.21.15.82:0/3911471578 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03002ca70 con 0x7fa038161fc0 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.698+0000 7fa0377fe640 1 -- 172.21.15.82:0/3911471578 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03002cc10 con 0x7fa038161fc0 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.698+0000 7fa0377fe640 1 -- 172.21.15.82:0/3911471578 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0300343f0 con 0x7fa038161fc0 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 -- 172.21.15.82:0/3911471578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa038161fc0 msgr2=0x7fa03815e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 --2- 172.21.15.82:0/3911471578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa038161fc0 0x7fa03815e490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0380721a0 tx=0x7fa03002bed0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 -- 172.21.15.82:0/3911471578 shutdown_connections 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 --2- 172.21.15.82:0/3911471578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa038161fc0 0x7fa03815e490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 --2- 172.21.15.82:0/3911471578 >> v2:172.21.15.164:3300/0 conn(0x7fa0380b5bb0 0x7fa038161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 --2- 172.21.15.82:0/3911471578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0380b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 -- 172.21.15.82:0/3911471578 >> 172.21.15.82:0/3911471578 conn(0x7fa038072ee0 msgr2=0x7fa03809a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 -- 172.21.15.82:0/3911471578 shutdown_connections 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.699+0000 7fa03f020640 1 -- 172.21.15.82:0/3911471578 wait complete. 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.700+0000 7fa03f020640 1 Processor -- start 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.700+0000 7fa03f020640 1 -- start start 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0380b5bb0 0x7fa0380b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0381e8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa038161fc0 0x7fa0380b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03815eb50 con 0x7fa0380b5bb0 2024-03-14T12:13:51.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03815e9d0 con 0x7fa038161fc0 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03815ecd0 con 0x7fa0381551d0 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0380b5bb0 0x7fa0380b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0380b5bb0 0x7fa0380b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48430/0 (socket says 172.21.15.82:48430) 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 -- 172.21.15.82:0/849011571 learned_addr learned my addr 172.21.15.82:0/849011571 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa037fff640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0381e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03d5bd640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa038161fc0 0x7fa0380b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 -- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 msgr2=0x7fa0381e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0381e8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 -- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa038161fc0 msgr2=0x7fa0380b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:51.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa038161fc0 0x7fa0380b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 -- 172.21.15.82:0/849011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0381e7880 con 0x7fa0380b5bb0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03cdbc640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0380b5bb0 0x7fa0380b2bc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa02800c820 tx=0x7fa02800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa0357fa640 1 -- 172.21.15.82:0/849011571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028015070 con 0x7fa0380b5bb0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa0357fa640 1 -- 172.21.15.82:0/849011571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028007a30 con 0x7fa0380b5bb0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0381e7b40 con 0x7fa0380b5bb0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa0357fa640 1 -- 172.21.15.82:0/849011571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028007e00 con 0x7fa0380b5bb0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.701+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0381e8100 con 0x7fa0380b5bb0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.702+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0380b3100 con 0x7fa0380b5bb0 2024-03-14T12:13:51.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.703+0000 7fa0357fa640 1 -- 172.21.15.82:0/849011571 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa028018070 con 0x7fa0380b5bb0 2024-03-14T12:13:51.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.703+0000 7fa0357fa640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa02406bbd0 0x7fa02406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.703+0000 7fa037fff640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa02406bbd0 0x7fa02406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.704+0000 7fa0357fa640 1 -- 172.21.15.82:0/849011571 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa02808a8f0 con 0x7fa0380b5bb0 2024-03-14T12:13:51.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.704+0000 7fa037fff640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa02406bbd0 0x7fa02406e090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0380cd410 tx=0x7fa02c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:51.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.708+0000 7fa0357fa640 1 -- 172.21.15.82:0/849011571 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa0380b3100 con 0x7fa0380b5bb0 2024-03-14T12:13:51.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.882+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa03804f050 con 0x7fa0380b5bb0 2024-03-14T12:13:51.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.883+0000 7fa0357fa640 1 -- 172.21.15.82:0/849011571 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16606 (secure 0 0 0) 0x7fa028058d90 con 0x7fa0380b5bb0 2024-03-14T12:13:51.883 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:51.883 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":20,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:50.902399+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa02406bbd0 msgr2=0x7fa02406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa02406bbd0 0x7fa02406e090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0380cd410 tx=0x7fa02c008040 comp rx=0 tx=0).stop 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0380b5bb0 msgr2=0x7fa0380b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0380b5bb0 0x7fa0380b2bc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa02800c820 tx=0x7fa02800cc20 comp rx=0 tx=0).stop 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 shutdown_connections 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa038161fc0 0x7fa0380b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa02406bbd0 0x7fa02406e090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0381551d0 0x7fa0381e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 --2- 172.21.15.82:0/849011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0380b5bb0 0x7fa0380b2bc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 >> 172.21.15.82:0/849011571 conn(0x7fa038072ee0 msgr2=0x7fa0380b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:51.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 shutdown_connections 2024-03-14T12:13:51.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.885+0000 7fa03f020640 1 -- 172.21.15.82:0/849011571 wait complete. 2024-03-14T12:13:51.896 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.995+0000 7f62b78f7640 1 Processor -- start 2024-03-14T12:13:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.996+0000 7f62b78f7640 1 -- start start 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.996+0000 7f62b78f7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b00b4220 0x7f62b0161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.996+0000 7f62b78f7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f62b0162010 0x7f62b006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.996+0000 7f62b78f7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b006fa90 con 0x7f62b015e150 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.996+0000 7f62b78f7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b006c750 con 0x7f62b0162010 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.996+0000 7f62b78f7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b006be20 con 0x7f62b00b4220 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.996+0000 7f62b5e94640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f62b0162010 0x7f62b006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b5e94640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f62b0162010 0x7f62b006cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48440/0 (socket says 172.21.15.82:48440) 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b5e94640 1 -- 172.21.15.82:0/262981294 learned_addr learned my addr 172.21.15.82:0/262981294 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b4e92640 1 --2- 172.21.15.82:0/262981294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b00b4220 0x7f62b0161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b5e94640 1 -- 172.21.15.82:0/262981294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b00b4220 msgr2=0x7f62b0161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b5e94640 1 --2- 172.21.15.82:0/262981294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b00b4220 0x7f62b0161ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b5e94640 1 -- 172.21.15.82:0/262981294 >> v1:172.21.15.164:6789/0 conn(0x7f62b015e150 legacy=0x7f62b006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b5e94640 1 -- 172.21.15.82:0/262981294 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b006c100 con 0x7f62b0162010 2024-03-14T12:13:51.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b4e92640 1 --2- 172.21.15.82:0/262981294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b00b4220 0x7f62b0161ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:51.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.997+0000 7f62b5e94640 1 --2- 172.21.15.82:0/262981294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b0162010 0x7f62b006cbe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009e50 tx=0x7f62a802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8cbfbf9acf70f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:51.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62a7fff640 1 -- 172.21.15.82:0/262981294 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a802ca70 con 0x7f62b0162010 2024-03-14T12:13:51.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62a7fff640 1 -- 172.21.15.82:0/262981294 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a802cc10 con 0x7f62b0162010 2024-03-14T12:13:51.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62a7fff640 1 -- 172.21.15.82:0/262981294 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80343f0 con 0x7f62b0162010 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 -- 172.21.15.82:0/262981294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b0162010 msgr2=0x7f62b006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 --2- 172.21.15.82:0/262981294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b0162010 0x7f62b006cbe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009e50 tx=0x7f62a802bed0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 -- 172.21.15.82:0/262981294 shutdown_connections 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 --2- 172.21.15.82:0/262981294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b0162010 0x7f62b006cbe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 --2- 172.21.15.82:0/262981294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b00b4220 0x7f62b0161ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 -- 172.21.15.82:0/262981294 >> 172.21.15.82:0/262981294 conn(0x7f62b00b3670 msgr2=0x7f62b00b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 -- 172.21.15.82:0/262981294 shutdown_connections 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.998+0000 7f62b78f7640 1 -- 172.21.15.82:0/262981294 wait complete. 2024-03-14T12:13:52.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.999+0000 7f62b78f7640 1 Processor -- start 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:51.999+0000 7f62b78f7640 1 -- start start 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b78f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b00b4220 0x7f62b01531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b78f7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b015e150 0x7f62b01e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b78f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b0162010 0x7f62b01f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b78f7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b006d2a0 con 0x7f62b00b4220 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b78f7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b006d120 con 0x7f62b0162010 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b78f7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b006d420 con 0x7f62b015e150 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b00b4220 0x7f62b01531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b00b4220 0x7f62b01531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48454/0 (socket says 172.21.15.82:48454) 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 -- 172.21.15.82:0/1730146504 learned_addr learned my addr 172.21.15.82:0/1730146504 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b4e92640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b015e150 0x7f62b01e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5e94640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b0162010 0x7f62b01f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 -- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b015e150 msgr2=0x7f62b01e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b015e150 0x7f62b01e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 -- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b0162010 msgr2=0x7f62b01f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b0162010 0x7f62b01f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b5693640 1 -- 172.21.15.82:0/1730146504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b01f5750 con 0x7f62b00b4220 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.000+0000 7f62b4e92640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b015e150 0x7f62b01e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.001+0000 7f62b5e94640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b0162010 0x7f62b01f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.001+0000 7f62b5693640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b00b4220 0x7f62b01531a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c460 tx=0x7f62ac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.001+0000 7f62a5ffb640 1 -- 172.21.15.82:0/1730146504 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac016020 con 0x7f62b00b4220 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.001+0000 7f62a5ffb640 1 -- 172.21.15.82:0/1730146504 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac012870 con 0x7f62b00b4220 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.001+0000 7f62a5ffb640 1 -- 172.21.15.82:0/1730146504 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac012c60 con 0x7f62b00b4220 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.001+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b01f5a30 con 0x7f62b00b4220 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.001+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b01f5ff0 con 0x7f62b00b4220 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.002+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b006b0e0 con 0x7f62b00b4220 2024-03-14T12:13:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.002+0000 7f62a5ffb640 1 -- 172.21.15.82:0/1730146504 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f62ac019020 con 0x7f62b00b4220 2024-03-14T12:13:52.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.003+0000 7f62a5ffb640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629406bbd0 0x7f629406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.003+0000 7f62a5ffb640 1 -- 172.21.15.82:0/1730146504 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62ac08aa30 con 0x7f62b00b4220 2024-03-14T12:13:52.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.003+0000 7f62b4e92640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629406bbd0 0x7f629406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.003+0000 7f62b4e92640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629406bbd0 0x7f629406e090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62b00c89c0 tx=0x7f62a0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.007+0000 7f62a5ffb640 1 -- 172.21.15.82:0/1730146504 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f62b006b0e0 con 0x7f62b00b4220 2024-03-14T12:13:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.178+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62b00b2460 con 0x7f62b00b4220 2024-03-14T12:13:52.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.179+0000 7f62a5ffb640 1 -- 172.21.15.82:0/1730146504 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16606 (secure 0 0 0) 0x7f62b00b2460 con 0x7f62b00b4220 2024-03-14T12:13:52.180 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:52.180 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":20,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:50.902399+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629406bbd0 msgr2=0x7f629406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629406bbd0 0x7f629406e090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62b00c89c0 tx=0x7f62a0008040 comp rx=0 tx=0).stop 2024-03-14T12:13:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b00b4220 msgr2=0x7f62b01531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b00b4220 0x7f62b01531a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c460 tx=0x7f62ac00c920 comp rx=0 tx=0).stop 2024-03-14T12:13:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 shutdown_connections 2024-03-14T12:13:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b0162010 0x7f62b01f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629406bbd0 0x7f629406e090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62b015e150 0x7f62b01e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 --2- 172.21.15.82:0/1730146504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b00b4220 0x7f62b01531a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 >> 172.21.15.82:0/1730146504 conn(0x7f62b00b3670 msgr2=0x7f62b006ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 shutdown_connections 2024-03-14T12:13:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.182+0000 7f62b78f7640 1 -- 172.21.15.82:0/1730146504 wait complete. 2024-03-14T12:13:52.193 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T12:13:52.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.295+0000 7f7159ba9640 1 Processor -- start 2024-03-14T12:13:52.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.295+0000 7f7159ba9640 1 -- start start 2024-03-14T12:13:52.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7159ba9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f715415e150 0x7f71540b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7159ba9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f71540b6bd0 0x7f71540d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7159ba9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715406ec70 con 0x7f715415dd70 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7159ba9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715406ea20 con 0x7f715415e150 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7159ba9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715406f880 con 0x7f71540b6bd0 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7152ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f715415e150 0x7f71540b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7152ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f715415e150 0x7f71540b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48458/0 (socket says 172.21.15.82:48458) 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7152ffd640 1 -- 172.21.15.82:0/4032111284 learned_addr learned my addr 172.21.15.82:0/4032111284 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7153fff640 1 --2- 172.21.15.82:0/4032111284 >> v2:172.21.15.164:3300/0 conn(0x7f71540b6bd0 0x7f71540d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7152ffd640 1 -- 172.21.15.82:0/4032111284 >> v2:172.21.15.164:3300/0 conn(0x7f71540b6bd0 msgr2=0x7f71540d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7152ffd640 1 --2- 172.21.15.82:0/4032111284 >> v2:172.21.15.164:3300/0 conn(0x7f71540b6bd0 0x7f71540d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7152ffd640 1 -- 172.21.15.82:0/4032111284 >> v1:172.21.15.164:6789/0 conn(0x7f715415dd70 legacy=0x7f715406fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.296+0000 7f7152ffd640 1 -- 172.21.15.82:0/4032111284 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715406ef50 con 0x7f715415e150 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.297+0000 7f7152ffd640 1 --2- 172.21.15.82:0/4032111284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415e150 0x7f71540b6690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7154072910 tx=0x7f714802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd266763f420a09 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.297+0000 7f71527fc640 1 -- 172.21.15.82:0/4032111284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148034070 con 0x7f715415e150 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.297+0000 7f71527fc640 1 -- 172.21.15.82:0/4032111284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148030860 con 0x7f715415e150 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.297+0000 7f71527fc640 1 -- 172.21.15.82:0/4032111284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148030c30 con 0x7f715415e150 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 -- 172.21.15.82:0/4032111284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415e150 msgr2=0x7f71540b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 --2- 172.21.15.82:0/4032111284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415e150 0x7f71540b6690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7154072910 tx=0x7f714802b9a0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 -- 172.21.15.82:0/4032111284 shutdown_connections 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 --2- 172.21.15.82:0/4032111284 >> v2:172.21.15.164:3300/0 conn(0x7f71540b6bd0 0x7f71540d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 --2- 172.21.15.82:0/4032111284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415e150 0x7f71540b6690 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 -- 172.21.15.82:0/4032111284 >> 172.21.15.82:0/4032111284 conn(0x7f7154072ec0 msgr2=0x7f715409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:52.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 -- 172.21.15.82:0/4032111284 shutdown_connections 2024-03-14T12:13:52.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.298+0000 7f7159ba9640 1 -- 172.21.15.82:0/4032111284 wait complete. 2024-03-14T12:13:52.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.299+0000 7f7159ba9640 1 Processor -- start 2024-03-14T12:13:52.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.299+0000 7f7159ba9640 1 -- start start 2024-03-14T12:13:52.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7159ba9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71540b6bd0 0x7f71540cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7159ba9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415dd70 0x7f71540afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7159ba9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f715415e150 0x7f71540b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7159ba9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71540d2040 con 0x7f715415dd70 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7159ba9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715406f510 con 0x7f71540b6bd0 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7159ba9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71540d21c0 con 0x7f715415e150 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415dd70 0x7f71540afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415dd70 0x7f71540afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48460/0 (socket says 172.21.15.82:48460) 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 -- 172.21.15.82:0/3982883939 learned_addr learned my addr 172.21.15.82:0/3982883939 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7153fff640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f715415e150 0x7f71540b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f71537fe640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71540b6bd0 0x7f71540cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 -- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f715415e150 msgr2=0x7f71540b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f715415e150 0x7f71540b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 -- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71540b6bd0 msgr2=0x7f71540cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71540b6bd0 0x7f71540cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 -- 172.21.15.82:0/3982883939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71540ce170 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f71537fe640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71540b6bd0 0x7f71540cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.300+0000 7f7152ffd640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415dd70 0x7f71540afbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f714802b9e0 tx=0x7f71480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.301+0000 7f713bfff640 1 -- 172.21.15.82:0/3982883939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148034040 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.301+0000 7f713bfff640 1 -- 172.21.15.82:0/3982883939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714803ad20 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.301+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715415aab0 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.301+0000 7f713bfff640 1 -- 172.21.15.82:0/3982883939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148044610 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.301+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715415ad70 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.301+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715404f050 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.302+0000 7f713bfff640 1 -- 172.21.15.82:0/3982883939 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f714804b060 con 0x7f715415dd70 2024-03-14T12:13:52.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.302+0000 7f713bfff640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f714006bbd0 0x7f714006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:52.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.302+0000 7f713bfff640 1 -- 172.21.15.82:0/3982883939 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71480acc10 con 0x7f715415dd70 2024-03-14T12:13:52.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.302+0000 7f71537fe640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f714006bbd0 0x7f714006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:52.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.303+0000 7f71537fe640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f714006bbd0 0x7f714006e090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7144001680 tx=0x7f7144009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:52.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.307+0000 7f713bfff640 1 -- 172.21.15.82:0/3982883939 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f714803f070 con 0x7f715415dd70 2024-03-14T12:13:52.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:52.515+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7154159080 con 0x7f715415dd70 2024-03-14T12:13:53.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.117+0000 7f713bfff640 1 -- 172.21.15.82:0/3982883939 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7148037360 con 0x7f715415dd70 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f714006bbd0 msgr2=0x7f714006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f714006bbd0 0x7f714006e090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7144001680 tx=0x7f7144009040 comp rx=0 tx=0).stop 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415dd70 msgr2=0x7f71540afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415dd70 0x7f71540afbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f714802b9e0 tx=0x7f71480046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 shutdown_connections 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f714006bbd0 0x7f714006e090 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f715415e150 0x7f71540b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f715415dd70 0x7f71540afbe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 --2- 172.21.15.82:0/3982883939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71540b6bd0 0x7f71540cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 >> 172.21.15.82:0/3982883939 conn(0x7f7154072ec0 msgr2=0x7f71540b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 shutdown_connections 2024-03-14T12:13:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.121+0000 7f7159ba9640 1 -- 172.21.15.82:0/3982883939 wait complete. 2024-03-14T12:13:53.133 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T12:13:53.134 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T12:13:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.240+0000 7fe8b5e4b640 1 Processor -- start 2024-03-14T12:13:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.240+0000 7fe8b5e4b640 1 -- start start 2024-03-14T12:13:53.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8b5e4b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8b015dec0 0x7fe8b00d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:53.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8b5e4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 0x7fe8b00d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:53.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8b5e4b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00d1420 con 0x7fe8b00ca540 2024-03-14T12:13:53.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8b5e4b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b00d1700 con 0x7fe8b015dec0 2024-03-14T12:13:53.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8b5e4b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00c80a0 con 0x7fe8b015e2a0 2024-03-14T12:13:53.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8af7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8b015dec0 0x7fe8b00d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8af7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8b015dec0 0x7fe8b00d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48472/0 (socket says 172.21.15.82:48472) 2024-03-14T12:13:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8af7fe640 1 -- 172.21.15.82:0/3407602338 learned_addr learned my addr 172.21.15.82:0/3407602338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8af7fe640 1 -- 172.21.15.82:0/3407602338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 msgr2=0x7fe8b00d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/3407602338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 0x7fe8b00d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8af7fe640 1 -- 172.21.15.82:0/3407602338 >> v1:172.21.15.164:6789/0 conn(0x7fe8b00ca540 legacy=0x7fe8b00d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.241+0000 7fe8af7fe640 1 -- 172.21.15.82:0/3407602338 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00c7a50 con 0x7fe8b015dec0 2024-03-14T12:13:53.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/3407602338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b00d0dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009870 tx=0x7fe8a002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514f78f1abdde44f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8ae7fc640 1 -- 172.21.15.82:0/3407602338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00317c0 con 0x7fe8b015dec0 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8ae7fc640 1 -- 172.21.15.82:0/3407602338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0037070 con 0x7fe8b015dec0 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8ae7fc640 1 -- 172.21.15.82:0/3407602338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0031b80 con 0x7fe8b015dec0 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3407602338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 msgr2=0x7fe8b00d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3407602338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b00d0dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009870 tx=0x7fe8a002b860 comp rx=0 tx=0).stop 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3407602338 shutdown_connections 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3407602338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 0x7fe8b00d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3407602338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b00d0dd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.242+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3407602338 >> 172.21.15.82:0/3407602338 conn(0x7fe8b00d3dd0 msgr2=0x7fe8b00d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.243+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3407602338 shutdown_connections 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.243+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3407602338 wait complete. 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 Processor -- start 2024-03-14T12:13:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 -- start start 2024-03-14T12:13:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8b00ca540 0x7fe8b00cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b01f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 0x7fe8b00cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d2270 con 0x7fe8b015dec0 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d20f0 con 0x7fe8b00ca540 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.244+0000 7fe8b5e4b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d23f0 con 0x7fe8b015e2a0 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b01f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b01f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48480/0 (socket says 172.21.15.82:48480) 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 -- 172.21.15.82:0/3114026906 learned_addr learned my addr 172.21.15.82:0/3114026906 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8affff640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 0x7fe8b00cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 -- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 msgr2=0x7fe8b00cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8b00ca540 0x7fe8b00cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 0x7fe8b00cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 -- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8b00ca540 msgr2=0x7fe8b00cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8b00ca540 0x7fe8b00cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 -- 172.21.15.82:0/3114026906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00b7b10 con 0x7fe8b015dec0 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8aeffd640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b01f4ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400b3f0 tx=0x7fe8a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe897fff640 1 -- 172.21.15.82:0/3114026906 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4015020 con 0x7fe8b015dec0 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe897fff640 1 -- 172.21.15.82:0/3114026906 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4011830 con 0x7fe8b015dec0 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b00b7df0 con 0x7fe8b015dec0 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe897fff640 1 -- 172.21.15.82:0/3114026906 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4011c00 con 0x7fe8b015dec0 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.245+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b00b83b0 con 0x7fe8b015dec0 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.246+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b004f050 con 0x7fe8b015dec0 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.247+0000 7fe897fff640 1 -- 172.21.15.82:0/3114026906 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe8a4018020 con 0x7fe8b015dec0 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.247+0000 7fe897fff640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bbd0 0x7fe89c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.247+0000 7fe897fff640 1 -- 172.21.15.82:0/3114026906 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a408a8b0 con 0x7fe8b015dec0 2024-03-14T12:13:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.247+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bbd0 0x7fe89c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.247+0000 7fe8af7fe640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bbd0 0x7fe89c06e090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0036000 tx=0x7fe8a0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.251+0000 7fe897fff640 1 -- 172.21.15.82:0/3114026906 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe8a401f020 con 0x7fe8b015dec0 2024-03-14T12:13:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:53.460+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe8b0074180 con 0x7fe8b015dec0 2024-03-14T12:13:54.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.122+0000 7fe897fff640 1 -- 172.21.15.82:0/3114026906 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe8a40172c0 con 0x7fe8b015dec0 2024-03-14T12:13:54.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bbd0 msgr2=0x7fe89c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bbd0 0x7fe89c06e090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0036000 tx=0x7fe8a0008030 comp rx=0 tx=0).stop 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 msgr2=0x7fe8b01f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b01f4ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400b3f0 tx=0x7fe8a400b8b0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 shutdown_connections 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe89c06bbd0 0x7fe89c06e090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8b015e2a0 0x7fe8b00cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b015dec0 0x7fe8b01f4ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 --2- 172.21.15.82:0/3114026906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8b00ca540 0x7fe8b00cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 >> 172.21.15.82:0/3114026906 conn(0x7fe8b00d3dd0 msgr2=0x7fe8b006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:54.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 shutdown_connections 2024-03-14T12:13:54.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.125+0000 7fe8b5e4b640 1 -- 172.21.15.82:0/3114026906 wait complete. 2024-03-14T12:13:54.138 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T12:13:54.138 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:13:54.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.241+0000 7faf59569640 1 Processor -- start 2024-03-14T12:13:54.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.241+0000 7faf59569640 1 -- start start 2024-03-14T12:13:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf59569640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf59569640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf5415e670 0x7faf540b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf59569640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf54143f60 0x7faf5415b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf59569640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf54072060 con 0x7faf5415e670 2024-03-14T12:13:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf59569640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5415ebe0 con 0x7faf54143f60 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf59569640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415eec0 con 0x7faf54071740 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf5415e670 0x7faf540b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf5415e670 0x7faf540b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48486/0 (socket says 172.21.15.82:48486) 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 -- 172.21.15.82:0/4163612977 learned_addr learned my addr 172.21.15.82:0/4163612977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf52ffd640 1 --2- 172.21.15.82:0/4163612977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf537fe640 1 --2- 172.21.15.82:0/4163612977 >> v2:172.21.15.164:3300/0 conn(0x7faf54143f60 0x7faf5415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 -- 172.21.15.82:0/4163612977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 msgr2=0x7faf54071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 --2- 172.21.15.82:0/4163612977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 -- 172.21.15.82:0/4163612977 >> v2:172.21.15.164:3300/0 conn(0x7faf54143f60 msgr2=0x7faf5415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 --2- 172.21.15.82:0/4163612977 >> v2:172.21.15.164:3300/0 conn(0x7faf54143f60 0x7faf5415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf527fc640 1 -- 172.21.15.82:0/4163612977 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5415f1a0 con 0x7faf5415e670 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.242+0000 7faf537fe640 1 --2- 172.21.15.82:0/4163612977 >> v2:172.21.15.164:3300/0 conn(0x7faf54143f60 0x7faf5415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.243+0000 7faf527fc640 1 --2- 172.21.15.82:0/4163612977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5415e670 0x7faf540b2940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf54072ac0 tx=0x7faf4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e978081de8f1334 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.243+0000 7faf51ffb640 1 -- 172.21.15.82:0/4163612977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48035020 con 0x7faf5415e670 2024-03-14T12:13:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.243+0000 7faf51ffb640 1 -- 172.21.15.82:0/4163612977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48031870 con 0x7faf5415e670 2024-03-14T12:13:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.243+0000 7faf51ffb640 1 -- 172.21.15.82:0/4163612977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48031c40 con 0x7faf5415e670 2024-03-14T12:13:54.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 -- 172.21.15.82:0/4163612977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5415e670 msgr2=0x7faf540b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 --2- 172.21.15.82:0/4163612977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5415e670 0x7faf540b2940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf54072ac0 tx=0x7faf4802b910 comp rx=0 tx=0).stop 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 -- 172.21.15.82:0/4163612977 shutdown_connections 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 --2- 172.21.15.82:0/4163612977 >> v2:172.21.15.164:3300/0 conn(0x7faf54143f60 0x7faf5415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 --2- 172.21.15.82:0/4163612977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5415e670 0x7faf540b2940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 --2- 172.21.15.82:0/4163612977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 -- 172.21.15.82:0/4163612977 >> 172.21.15.82:0/4163612977 conn(0x7faf540b1c70 msgr2=0x7faf540cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 -- 172.21.15.82:0/4163612977 shutdown_connections 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.244+0000 7faf59569640 1 -- 172.21.15.82:0/4163612977 wait complete. 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.245+0000 7faf59569640 1 Processor -- start 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.245+0000 7faf59569640 1 -- start start 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf59569640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf59569640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf54143f60 0x7faf5413c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf59569640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5415e670 0x7faf541575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf59569640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415b080 con 0x7faf54143f60 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf59569640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf54143ce0 con 0x7faf5415e670 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf59569640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf54144670 con 0x7faf54071740 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf54143f60 0x7faf5413c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf54143f60 0x7faf5413c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48494/0 (socket says 172.21.15.82:48494) 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 -- 172.21.15.82:0/4158954040 learned_addr learned my addr 172.21.15.82:0/4158954040 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf52ffd640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 -- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 msgr2=0x7faf54140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf537fe640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5415e670 0x7faf541575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 -- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5415e670 msgr2=0x7faf541575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5415e670 0x7faf541575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 -- 172.21.15.82:0/4158954040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54157af0 con 0x7faf54143f60 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf527fc640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf54143f60 0x7faf5413c280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf4802b950 tx=0x7faf480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.246+0000 7faf3b7fe640 1 -- 172.21.15.82:0/4158954040 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48041070 con 0x7faf54143f60 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.247+0000 7faf3b7fe640 1 -- 172.21.15.82:0/4158954040 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4803b430 con 0x7faf54143f60 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.247+0000 7faf3b7fe640 1 -- 172.21.15.82:0/4158954040 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48035040 con 0x7faf54143f60 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.247+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5413edd0 con 0x7faf54143f60 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.247+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5413f390 con 0x7faf54143f60 2024-03-14T12:13:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.248+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5404f050 con 0x7faf54143f60 2024-03-14T12:13:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.248+0000 7faf3b7fe640 1 -- 172.21.15.82:0/4158954040 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faf4804c060 con 0x7faf54143f60 2024-03-14T12:13:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.248+0000 7faf3b7fe640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bbd0 0x7faf4006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.249+0000 7faf3b7fe640 1 -- 172.21.15.82:0/4158954040 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf480aca50 con 0x7faf54143f60 2024-03-14T12:13:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.249+0000 7faf52ffd640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bbd0 0x7faf4006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.249+0000 7faf52ffd640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bbd0 0x7faf4006e090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf440070b0 tx=0x7faf44007040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.253+0000 7faf3b7fe640 1 -- 172.21.15.82:0/4158954040 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7faf54157af0 con 0x7faf54143f60 2024-03-14T12:13:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.460+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf5413cb00 con 0x7faf54143f60 2024-03-14T12:13:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.460+0000 7faf3b7fe640 1 -- 172.21.15.82:0/4158954040 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7faf4807b540 con 0x7faf54143f60 2024-03-14T12:13:54.461 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:54.461 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:13:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bbd0 msgr2=0x7faf4006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bbd0 0x7faf4006e090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf440070b0 tx=0x7faf44007040 comp rx=0 tx=0).stop 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf54143f60 msgr2=0x7faf5413c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf54143f60 0x7faf5413c280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf4802b950 tx=0x7faf480046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 shutdown_connections 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5415e670 0x7faf541575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf54143f60 0x7faf5413c280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bbd0 0x7faf4006e090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 --2- 172.21.15.82:0/4158954040 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf54071740 0x7faf54140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 >> 172.21.15.82:0/4158954040 conn(0x7faf540b1c70 msgr2=0x7faf54159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 shutdown_connections 2024-03-14T12:13:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.462+0000 7faf59569640 1 -- 172.21.15.82:0/4158954040 wait complete. 2024-03-14T12:13:54.473 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:13:54.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.573+0000 7f83f7980640 1 Processor -- start 2024-03-14T12:13:54.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.573+0000 7f83f7980640 1 -- start start 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f7980640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83f015e190 0x7f83f00b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f7980640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f83f00ce480 0x7f83f00ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f7980640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f006ed00 con 0x7f83f00ccb40 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f7980640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f00cee20 con 0x7f83f015e190 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f7980640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f00cc0f0 con 0x7f83f00ce480 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f571c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83f015e190 0x7f83f00b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f571c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83f015e190 0x7f83f00b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48496/0 (socket says 172.21.15.82:48496) 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f571c640 1 -- 172.21.15.82:0/2343969592 learned_addr learned my addr 172.21.15.82:0/2343969592 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f5f1d640 1 --1- 172.21.15.82:0/2343969592 >> v1:172.21.15.82:6789/0 conn(0x7f83f00ccb40 0x7f83f00ccfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34496/0 (socket says 172.21.15.82:34496) 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f4f1b640 1 --2- 172.21.15.82:0/2343969592 >> v2:172.21.15.164:3300/0 conn(0x7f83f00ce480 0x7f83f00ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f571c640 1 -- 172.21.15.82:0/2343969592 >> v2:172.21.15.164:3300/0 conn(0x7f83f00ce480 msgr2=0x7f83f00ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f571c640 1 --2- 172.21.15.82:0/2343969592 >> v2:172.21.15.164:3300/0 conn(0x7f83f00ce480 0x7f83f00ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f571c640 1 -- 172.21.15.82:0/2343969592 >> v1:172.21.15.82:6789/0 conn(0x7f83f00ccb40 legacy=0x7f83f00ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.574+0000 7f83f571c640 1 -- 172.21.15.82:0/2343969592 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f006efe0 con 0x7f83f015e190 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.575+0000 7f83f571c640 1 --2- 172.21.15.82:0/2343969592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f015e190 0x7f83f00b4130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83f0071280 tx=0x7f83ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe79e738f092d8a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.575+0000 7f83e7fff640 1 -- 172.21.15.82:0/2343969592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec02e070 con 0x7f83f015e190 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.575+0000 7f83e7fff640 1 -- 172.21.15.82:0/2343969592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec037070 con 0x7f83f015e190 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.575+0000 7f83e7fff640 1 -- 172.21.15.82:0/2343969592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec031a00 con 0x7f83f015e190 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.575+0000 7f83f7980640 1 -- 172.21.15.82:0/2343969592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f015e190 msgr2=0x7f83f00b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.575+0000 7f83f7980640 1 --2- 172.21.15.82:0/2343969592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f015e190 0x7f83f00b4130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83f0071280 tx=0x7f83ec02b880 comp rx=0 tx=0).stop 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.576+0000 7f83f7980640 1 -- 172.21.15.82:0/2343969592 shutdown_connections 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.576+0000 7f83f7980640 1 --2- 172.21.15.82:0/2343969592 >> v2:172.21.15.164:3300/0 conn(0x7f83f00ce480 0x7f83f00ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.576+0000 7f83f7980640 1 --2- 172.21.15.82:0/2343969592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f015e190 0x7f83f00b4130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.576+0000 7f83f7980640 1 -- 172.21.15.82:0/2343969592 >> 172.21.15.82:0/2343969592 conn(0x7f83f00b3610 msgr2=0x7f83f00b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:54.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.576+0000 7f83f7980640 1 -- 172.21.15.82:0/2343969592 shutdown_connections 2024-03-14T12:13:54.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.576+0000 7f83f7980640 1 -- 172.21.15.82:0/2343969592 wait complete. 2024-03-14T12:13:54.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 Processor -- start 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 -- start start 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00ccb40 0x7f83f01f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f00ce480 0x7f83f006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83f015e190 0x7f83f006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00d0c40 con 0x7f83f00ccb40 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00cc6b0 con 0x7f83f00ce480 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f7980640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00d0dc0 con 0x7f83f015e190 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.577+0000 7f83f571c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00ccb40 0x7f83f01f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00ccb40 0x7f83f01f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48512/0 (socket says 172.21.15.82:48512) 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 -- 172.21.15.82:0/525894361 learned_addr learned my addr 172.21.15.82:0/525894361 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f4f1b640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f00ce480 0x7f83f006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f5f1d640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83f015e190 0x7f83f006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 -- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83f015e190 msgr2=0x7f83f006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83f015e190 0x7f83f006d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 -- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f00ce480 msgr2=0x7f83f006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f00ce480 0x7f83f006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 -- 172.21.15.82:0/525894361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f015a970 con 0x7f83f00ccb40 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f5f1d640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83f015e190 0x7f83f006d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f4f1b640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f00ce480 0x7f83f006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f571c640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00ccb40 0x7f83f01f3090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f83ec000c00 tx=0x7f83ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83e5ffb640 1 -- 172.21.15.82:0/525894361 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec032670 con 0x7f83f00ccb40 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83e5ffb640 1 -- 172.21.15.82:0/525894361 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec032850 con 0x7f83f00ccb40 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f015b820 con 0x7f83f00ccb40 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83e5ffb640 1 -- 172.21.15.82:0/525894361 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec032c40 con 0x7f83f00ccb40 2024-03-14T12:13:54.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.578+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f015aef0 con 0x7f83f00ccb40 2024-03-14T12:13:54.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.579+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f004f050 con 0x7f83f00ccb40 2024-03-14T12:13:54.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.580+0000 7f83e5ffb640 1 -- 172.21.15.82:0/525894361 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f83ec04c060 con 0x7f83f00ccb40 2024-03-14T12:13:54.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.580+0000 7f83e5ffb640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f83d406bbd0 0x7f83d406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.580+0000 7f83f4f1b640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f83d406bbd0 0x7f83d406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.580+0000 7f83e5ffb640 1 -- 172.21.15.82:0/525894361 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83ec0ac450 con 0x7f83f00ccb40 2024-03-14T12:13:54.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.581+0000 7f83f4f1b640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f83d406bbd0 0x7f83d406e090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83f015ca80 tx=0x7f83e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.585+0000 7f83e5ffb640 1 -- 172.21.15.82:0/525894361 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f83ec040070 con 0x7f83f00ccb40 2024-03-14T12:13:54.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.759+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83f0074740 con 0x7f83f00ccb40 2024-03-14T12:13:54.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.760+0000 7f83e5ffb640 1 -- 172.21.15.82:0/525894361 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f83ec07afd0 con 0x7f83f00ccb40 2024-03-14T12:13:54.761 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:54.761 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f83d406bbd0 msgr2=0x7f83d406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f83d406bbd0 0x7f83d406e090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83f015ca80 tx=0x7f83e0008040 comp rx=0 tx=0).stop 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00ccb40 msgr2=0x7f83f01f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00ccb40 0x7f83f01f3090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f83ec000c00 tx=0x7f83ec0046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 shutdown_connections 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f83d406bbd0 0x7f83d406e090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83f015e190 0x7f83f006d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83f00ce480 0x7f83f006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 --2- 172.21.15.82:0/525894361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00ccb40 0x7f83f01f3090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 >> 172.21.15.82:0/525894361 conn(0x7f83f00b3610 msgr2=0x7f83f00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:54.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 shutdown_connections 2024-03-14T12:13:54.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.762+0000 7f83f7980640 1 -- 172.21.15.82:0/525894361 wait complete. 2024-03-14T12:13:54.774 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T12:13:54.774 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:13:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.873+0000 7f9b732cb640 1 Processor -- start 2024-03-14T12:13:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.873+0000 7f9b732cb640 1 -- start start 2024-03-14T12:13:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b732cb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b6c15f1f0 0x7f9b6c06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b732cb640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c0b58f0 0x7f9b6c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b732cb640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c0b01d0 con 0x7f9b6c0af150 2024-03-14T12:13:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b732cb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6c0aff90 con 0x7f9b6c15f1f0 2024-03-14T12:13:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b732cb640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c15e710 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b70866640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b6c15f1f0 0x7f9b6c06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b70866640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b6c15f1f0 0x7f9b6c06ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48514/0 (socket says 172.21.15.82:48514) 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.874+0000 7f9b70866640 1 -- 172.21.15.82:0/3826206559 learned_addr learned my addr 172.21.15.82:0/3826206559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b71868640 1 --2- 172.21.15.82:0/3826206559 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c0b58f0 0x7f9b6c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b70866640 1 -- 172.21.15.82:0/3826206559 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c0b58f0 msgr2=0x7f9b6c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b70866640 1 --2- 172.21.15.82:0/3826206559 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c0b58f0 0x7f9b6c06b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b70866640 1 -- 172.21.15.82:0/3826206559 >> v1:172.21.15.164:6789/0 conn(0x7f9b6c0af150 legacy=0x7f9b6c0af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b70866640 1 -- 172.21.15.82:0/3826206559 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c15e9f0 con 0x7f9b6c15f1f0 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b70866640 1 --2- 172.21.15.82:0/3826206559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c15f1f0 0x7f9b6c06ad10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009870 tx=0x7f9b5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c625fa6cb623449a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b63fff640 1 -- 172.21.15.82:0/3826206559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c035020 con 0x7f9b6c15f1f0 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b63fff640 1 -- 172.21.15.82:0/3826206559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c031640 con 0x7f9b6c15f1f0 2024-03-14T12:13:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.875+0000 7f9b63fff640 1 -- 172.21.15.82:0/3826206559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c02d040 con 0x7f9b6c15f1f0 2024-03-14T12:13:54.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 -- 172.21.15.82:0/3826206559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c15f1f0 msgr2=0x7f9b6c06ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 --2- 172.21.15.82:0/3826206559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c15f1f0 0x7f9b6c06ad10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009870 tx=0x7f9b5c02b8f0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 -- 172.21.15.82:0/3826206559 shutdown_connections 2024-03-14T12:13:54.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 --2- 172.21.15.82:0/3826206559 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c0b58f0 0x7f9b6c06b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 --2- 172.21.15.82:0/3826206559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c15f1f0 0x7f9b6c06ad10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 -- 172.21.15.82:0/3826206559 >> 172.21.15.82:0/3826206559 conn(0x7f9b6c0d0a20 msgr2=0x7f9b6c0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 -- 172.21.15.82:0/3826206559 shutdown_connections 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.876+0000 7f9b732cb640 1 -- 172.21.15.82:0/3826206559 wait complete. 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.877+0000 7f9b732cb640 1 Processor -- start 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.877+0000 7f9b732cb640 1 -- start start 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b732cb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b6c0af150 0x7f9b6c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b732cb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c0b58f0 0x7f9b6c1f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b732cb640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c15f1f0 0x7f9b6c14f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b732cb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c157cd0 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b732cb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c15e400 con 0x7f9b6c0af150 2024-03-14T12:13:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b732cb640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c157e50 con 0x7f9b6c15f1f0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c0b58f0 0x7f9b6c1f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c0b58f0 0x7f9b6c1f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48524/0 (socket says 172.21.15.82:48524) 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 -- 172.21.15.82:0/996141609 learned_addr learned my addr 172.21.15.82:0/996141609 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b71868640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c15f1f0 0x7f9b6c14f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b71067640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b6c0af150 0x7f9b6c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 -- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c15f1f0 msgr2=0x7f9b6c14f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c15f1f0 0x7f9b6c14f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 -- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b6c0af150 msgr2=0x7f9b6c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b6c0af150 0x7f9b6c0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 -- 172.21.15.82:0/996141609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c1e7750 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.878+0000 7f9b70866640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c0b58f0 0x7f9b6c1f8340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c000c00 tx=0x7f9b5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.879+0000 7f9b61ffb640 1 -- 172.21.15.82:0/996141609 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c040070 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.879+0000 7f9b61ffb640 1 -- 172.21.15.82:0/996141609 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c039bf0 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.879+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c1e7a10 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.879+0000 7f9b61ffb640 1 -- 172.21.15.82:0/996141609 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c035040 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.879+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c1e7fd0 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.880+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c0b2550 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.880+0000 7f9b61ffb640 1 -- 172.21.15.82:0/996141609 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9b5c04b060 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.880+0000 7f9b61ffb640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b5006bbd0 0x7f9b5006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.880+0000 7f9b61ffb640 1 -- 172.21.15.82:0/996141609 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b5c0ac390 con 0x7f9b6c0b58f0 2024-03-14T12:13:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.881+0000 7f9b71067640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b5006bbd0 0x7f9b5006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.881+0000 7f9b71067640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b5006bbd0 0x7f9b5006e090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001d40 tx=0x7f9b68009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:54.885+0000 7f9b61ffb640 1 -- 172.21.15.82:0/996141609 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9b6c0b2550 con 0x7f9b6c0b58f0 2024-03-14T12:13:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.053+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b6c04f050 con 0x7f9b5006bbd0 2024-03-14T12:13:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.059+0000 7f9b61ffb640 1 -- 172.21.15.82:0/996141609 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237079 (secure 0 0 0) 0x7f9b6c04f050 con 0x7f9b5006bbd0 2024-03-14T12:13:55.061 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:13:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b5006bbd0 msgr2=0x7f9b5006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b5006bbd0 0x7f9b5006e090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001d40 tx=0x7f9b68009040 comp rx=0 tx=0).stop 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c0b58f0 msgr2=0x7f9b6c1f8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c0b58f0 0x7f9b6c1f8340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c000c00 tx=0x7f9b5c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 shutdown_connections 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b5006bbd0 0x7f9b5006e090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b6c15f1f0 0x7f9b6c14f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b6c0b58f0 0x7f9b6c1f8340 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 --2- 172.21.15.82:0/996141609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b6c0af150 0x7f9b6c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.061+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 >> 172.21.15.82:0/996141609 conn(0x7f9b6c0d0a20 msgr2=0x7f9b6c06e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.062+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 shutdown_connections 2024-03-14T12:13:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.062+0000 7f9b732cb640 1 -- 172.21.15.82:0/996141609 wait complete. 2024-03-14T12:13:55.072 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-14T12:13:53.498460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219396,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755324,"statfs":{"total":773094113280,"available":772869451776,"internally_reserved":0,"allocated":2531328,"data_stored":1584240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000714"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-03-14T12:13:42.951955+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:13:42.951955+0000","last_peered":"2024-03-14T12:13:42.951955+0000","last_clean":"2024-03-14T12:13:42.951955+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:13:42.951955+0000","last_undegraded":"2024-03-14T12:13:42.951955+0000","last_fullsized":"2024-03-14T12:13:42.951955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:13:55.077 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:13:55.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.213+0000 7f00c1f53640 1 Processor -- start 2024-03-14T12:13:55.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.213+0000 7f00c1f53640 1 -- start start 2024-03-14T12:13:55.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00c1f53640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f00bc15e670 0x7f00bc0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00c1f53640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc15ebe0 con 0x7f00bc071740 2024-03-14T12:13:55.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00c1f53640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc15eec0 con 0x7f00bc143f60 2024-03-14T12:13:55.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00c1f53640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc15f1a0 con 0x7f00bc15e670 2024-03-14T12:13:55.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00bb7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f00bc071740 0x7f00bc071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34498/0 (socket says 172.21.15.82:34498) 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00bb7fe640 1 -- 172.21.15.82:0/803102475 learned_addr learned my addr 172.21.15.82:0/803102475 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388847452 0 0) 0x7f00bc15ebe0 con 0x7f00bc071740 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a4003520 con 0x7f00bc071740 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00baffd640 1 --2- 172.21.15.82:0/803102475 >> v2:172.21.15.164:3300/0 conn(0x7f00bc15e670 0x7f00bc0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1894359696 0 0) 0x7f00a4003520 con 0x7f00bc071740 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 >> v2:172.21.15.164:3300/0 conn(0x7f00bc15e670 msgr2=0x7f00bc0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 --2- 172.21.15.82:0/803102475 >> v2:172.21.15.164:3300/0 conn(0x7f00bc15e670 0x7f00bc0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 >> v1:172.21.15.164:6789/0 conn(0x7f00bc143f60 legacy=0x7f00bc15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc15f480 con 0x7f00bc071740 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f00ac001670 con 0x7f00bc071740 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 >> v1:172.21.15.82:6789/0 conn(0x7f00bc071740 legacy=0x7f00bc071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00a40039e0 0x7f00a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a4004800 0x7f00a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc15f1a0 con 0x7f00a40051c0 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.214+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a400c090 con 0x7f00a40039e0 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a400c290 con 0x7f00a4004800 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bbfff640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bb7fe640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00a40039e0 0x7f00a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00baffd640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a4004800 0x7f00a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bbfff640 1 -- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a4004800 msgr2=0x7f00a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bbfff640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a4004800 0x7f00a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bbfff640 1 -- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00a40039e0 msgr2=0x7f00a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bbfff640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00a40039e0 0x7f00a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bbfff640 1 -- 172.21.15.82:0/803102475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a4003520 con 0x7f00a40051c0 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.215+0000 7f00bbfff640 1 -- 172.21.15.82:0/803102475 reap_dead start 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00bbfff640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00a400b9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001e50 tx=0x7f00a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684c3deea9bca5ab server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4001a70 con 0x7f00a40051c0 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b4003bb0 con 0x7f00a40051c0 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00ba7fc640 1 -- 172.21.15.82:0/803102475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40050d0 con 0x7f00a40051c0 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00c1f53640 1 -- 172.21.15.82:0/803102475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 msgr2=0x7f00a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00c1f53640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00a400b9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001e50 tx=0x7f00a4005830 comp rx=0 tx=0).stop 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00c1f53640 1 -- 172.21.15.82:0/803102475 shutdown_connections 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00c1f53640 1 --2- 172.21.15.82:0/803102475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00a400b9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00c1f53640 1 -- 172.21.15.82:0/803102475 >> 172.21.15.82:0/803102475 conn(0x7f00bc0b1c70 msgr2=0x7f00bc0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00c1f53640 1 -- 172.21.15.82:0/803102475 shutdown_connections 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.216+0000 7f00c1f53640 1 -- 172.21.15.82:0/803102475 wait complete. 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.217+0000 7f00c1f53640 1 Processor -- start 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00c1f53640 1 -- start start 2024-03-14T12:13:55.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00c1f53640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00bc13cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00c1f53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00bc13e160 0x7f00bc13e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00c1f53640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00bc0cce40 0x7f00bc0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00c1f53640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc144670 con 0x7f00a40051c0 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00c1f53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc15b080 con 0x7f00bc13e160 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00c1f53640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc1447f0 con 0x7f00bc0cce40 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00bb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00bc13cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00bb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00bc13cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48546/0 (socket says 172.21.15.82:48546) 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.218+0000 7f00bb7fe640 1 -- 172.21.15.82:0/1649315283 learned_addr learned my addr 172.21.15.82:0/1649315283 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bbfff640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00bc0cce40 0x7f00bc0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00baffd640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00bc13e160 0x7f00bc13e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bb7fe640 1 -- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00bc0cce40 msgr2=0x7f00bc0cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bb7fe640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00bc0cce40 0x7f00bc0cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bb7fe640 1 -- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00bc13e160 msgr2=0x7f00bc13e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bb7fe640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00bc13e160 0x7f00bc13e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bb7fe640 1 -- 172.21.15.82:0/1649315283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0cec60 con 0x7f00a40051c0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bbfff640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00bc0cce40 0x7f00bc0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00baffd640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00bc13e160 0x7f00bc13e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00bb7fe640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00bc13cc00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f00ac004fa0 tx=0x7f00ac0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00a3fff640 1 -- 172.21.15.82:0/1649315283 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac007740 con 0x7f00a40051c0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00a3fff640 1 -- 172.21.15.82:0/1649315283 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac018070 con 0x7f00a40051c0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc0cdbc0 con 0x7f00a40051c0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00a3fff640 1 -- 172.21.15.82:0/1649315283 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00fbb0 con 0x7f00a40051c0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.219+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc0ce180 con 0x7f00a40051c0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.220+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc04f050 con 0x7f00a40051c0 2024-03-14T12:13:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.220+0000 7f00a3fff640 1 -- 172.21.15.82:0/1649315283 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f00ac00fd50 con 0x7f00a40051c0 2024-03-14T12:13:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.221+0000 7f00a3fff640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00a806bda0 0x7f00a806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.221+0000 7f00a3fff640 1 -- 172.21.15.82:0/1649315283 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00ac08aaf0 con 0x7f00a40051c0 2024-03-14T12:13:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.221+0000 7f00baffd640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00a806bda0 0x7f00a806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.221+0000 7f00baffd640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00a806bda0 0x7f00a806e260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00b00093d0 tx=0x7f00b0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.225+0000 7f00a3fff640 1 -- 172.21.15.82:0/1649315283 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f00bc0cec60 con 0x7f00a40051c0 2024-03-14T12:13:55.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.392+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00bc0d07b0 con 0x7f00a806bda0 2024-03-14T12:13:55.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.397+0000 7f00a3fff640 1 -- 172.21.15.82:0/1649315283 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237079 (secure 0 0 0) 0x7f00bc0d07b0 con 0x7f00a806bda0 2024-03-14T12:13:55.400 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:55.401 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:13:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00a806bda0 msgr2=0x7f00a806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00a806bda0 0x7f00a806e260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00b00093d0 tx=0x7f00b0008040 comp rx=0 tx=0).stop 2024-03-14T12:13:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 msgr2=0x7f00bc13cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00bc13cc00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f00ac004fa0 tx=0x7f00ac0074f0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 shutdown_connections 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00a806bda0 0x7f00a806e260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00bc0cce40 0x7f00bc0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00bc13e160 0x7f00bc13e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 --2- 172.21.15.82:0/1649315283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40051c0 0x7f00bc13cc00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.400+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 >> 172.21.15.82:0/1649315283 conn(0x7f00bc0b1c70 msgr2=0x7f00bc141370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.401+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 shutdown_connections 2024-03-14T12:13:55.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.401+0000 7f00c1f53640 1 -- 172.21.15.82:0/1649315283 wait complete. 2024-03-14T12:13:55.411 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-14T12:13:53.498460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219396,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755324,"statfs":{"total":773094113280,"available":772869451776,"internally_reserved":0,"allocated":2531328,"data_stored":1584240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000714"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-03-14T12:13:42.951955+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:13:42.951955+0000","last_peered":"2024-03-14T12:13:42.951955+0000","last_clean":"2024-03-14T12:13:42.951955+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:13:42.951955+0000","last_undegraded":"2024-03-14T12:13:42.951955+0000","last_fullsized":"2024-03-14T12:13:42.951955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:50.546003+0000","last_change":"2024-03-14T12:13:50.546003+0000","last_active":"2024-03-14T12:13:50.546003+0000","last_peered":"2024-03-14T12:13:50.546003+0000","last_clean":"2024-03-14T12:13:50.546003+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:50.546003+0000","last_undegraded":"2024-03-14T12:13:50.546003+0000","last_fullsized":"2024-03-14T12:13:50.546003+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:13:49.542136+0000","last_change":"2024-03-14T12:13:49.542136+0000","last_active":"2024-03-14T12:13:49.542136+0000","last_peered":"2024-03-14T12:13:49.542136+0000","last_clean":"2024-03-14T12:13:49.542136+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:13:49.542136+0000","last_undegraded":"2024-03-14T12:13:49.542136+0000","last_fullsized":"2024-03-14T12:13:49.542136+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:13:55.417 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:13:55.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.552+0000 7f2d58b10640 1 Processor -- start 2024-03-14T12:13:55.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.552+0000 7f2d58b10640 1 -- start start 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d58b10640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d540cdd60 0x7f2d540ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d58b10640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2d54144ac0 0x7f2d54144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d58b10640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d540ce8b0 con 0x7f2d5415f7a0 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d58b10640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d540cf150 con 0x7f2d540cdd60 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d58b10640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d540ceb90 con 0x7f2d54144ac0 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d4bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d540cdd60 0x7f2d540ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d4bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d540cdd60 0x7f2d540ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48548/0 (socket says 172.21.15.82:48548) 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d4bfff640 1 -- 172.21.15.82:0/2235070323 learned_addr learned my addr 172.21.15.82:0/2235070323 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:55.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d52d9d640 1 --2- 172.21.15.82:0/2235070323 >> v2:172.21.15.164:3300/0 conn(0x7f2d54144ac0 0x7f2d54144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d4bfff640 1 -- 172.21.15.82:0/2235070323 >> v2:172.21.15.164:3300/0 conn(0x7f2d54144ac0 msgr2=0x7f2d54144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d4bfff640 1 --2- 172.21.15.82:0/2235070323 >> v2:172.21.15.164:3300/0 conn(0x7f2d54144ac0 0x7f2d54144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d4bfff640 1 -- 172.21.15.82:0/2235070323 >> v1:172.21.15.82:6789/0 conn(0x7f2d5415f7a0 legacy=0x7f2d5415fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d4bfff640 1 -- 172.21.15.82:0/2235070323 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d540cee70 con 0x7f2d540cdd60 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.553+0000 7f2d52d9d640 1 --2- 172.21.15.82:0/2235070323 >> v2:172.21.15.164:3300/0 conn(0x7f2d54144ac0 0x7f2d54144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.554+0000 7f2d4bfff640 1 --2- 172.21.15.82:0/2235070323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d540ce210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d540cc280 tx=0x7f2d3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06ba6f5f5919bcc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.554+0000 7f2d51d9b640 1 -- 172.21.15.82:0/2235070323 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c035020 con 0x7f2d540cdd60 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.554+0000 7f2d51d9b640 1 -- 172.21.15.82:0/2235070323 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c031870 con 0x7f2d540cdd60 2024-03-14T12:13:55.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.554+0000 7f2d51d9b640 1 -- 172.21.15.82:0/2235070323 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c031c40 con 0x7f2d540cdd60 2024-03-14T12:13:55.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 -- 172.21.15.82:0/2235070323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 msgr2=0x7f2d540ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 --2- 172.21.15.82:0/2235070323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d540ce210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d540cc280 tx=0x7f2d3c02b910 comp rx=0 tx=0).stop 2024-03-14T12:13:55.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 -- 172.21.15.82:0/2235070323 shutdown_connections 2024-03-14T12:13:55.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 --2- 172.21.15.82:0/2235070323 >> v2:172.21.15.164:3300/0 conn(0x7f2d54144ac0 0x7f2d54144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 --2- 172.21.15.82:0/2235070323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d540ce210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 -- 172.21.15.82:0/2235070323 >> 172.21.15.82:0/2235070323 conn(0x7f2d540b1c70 msgr2=0x7f2d540cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:55.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 -- 172.21.15.82:0/2235070323 shutdown_connections 2024-03-14T12:13:55.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.555+0000 7f2d58b10640 1 -- 172.21.15.82:0/2235070323 wait complete. 2024-03-14T12:13:55.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.556+0000 7f2d58b10640 1 Processor -- start 2024-03-14T12:13:55.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.556+0000 7f2d58b10640 1 -- start start 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d54157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d54144ac0 0x7f2d54143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d5415f7a0 0x7f2d5406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540cc670 con 0x7f2d540cdd60 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540cc4f0 con 0x7f2d5415f7a0 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54070630 con 0x7f2d54144ac0 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d54157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d54157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48558/0 (socket says 172.21.15.82:48558) 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 -- 172.21.15.82:0/3320428768 learned_addr learned my addr 172.21.15.82:0/3320428768 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d52d9d640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d5415f7a0 0x7f2d5406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d4bfff640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d54144ac0 0x7f2d54143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 -- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d54144ac0 msgr2=0x7f2d54143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d54144ac0 0x7f2d54143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 -- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d5415f7a0 msgr2=0x7f2d5406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d5415f7a0 0x7f2d5406c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 -- 172.21.15.82:0/3320428768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54142970 con 0x7f2d540cdd60 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d5259c640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d54157c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c400 tx=0x7f2d4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d4affd640 1 -- 172.21.15.82:0/3320428768 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d440052d0 con 0x7f2d540cdd60 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d4affd640 1 -- 172.21.15.82:0/3320428768 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d44005490 con 0x7f2d540cdd60 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d54142c50 con 0x7f2d540cdd60 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d4affd640 1 -- 172.21.15.82:0/3320428768 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44007260 con 0x7f2d540cdd60 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.557+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d54143210 con 0x7f2d540cdd60 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.558+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5404f050 con 0x7f2d540cdd60 2024-03-14T12:13:55.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.559+0000 7f2d4affd640 1 -- 172.21.15.82:0/3320428768 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2d44007400 con 0x7f2d540cdd60 2024-03-14T12:13:55.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.559+0000 7f2d4affd640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2d3406bbd0 0x7f2d3406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.559+0000 7f2d4bfff640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2d3406bbd0 0x7f2d3406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.559+0000 7f2d4affd640 1 -- 172.21.15.82:0/3320428768 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d44089fb0 con 0x7f2d540cdd60 2024-03-14T12:13:55.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.560+0000 7f2d4bfff640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2d3406bbd0 0x7f2d3406e090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c036000 tx=0x7f2d3c03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.564+0000 7f2d4affd640 1 -- 172.21.15.82:0/3320428768 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2d44058aa0 con 0x7f2d540cdd60 2024-03-14T12:13:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.736+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d5415ad20 con 0x7f2d3406bbd0 2024-03-14T12:13:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.741+0000 7f2d4affd640 1 -- 172.21.15.82:0/3320428768 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251923 (secure 0 0 0) 0x7f2d5415ad20 con 0x7f2d3406bbd0 2024-03-14T12:13:55.744 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:13:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2d3406bbd0 msgr2=0x7f2d3406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2d3406bbd0 0x7f2d3406e090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c036000 tx=0x7f2d3c03c040 comp rx=0 tx=0).stop 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 msgr2=0x7f2d54157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d54157c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c400 tx=0x7f2d4400c8c0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 shutdown_connections 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d5415f7a0 0x7f2d5406c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2d3406bbd0 0x7f2d3406e090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d54144ac0 0x7f2d54143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 --2- 172.21.15.82:0/3320428768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d540cdd60 0x7f2d54157c40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 >> 172.21.15.82:0/3320428768 conn(0x7f2d540b1c70 msgr2=0x7f2d540cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.744+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 shutdown_connections 2024-03-14T12:13:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.745+0000 7f2d58b10640 1 -- 172.21.15.82:0/3320428768 wait complete. 2024-03-14T12:13:55.755 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-14T12:13:55.498766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219060,"kb_used_data":2872,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755660,"statfs":{"total":773094113280,"available":772869795840,"internally_reserved":0,"allocated":2940928,"data_stored":1765196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952078"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.588354+0000","last_change":"2024-03-14T12:13:51.582275+0000","last_active":"2024-03-14T12:13:51.588354+0000","last_peered":"2024-03-14T12:13:51.588354+0000","last_clean":"2024-03-14T12:13:51.588354+0000","last_became_active":"2024-03-14T12:13:51.581999+0000","last_became_peered":"2024-03-14T12:13:51.581999+0000","last_unstale":"2024-03-14T12:13:51.588354+0000","last_undegraded":"2024-03-14T12:13:51.588354+0000","last_fullsized":"2024-03-14T12:13:51.588354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574007+0000","last_change":"2024-03-14T12:13:51.575382+0000","last_active":"2024-03-14T12:13:52.574007+0000","last_peered":"2024-03-14T12:13:52.574007+0000","last_clean":"2024-03-14T12:13:52.574007+0000","last_became_active":"2024-03-14T12:13:51.575139+0000","last_became_peered":"2024-03-14T12:13:51.575139+0000","last_unstale":"2024-03-14T12:13:52.574007+0000","last_undegraded":"2024-03-14T12:13:52.574007+0000","last_fullsized":"2024-03-14T12:13:52.574007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777473+0000","last_change":"2024-03-14T12:13:51.574924+0000","last_active":"2024-03-14T12:13:52.777473+0000","last_peered":"2024-03-14T12:13:52.777473+0000","last_clean":"2024-03-14T12:13:52.777473+0000","last_became_active":"2024-03-14T12:13:51.574299+0000","last_became_peered":"2024-03-14T12:13:51.574299+0000","last_unstale":"2024-03-14T12:13:52.777473+0000","last_undegraded":"2024-03-14T12:13:52.777473+0000","last_fullsized":"2024-03-14T12:13:52.777473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579676+0000","last_change":"2024-03-14T12:13:51.581446+0000","last_active":"2024-03-14T12:13:52.579676+0000","last_peered":"2024-03-14T12:13:52.579676+0000","last_clean":"2024-03-14T12:13:52.579676+0000","last_became_active":"2024-03-14T12:13:51.581192+0000","last_became_peered":"2024-03-14T12:13:51.581192+0000","last_unstale":"2024-03-14T12:13:52.579676+0000","last_undegraded":"2024-03-14T12:13:52.579676+0000","last_fullsized":"2024-03-14T12:13:52.579676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.593758+0000","last_change":"2024-03-14T12:13:51.587466+0000","last_active":"2024-03-14T12:13:51.593758+0000","last_peered":"2024-03-14T12:13:51.593758+0000","last_clean":"2024-03-14T12:13:51.593758+0000","last_became_active":"2024-03-14T12:13:51.585375+0000","last_became_peered":"2024-03-14T12:13:51.585375+0000","last_unstale":"2024-03-14T12:13:51.593758+0000","last_undegraded":"2024-03-14T12:13:51.593758+0000","last_fullsized":"2024-03-14T12:13:51.593758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:21:17.245054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580498+0000","last_change":"2024-03-14T12:13:51.580794+0000","last_active":"2024-03-14T12:13:52.580498+0000","last_peered":"2024-03-14T12:13:52.580498+0000","last_clean":"2024-03-14T12:13:52.580498+0000","last_became_active":"2024-03-14T12:13:51.580511+0000","last_became_peered":"2024-03-14T12:13:51.580511+0000","last_unstale":"2024-03-14T12:13:52.580498+0000","last_undegraded":"2024-03-14T12:13:52.580498+0000","last_fullsized":"2024-03-14T12:13:52.580498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596271+0000","last_change":"2024-03-14T12:13:51.588949+0000","last_active":"2024-03-14T12:13:51.596271+0000","last_peered":"2024-03-14T12:13:51.596271+0000","last_clean":"2024-03-14T12:13:51.596271+0000","last_became_active":"2024-03-14T12:13:51.588671+0000","last_became_peered":"2024-03-14T12:13:51.588671+0000","last_unstale":"2024-03-14T12:13:51.596271+0000","last_undegraded":"2024-03-14T12:13:51.596271+0000","last_fullsized":"2024-03-14T12:13:51.596271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:01.290201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.582077+0000","last_change":"2024-03-14T12:13:51.579815+0000","last_active":"2024-03-14T12:13:51.582077+0000","last_peered":"2024-03-14T12:13:51.582077+0000","last_clean":"2024-03-14T12:13:51.582077+0000","last_became_active":"2024-03-14T12:13:51.579580+0000","last_became_peered":"2024-03-14T12:13:51.579580+0000","last_unstale":"2024-03-14T12:13:51.582077+0000","last_undegraded":"2024-03-14T12:13:51.582077+0000","last_fullsized":"2024-03-14T12:13:51.582077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.598887+0000","last_change":"2024-03-14T12:13:51.593442+0000","last_active":"2024-03-14T12:13:51.598887+0000","last_peered":"2024-03-14T12:13:51.598887+0000","last_clean":"2024-03-14T12:13:51.598887+0000","last_became_active":"2024-03-14T12:13:51.593151+0000","last_became_peered":"2024-03-14T12:13:51.593151+0000","last_unstale":"2024-03-14T12:13:51.598887+0000","last_undegraded":"2024-03-14T12:13:51.598887+0000","last_fullsized":"2024-03-14T12:13:51.598887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582259+0000","last_change":"2024-03-14T12:13:51.584838+0000","last_active":"2024-03-14T12:13:52.582259+0000","last_peered":"2024-03-14T12:13:52.582259+0000","last_clean":"2024-03-14T12:13:52.582259+0000","last_became_active":"2024-03-14T12:13:51.584501+0000","last_became_peered":"2024-03-14T12:13:51.584501+0000","last_unstale":"2024-03-14T12:13:52.582259+0000","last_undegraded":"2024-03-14T12:13:52.582259+0000","last_fullsized":"2024-03-14T12:13:52.582259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581960+0000","last_change":"2024-03-14T12:13:51.584374+0000","last_active":"2024-03-14T12:13:52.581960+0000","last_peered":"2024-03-14T12:13:52.581960+0000","last_clean":"2024-03-14T12:13:52.581960+0000","last_became_active":"2024-03-14T12:13:51.584094+0000","last_became_peered":"2024-03-14T12:13:51.584094+0000","last_unstale":"2024-03-14T12:13:52.581960+0000","last_undegraded":"2024-03-14T12:13:52.581960+0000","last_fullsized":"2024-03-14T12:13:52.581960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580082+0000","last_change":"2024-03-14T12:13:51.583802+0000","last_active":"2024-03-14T12:13:52.580082+0000","last_peered":"2024-03-14T12:13:52.580082+0000","last_clean":"2024-03-14T12:13:52.580082+0000","last_became_active":"2024-03-14T12:13:51.583284+0000","last_became_peered":"2024-03-14T12:13:51.583284+0000","last_unstale":"2024-03-14T12:13:52.580082+0000","last_undegraded":"2024-03-14T12:13:52.580082+0000","last_fullsized":"2024-03-14T12:13:52.580082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.580263+0000","last_change":"2024-03-14T12:13:51.578671+0000","last_active":"2024-03-14T12:13:51.580263+0000","last_peered":"2024-03-14T12:13:51.580263+0000","last_clean":"2024-03-14T12:13:51.580263+0000","last_became_active":"2024-03-14T12:13:51.578427+0000","last_became_peered":"2024-03-14T12:13:51.578427+0000","last_unstale":"2024-03-14T12:13:51.580263+0000","last_undegraded":"2024-03-14T12:13:51.580263+0000","last_fullsized":"2024-03-14T12:13:51.580263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:39.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777400+0000","last_change":"2024-03-14T12:13:51.581289+0000","last_active":"2024-03-14T12:13:52.777400+0000","last_peered":"2024-03-14T12:13:52.777400+0000","last_clean":"2024-03-14T12:13:52.777400+0000","last_became_active":"2024-03-14T12:13:51.580645+0000","last_became_peered":"2024-03-14T12:13:51.580645+0000","last_unstale":"2024-03-14T12:13:52.777400+0000","last_undegraded":"2024-03-14T12:13:52.777400+0000","last_fullsized":"2024-03-14T12:13:52.777400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:01.290201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573796+0000","last_change":"2024-03-14T12:13:51.571032+0000","last_active":"2024-03-14T12:13:52.573796+0000","last_peered":"2024-03-14T12:13:52.573796+0000","last_clean":"2024-03-14T12:13:52.573796+0000","last_became_active":"2024-03-14T12:13:51.570723+0000","last_became_peered":"2024-03-14T12:13:51.570723+0000","last_unstale":"2024-03-14T12:13:52.573796+0000","last_undegraded":"2024-03-14T12:13:52.573796+0000","last_fullsized":"2024-03-14T12:13:52.573796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580831+0000","last_change":"2024-03-14T12:13:51.582495+0000","last_active":"2024-03-14T12:13:52.580831+0000","last_peered":"2024-03-14T12:13:52.580831+0000","last_clean":"2024-03-14T12:13:52.580831+0000","last_became_active":"2024-03-14T12:13:51.582170+0000","last_became_peered":"2024-03-14T12:13:51.582170+0000","last_unstale":"2024-03-14T12:13:52.580831+0000","last_undegraded":"2024-03-14T12:13:52.580831+0000","last_fullsized":"2024-03-14T12:13:52.580831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580861+0000","last_change":"2024-03-14T12:13:51.582330+0000","last_active":"2024-03-14T12:13:52.580861+0000","last_peered":"2024-03-14T12:13:52.580861+0000","last_clean":"2024-03-14T12:13:52.580861+0000","last_became_active":"2024-03-14T12:13:51.581920+0000","last_became_peered":"2024-03-14T12:13:51.581920+0000","last_unstale":"2024-03-14T12:13:52.580861+0000","last_undegraded":"2024-03-14T12:13:52.580861+0000","last_fullsized":"2024-03-14T12:13:52.580861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.583893+0000","last_change":"2024-03-14T12:13:51.582708+0000","last_active":"2024-03-14T12:13:51.583893+0000","last_peered":"2024-03-14T12:13:51.583893+0000","last_clean":"2024-03-14T12:13:51.583893+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:51.583893+0000","last_undegraded":"2024-03-14T12:13:51.583893+0000","last_fullsized":"2024-03-14T12:13:51.583893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.593704+0000","last_change":"2024-03-14T12:13:51.587758+0000","last_active":"2024-03-14T12:13:51.593704+0000","last_peered":"2024-03-14T12:13:51.593704+0000","last_clean":"2024-03-14T12:13:51.593704+0000","last_became_active":"2024-03-14T12:13:51.586310+0000","last_became_peered":"2024-03-14T12:13:51.586310+0000","last_unstale":"2024-03-14T12:13:51.593704+0000","last_undegraded":"2024-03-14T12:13:51.593704+0000","last_fullsized":"2024-03-14T12:13:51.593704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582057+0000","last_change":"2024-03-14T12:13:51.589443+0000","last_active":"2024-03-14T12:13:52.582057+0000","last_peered":"2024-03-14T12:13:52.582057+0000","last_clean":"2024-03-14T12:13:52.582057+0000","last_became_active":"2024-03-14T12:13:51.589174+0000","last_became_peered":"2024-03-14T12:13:51.589174+0000","last_unstale":"2024-03-14T12:13:52.582057+0000","last_undegraded":"2024-03-14T12:13:52.582057+0000","last_fullsized":"2024-03-14T12:13:52.582057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777419+0000","last_change":"2024-03-14T12:13:51.580813+0000","last_active":"2024-03-14T12:13:52.777419+0000","last_peered":"2024-03-14T12:13:52.777419+0000","last_clean":"2024-03-14T12:13:52.777419+0000","last_became_active":"2024-03-14T12:13:51.580334+0000","last_became_peered":"2024-03-14T12:13:51.580334+0000","last_unstale":"2024-03-14T12:13:52.777419+0000","last_undegraded":"2024-03-14T12:13:52.777419+0000","last_fullsized":"2024-03-14T12:13:52.777419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580443+0000","last_change":"2024-03-14T12:13:51.583228+0000","last_active":"2024-03-14T12:13:52.580443+0000","last_peered":"2024-03-14T12:13:52.580443+0000","last_clean":"2024-03-14T12:13:52.580443+0000","last_became_active":"2024-03-14T12:13:51.582937+0000","last_became_peered":"2024-03-14T12:13:51.582937+0000","last_unstale":"2024-03-14T12:13:52.580443+0000","last_undegraded":"2024-03-14T12:13:52.580443+0000","last_fullsized":"2024-03-14T12:13:52.580443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.584037+0000","last_change":"2024-03-14T12:13:51.582656+0000","last_active":"2024-03-14T12:13:51.584037+0000","last_peered":"2024-03-14T12:13:51.584037+0000","last_clean":"2024-03-14T12:13:51.584037+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:51.584037+0000","last_undegraded":"2024-03-14T12:13:51.584037+0000","last_fullsized":"2024-03-14T12:13:51.584037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580802+0000","last_change":"2024-03-14T12:13:51.579810+0000","last_active":"2024-03-14T12:13:52.580802+0000","last_peered":"2024-03-14T12:13:52.580802+0000","last_clean":"2024-03-14T12:13:52.580802+0000","last_became_active":"2024-03-14T12:13:51.579523+0000","last_became_peered":"2024-03-14T12:13:51.579523+0000","last_unstale":"2024-03-14T12:13:52.580802+0000","last_undegraded":"2024-03-14T12:13:52.580802+0000","last_fullsized":"2024-03-14T12:13:52.580802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.584031+0000","last_change":"2024-03-14T12:13:51.582442+0000","last_active":"2024-03-14T12:13:51.584031+0000","last_peered":"2024-03-14T12:13:51.584031+0000","last_clean":"2024-03-14T12:13:51.584031+0000","last_became_active":"2024-03-14T12:13:51.582175+0000","last_became_peered":"2024-03-14T12:13:51.582175+0000","last_unstale":"2024-03-14T12:13:51.584031+0000","last_undegraded":"2024-03-14T12:13:51.584031+0000","last_fullsized":"2024-03-14T12:13:51.584031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:41:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582355+0000","last_change":"2024-03-14T12:13:51.590516+0000","last_active":"2024-03-14T12:13:52.582355+0000","last_peered":"2024-03-14T12:13:52.582355+0000","last_clean":"2024-03-14T12:13:52.582355+0000","last_became_active":"2024-03-14T12:13:51.590223+0000","last_became_peered":"2024-03-14T12:13:51.590223+0000","last_unstale":"2024-03-14T12:13:52.582355+0000","last_undegraded":"2024-03-14T12:13:52.582355+0000","last_fullsized":"2024-03-14T12:13:52.582355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573995+0000","last_change":"2024-03-14T12:13:51.574721+0000","last_active":"2024-03-14T12:13:52.573995+0000","last_peered":"2024-03-14T12:13:52.573995+0000","last_clean":"2024-03-14T12:13:52.573995+0000","last_became_active":"2024-03-14T12:13:51.574443+0000","last_became_peered":"2024-03-14T12:13:51.574443+0000","last_unstale":"2024-03-14T12:13:52.573995+0000","last_undegraded":"2024-03-14T12:13:52.573995+0000","last_fullsized":"2024-03-14T12:13:52.573995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.584024+0000","last_change":"2024-03-14T12:13:51.582640+0000","last_active":"2024-03-14T12:13:51.584024+0000","last_peered":"2024-03-14T12:13:51.584024+0000","last_clean":"2024-03-14T12:13:51.584024+0000","last_became_active":"2024-03-14T12:13:51.582355+0000","last_became_peered":"2024-03-14T12:13:51.582355+0000","last_unstale":"2024-03-14T12:13:51.584024+0000","last_undegraded":"2024-03-14T12:13:51.584024+0000","last_fullsized":"2024-03-14T12:13:51.584024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:21:17.245054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777477+0000","last_change":"2024-03-14T12:13:51.577794+0000","last_active":"2024-03-14T12:13:52.777477+0000","last_peered":"2024-03-14T12:13:52.777477+0000","last_clean":"2024-03-14T12:13:52.777477+0000","last_became_active":"2024-03-14T12:13:51.577518+0000","last_became_peered":"2024-03-14T12:13:51.577518+0000","last_unstale":"2024-03-14T12:13:52.777477+0000","last_undegraded":"2024-03-14T12:13:52.777477+0000","last_fullsized":"2024-03-14T12:13:52.777477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580629+0000","last_change":"2024-03-14T12:13:51.580193+0000","last_active":"2024-03-14T12:13:52.580629+0000","last_peered":"2024-03-14T12:13:52.580629+0000","last_clean":"2024-03-14T12:13:52.580629+0000","last_became_active":"2024-03-14T12:13:51.577667+0000","last_became_peered":"2024-03-14T12:13:51.577667+0000","last_unstale":"2024-03-14T12:13:52.580629+0000","last_undegraded":"2024-03-14T12:13:52.580629+0000","last_fullsized":"2024-03-14T12:13:52.580629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.581932+0000","last_change":"2024-03-14T12:13:51.579889+0000","last_active":"2024-03-14T12:13:51.581932+0000","last_peered":"2024-03-14T12:13:51.581932+0000","last_clean":"2024-03-14T12:13:51.581932+0000","last_became_active":"2024-03-14T12:13:51.579585+0000","last_became_peered":"2024-03-14T12:13:51.579585+0000","last_unstale":"2024-03-14T12:13:51.581932+0000","last_undegraded":"2024-03-14T12:13:51.581932+0000","last_fullsized":"2024-03-14T12:13:51.581932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:55.327329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581817+0000","last_change":"2024-03-14T12:13:51.577340+0000","last_active":"2024-03-14T12:13:52.581817+0000","last_peered":"2024-03-14T12:13:52.581817+0000","last_clean":"2024-03-14T12:13:52.581817+0000","last_became_active":"2024-03-14T12:13:51.577080+0000","last_became_peered":"2024-03-14T12:13:51.577080+0000","last_unstale":"2024-03-14T12:13:52.581817+0000","last_undegraded":"2024-03-14T12:13:52.581817+0000","last_fullsized":"2024-03-14T12:13:52.581817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579599+0000","last_change":"2024-03-14T12:13:51.573447+0000","last_active":"2024-03-14T12:13:52.579599+0000","last_peered":"2024-03-14T12:13:52.579599+0000","last_clean":"2024-03-14T12:13:52.579599+0000","last_became_active":"2024-03-14T12:13:51.573137+0000","last_became_peered":"2024-03-14T12:13:51.573137+0000","last_unstale":"2024-03-14T12:13:52.579599+0000","last_undegraded":"2024-03-14T12:13:52.579599+0000","last_fullsized":"2024-03-14T12:13:52.579599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596461+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:51.596461+0000","last_peered":"2024-03-14T12:13:51.596461+0000","last_clean":"2024-03-14T12:13:51.596461+0000","last_became_active":"2024-03-14T12:13:50.569748+0000","last_became_peered":"2024-03-14T12:13:50.569748+0000","last_unstale":"2024-03-14T12:13:51.596461+0000","last_undegraded":"2024-03-14T12:13:51.596461+0000","last_fullsized":"2024-03-14T12:13:51.596461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777628+0000","last_change":"2024-03-14T12:13:51.581108+0000","last_active":"2024-03-14T12:13:52.777628+0000","last_peered":"2024-03-14T12:13:52.777628+0000","last_clean":"2024-03-14T12:13:52.777628+0000","last_became_active":"2024-03-14T12:13:51.580829+0000","last_became_peered":"2024-03-14T12:13:51.580829+0000","last_unstale":"2024-03-14T12:13:52.777628+0000","last_undegraded":"2024-03-14T12:13:52.777628+0000","last_fullsized":"2024-03-14T12:13:52.777628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581951+0000","last_change":"2024-03-14T12:13:50.565826+0000","last_active":"2024-03-14T12:13:52.581951+0000","last_peered":"2024-03-14T12:13:52.581951+0000","last_clean":"2024-03-14T12:13:52.581951+0000","last_became_active":"2024-03-14T12:13:50.565553+0000","last_became_peered":"2024-03-14T12:13:50.565553+0000","last_unstale":"2024-03-14T12:13:52.581951+0000","last_undegraded":"2024-03-14T12:13:52.581951+0000","last_fullsized":"2024-03-14T12:13:52.581951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.568911+0000","last_change":"2024-03-14T12:13:50.563462+0000","last_active":"2024-03-14T12:13:51.568911+0000","last_peered":"2024-03-14T12:13:51.568911+0000","last_clean":"2024-03-14T12:13:51.568911+0000","last_became_active":"2024-03-14T12:13:50.563198+0000","last_became_peered":"2024-03-14T12:13:50.563198+0000","last_unstale":"2024-03-14T12:13:51.568911+0000","last_undegraded":"2024-03-14T12:13:51.568911+0000","last_fullsized":"2024-03-14T12:13:51.568911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.578083+0000","last_change":"2024-03-14T12:13:51.577103+0000","last_active":"2024-03-14T12:13:51.578083+0000","last_peered":"2024-03-14T12:13:51.578083+0000","last_clean":"2024-03-14T12:13:51.578083+0000","last_became_active":"2024-03-14T12:13:51.576602+0000","last_became_peered":"2024-03-14T12:13:51.576602+0000","last_unstale":"2024-03-14T12:13:51.578083+0000","last_undegraded":"2024-03-14T12:13:51.578083+0000","last_fullsized":"2024-03-14T12:13:51.578083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580881+0000","last_change":"2024-03-14T12:13:50.559389+0000","last_active":"2024-03-14T12:13:52.580881+0000","last_peered":"2024-03-14T12:13:52.580881+0000","last_clean":"2024-03-14T12:13:52.580881+0000","last_became_active":"2024-03-14T12:13:50.559100+0000","last_became_peered":"2024-03-14T12:13:50.559100+0000","last_unstale":"2024-03-14T12:13:52.580881+0000","last_undegraded":"2024-03-14T12:13:52.580881+0000","last_fullsized":"2024-03-14T12:13:52.580881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573924+0000","last_change":"2024-03-14T12:13:51.576591+0000","last_active":"2024-03-14T12:13:52.573924+0000","last_peered":"2024-03-14T12:13:52.573924+0000","last_clean":"2024-03-14T12:13:52.573924+0000","last_became_active":"2024-03-14T12:13:51.576232+0000","last_became_peered":"2024-03-14T12:13:51.576232+0000","last_unstale":"2024-03-14T12:13:52.573924+0000","last_undegraded":"2024-03-14T12:13:52.573924+0000","last_fullsized":"2024-03-14T12:13:52.573924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581963+0000","last_change":"2024-03-14T12:13:50.568392+0000","last_active":"2024-03-14T12:13:52.581963+0000","last_peered":"2024-03-14T12:13:52.581963+0000","last_clean":"2024-03-14T12:13:52.581963+0000","last_became_active":"2024-03-14T12:13:50.568127+0000","last_became_peered":"2024-03-14T12:13:50.568127+0000","last_unstale":"2024-03-14T12:13:52.581963+0000","last_undegraded":"2024-03-14T12:13:52.581963+0000","last_fullsized":"2024-03-14T12:13:52.581963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580126+0000","last_change":"2024-03-14T12:13:51.584429+0000","last_active":"2024-03-14T12:13:52.580126+0000","last_peered":"2024-03-14T12:13:52.580126+0000","last_clean":"2024-03-14T12:13:52.580126+0000","last_became_active":"2024-03-14T12:13:51.584098+0000","last_became_peered":"2024-03-14T12:13:51.584098+0000","last_unstale":"2024-03-14T12:13:52.580126+0000","last_undegraded":"2024-03-14T12:13:52.580126+0000","last_fullsized":"2024-03-14T12:13:52.580126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580241+0000","last_change":"2024-03-14T12:13:50.567672+0000","last_active":"2024-03-14T12:13:52.580241+0000","last_peered":"2024-03-14T12:13:52.580241+0000","last_clean":"2024-03-14T12:13:52.580241+0000","last_became_active":"2024-03-14T12:13:50.567366+0000","last_became_peered":"2024-03-14T12:13:50.567366+0000","last_unstale":"2024-03-14T12:13:52.580241+0000","last_undegraded":"2024-03-14T12:13:52.580241+0000","last_fullsized":"2024-03-14T12:13:52.580241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777722+0000","last_change":"2024-03-14T12:13:51.575322+0000","last_active":"2024-03-14T12:13:52.777722+0000","last_peered":"2024-03-14T12:13:52.777722+0000","last_clean":"2024-03-14T12:13:52.777722+0000","last_became_active":"2024-03-14T12:13:51.574694+0000","last_became_peered":"2024-03-14T12:13:51.574694+0000","last_unstale":"2024-03-14T12:13:52.777722+0000","last_undegraded":"2024-03-14T12:13:52.777722+0000","last_fullsized":"2024-03-14T12:13:52.777722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573967+0000","last_change":"2024-03-14T12:13:50.567057+0000","last_active":"2024-03-14T12:13:52.573967+0000","last_peered":"2024-03-14T12:13:52.573967+0000","last_clean":"2024-03-14T12:13:52.573967+0000","last_became_active":"2024-03-14T12:13:50.566800+0000","last_became_peered":"2024-03-14T12:13:50.566800+0000","last_unstale":"2024-03-14T12:13:52.573967+0000","last_undegraded":"2024-03-14T12:13:52.573967+0000","last_fullsized":"2024-03-14T12:13:52.573967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580236+0000","last_change":"2024-03-14T12:13:51.585716+0000","last_active":"2024-03-14T12:13:52.580236+0000","last_peered":"2024-03-14T12:13:52.580236+0000","last_clean":"2024-03-14T12:13:52.580236+0000","last_became_active":"2024-03-14T12:13:51.585226+0000","last_became_peered":"2024-03-14T12:13:51.585226+0000","last_unstale":"2024-03-14T12:13:52.580236+0000","last_undegraded":"2024-03-14T12:13:52.580236+0000","last_fullsized":"2024-03-14T12:13:52.580236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582041+0000","last_change":"2024-03-14T12:13:50.562666+0000","last_active":"2024-03-14T12:13:52.582041+0000","last_peered":"2024-03-14T12:13:52.582041+0000","last_clean":"2024-03-14T12:13:52.582041+0000","last_became_active":"2024-03-14T12:13:50.562367+0000","last_became_peered":"2024-03-14T12:13:50.562367+0000","last_unstale":"2024-03-14T12:13:52.582041+0000","last_undegraded":"2024-03-14T12:13:52.582041+0000","last_fullsized":"2024-03-14T12:13:52.582041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.593676+0000","last_change":"2024-03-14T12:13:51.587183+0000","last_active":"2024-03-14T12:13:51.593676+0000","last_peered":"2024-03-14T12:13:51.593676+0000","last_clean":"2024-03-14T12:13:51.593676+0000","last_became_active":"2024-03-14T12:13:51.585685+0000","last_became_peered":"2024-03-14T12:13:51.585685+0000","last_unstale":"2024-03-14T12:13:51.593676+0000","last_undegraded":"2024-03-14T12:13:51.593676+0000","last_fullsized":"2024-03-14T12:13:51.593676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:41:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777610+0000","last_change":"2024-03-14T12:13:50.557053+0000","last_active":"2024-03-14T12:13:52.777610+0000","last_peered":"2024-03-14T12:13:52.777610+0000","last_clean":"2024-03-14T12:13:52.777610+0000","last_became_active":"2024-03-14T12:13:50.556737+0000","last_became_peered":"2024-03-14T12:13:50.556737+0000","last_unstale":"2024-03-14T12:13:52.777610+0000","last_undegraded":"2024-03-14T12:13:52.777610+0000","last_fullsized":"2024-03-14T12:13:52.777610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777653+0000","last_change":"2024-03-14T12:13:51.581665+0000","last_active":"2024-03-14T12:13:52.777653+0000","last_peered":"2024-03-14T12:13:52.777653+0000","last_clean":"2024-03-14T12:13:52.777653+0000","last_became_active":"2024-03-14T12:13:51.580980+0000","last_became_peered":"2024-03-14T12:13:51.580980+0000","last_unstale":"2024-03-14T12:13:52.777653+0000","last_undegraded":"2024-03-14T12:13:52.777653+0000","last_fullsized":"2024-03-14T12:13:52.777653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580677+0000","last_change":"2024-03-14T12:13:50.566403+0000","last_active":"2024-03-14T12:13:52.580677+0000","last_peered":"2024-03-14T12:13:52.580677+0000","last_clean":"2024-03-14T12:13:52.580677+0000","last_became_active":"2024-03-14T12:13:50.565782+0000","last_became_peered":"2024-03-14T12:13:50.565782+0000","last_unstale":"2024-03-14T12:13:52.580677+0000","last_undegraded":"2024-03-14T12:13:52.580677+0000","last_fullsized":"2024-03-14T12:13:52.580677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596562+0000","last_change":"2024-03-14T12:13:51.592512+0000","last_active":"2024-03-14T12:13:51.596562+0000","last_peered":"2024-03-14T12:13:51.596562+0000","last_clean":"2024-03-14T12:13:51.596562+0000","last_became_active":"2024-03-14T12:13:51.592273+0000","last_became_peered":"2024-03-14T12:13:51.592273+0000","last_unstale":"2024-03-14T12:13:51.596562+0000","last_undegraded":"2024-03-14T12:13:51.596562+0000","last_fullsized":"2024-03-14T12:13:51.596562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581835+0000","last_change":"2024-03-14T12:13:50.564239+0000","last_active":"2024-03-14T12:13:52.581835+0000","last_peered":"2024-03-14T12:13:52.581835+0000","last_clean":"2024-03-14T12:13:52.581835+0000","last_became_active":"2024-03-14T12:13:50.563920+0000","last_became_peered":"2024-03-14T12:13:50.563920+0000","last_unstale":"2024-03-14T12:13:52.581835+0000","last_undegraded":"2024-03-14T12:13:52.581835+0000","last_fullsized":"2024-03-14T12:13:52.581835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581784+0000","last_change":"2024-03-14T12:13:51.576682+0000","last_active":"2024-03-14T12:13:52.581784+0000","last_peered":"2024-03-14T12:13:52.581784+0000","last_clean":"2024-03-14T12:13:52.581784+0000","last_became_active":"2024-03-14T12:13:51.576420+0000","last_became_peered":"2024-03-14T12:13:51.576420+0000","last_unstale":"2024-03-14T12:13:52.581784+0000","last_undegraded":"2024-03-14T12:13:52.581784+0000","last_fullsized":"2024-03-14T12:13:52.581784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.592862+0000","last_change":"2024-03-14T12:13:50.564352+0000","last_active":"2024-03-14T12:13:51.592862+0000","last_peered":"2024-03-14T12:13:51.592862+0000","last_clean":"2024-03-14T12:13:51.592862+0000","last_became_active":"2024-03-14T12:13:50.563979+0000","last_became_peered":"2024-03-14T12:13:50.563979+0000","last_unstale":"2024-03-14T12:13:51.592862+0000","last_undegraded":"2024-03-14T12:13:51.592862+0000","last_fullsized":"2024-03-14T12:13:51.592862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596570+0000","last_change":"2024-03-14T12:13:51.592382+0000","last_active":"2024-03-14T12:13:51.596570+0000","last_peered":"2024-03-14T12:13:51.596570+0000","last_clean":"2024-03-14T12:13:51.596570+0000","last_became_active":"2024-03-14T12:13:51.592109+0000","last_became_peered":"2024-03-14T12:13:51.592109+0000","last_unstale":"2024-03-14T12:13:51.596570+0000","last_undegraded":"2024-03-14T12:13:51.596570+0000","last_fullsized":"2024-03-14T12:13:51.596570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579929+0000","last_change":"2024-03-14T12:13:50.565493+0000","last_active":"2024-03-14T12:13:52.579929+0000","last_peered":"2024-03-14T12:13:52.579929+0000","last_clean":"2024-03-14T12:13:52.579929+0000","last_became_active":"2024-03-14T12:13:50.565247+0000","last_became_peered":"2024-03-14T12:13:50.565247+0000","last_unstale":"2024-03-14T12:13:52.579929+0000","last_undegraded":"2024-03-14T12:13:52.579929+0000","last_fullsized":"2024-03-14T12:13:52.579929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:39:10.529585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596442+0000","last_change":"2024-03-14T12:13:51.591007+0000","last_active":"2024-03-14T12:13:51.596442+0000","last_peered":"2024-03-14T12:13:51.596442+0000","last_clean":"2024-03-14T12:13:51.596442+0000","last_became_active":"2024-03-14T12:13:51.590750+0000","last_became_peered":"2024-03-14T12:13:51.590750+0000","last_unstale":"2024-03-14T12:13:51.596442+0000","last_undegraded":"2024-03-14T12:13:51.596442+0000","last_fullsized":"2024-03-14T12:13:51.596442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.589972+0000","last_change":"2024-03-14T12:13:50.568050+0000","last_active":"2024-03-14T12:13:51.589972+0000","last_peered":"2024-03-14T12:13:51.589972+0000","last_clean":"2024-03-14T12:13:51.589972+0000","last_became_active":"2024-03-14T12:13:50.567713+0000","last_became_peered":"2024-03-14T12:13:50.567713+0000","last_unstale":"2024-03-14T12:13:51.589972+0000","last_undegraded":"2024-03-14T12:13:51.589972+0000","last_fullsized":"2024-03-14T12:13:51.589972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596589+0000","last_change":"2024-03-14T12:13:51.592562+0000","last_active":"2024-03-14T12:13:51.596589+0000","last_peered":"2024-03-14T12:13:51.596589+0000","last_clean":"2024-03-14T12:13:51.596589+0000","last_became_active":"2024-03-14T12:13:51.592255+0000","last_became_peered":"2024-03-14T12:13:51.592255+0000","last_unstale":"2024-03-14T12:13:51.596589+0000","last_undegraded":"2024-03-14T12:13:51.596589+0000","last_fullsized":"2024-03-14T12:13:51.596589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574041+0000","last_change":"2024-03-14T12:13:50.566339+0000","last_active":"2024-03-14T12:13:52.574041+0000","last_peered":"2024-03-14T12:13:52.574041+0000","last_clean":"2024-03-14T12:13:52.574041+0000","last_became_active":"2024-03-14T12:13:50.566058+0000","last_became_peered":"2024-03-14T12:13:50.566058+0000","last_unstale":"2024-03-14T12:13:52.574041+0000","last_undegraded":"2024-03-14T12:13:52.574041+0000","last_fullsized":"2024-03-14T12:13:52.574041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579993+0000","last_change":"2024-03-14T12:13:51.581453+0000","last_active":"2024-03-14T12:13:52.579993+0000","last_peered":"2024-03-14T12:13:52.579993+0000","last_clean":"2024-03-14T12:13:52.579993+0000","last_became_active":"2024-03-14T12:13:51.581168+0000","last_became_peered":"2024-03-14T12:13:51.581168+0000","last_unstale":"2024-03-14T12:13:52.579993+0000","last_undegraded":"2024-03-14T12:13:52.579993+0000","last_fullsized":"2024-03-14T12:13:52.579993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.570338+0000","last_change":"2024-03-14T12:13:50.568172+0000","last_active":"2024-03-14T12:13:51.570338+0000","last_peered":"2024-03-14T12:13:51.570338+0000","last_clean":"2024-03-14T12:13:51.570338+0000","last_became_active":"2024-03-14T12:13:50.567825+0000","last_became_peered":"2024-03-14T12:13:50.567825+0000","last_unstale":"2024-03-14T12:13:51.570338+0000","last_undegraded":"2024-03-14T12:13:51.570338+0000","last_fullsized":"2024-03-14T12:13:51.570338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.598902+0000","last_change":"2024-03-14T12:13:51.593995+0000","last_active":"2024-03-14T12:13:51.598902+0000","last_peered":"2024-03-14T12:13:51.598902+0000","last_clean":"2024-03-14T12:13:51.598902+0000","last_became_active":"2024-03-14T12:13:51.593473+0000","last_became_peered":"2024-03-14T12:13:51.593473+0000","last_unstale":"2024-03-14T12:13:51.598902+0000","last_undegraded":"2024-03-14T12:13:51.598902+0000","last_fullsized":"2024-03-14T12:13:51.598902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.575200+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:13:51.575200+0000","last_peered":"2024-03-14T12:13:51.575200+0000","last_clean":"2024-03-14T12:13:51.575200+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:13:51.575200+0000","last_undegraded":"2024-03-14T12:13:51.575200+0000","last_fullsized":"2024-03-14T12:13:51.575200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580336+0000","last_change":"2024-03-14T12:13:50.565229+0000","last_active":"2024-03-14T12:13:52.580336+0000","last_peered":"2024-03-14T12:13:52.580336+0000","last_clean":"2024-03-14T12:13:52.580336+0000","last_became_active":"2024-03-14T12:13:50.564952+0000","last_became_peered":"2024-03-14T12:13:50.564952+0000","last_unstale":"2024-03-14T12:13:52.580336+0000","last_undegraded":"2024-03-14T12:13:52.580336+0000","last_fullsized":"2024-03-14T12:13:52.580336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.576826+0000","last_change":"2024-03-14T12:13:51.573189+0000","last_active":"2024-03-14T12:13:51.576826+0000","last_peered":"2024-03-14T12:13:51.576826+0000","last_clean":"2024-03-14T12:13:51.576826+0000","last_became_active":"2024-03-14T12:13:51.572900+0000","last_became_peered":"2024-03-14T12:13:51.572900+0000","last_unstale":"2024-03-14T12:13:51.576826+0000","last_undegraded":"2024-03-14T12:13:51.576826+0000","last_fullsized":"2024-03-14T12:13:51.576826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:31.510461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.575144+0000","last_change":"2024-03-14T12:13:50.571689+0000","last_active":"2024-03-14T12:13:51.575144+0000","last_peered":"2024-03-14T12:13:51.575144+0000","last_clean":"2024-03-14T12:13:51.575144+0000","last_became_active":"2024-03-14T12:13:50.571292+0000","last_became_peered":"2024-03-14T12:13:50.571292+0000","last_unstale":"2024-03-14T12:13:51.575144+0000","last_undegraded":"2024-03-14T12:13:51.575144+0000","last_fullsized":"2024-03-14T12:13:51.575144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777395+0000","last_change":"2024-03-14T12:13:51.582845+0000","last_active":"2024-03-14T12:13:52.777395+0000","last_peered":"2024-03-14T12:13:52.777395+0000","last_clean":"2024-03-14T12:13:52.777395+0000","last_became_active":"2024-03-14T12:13:51.582580+0000","last_became_peered":"2024-03-14T12:13:51.582580+0000","last_unstale":"2024-03-14T12:13:52.777395+0000","last_undegraded":"2024-03-14T12:13:52.777395+0000","last_fullsized":"2024-03-14T12:13:52.777395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.570680+0000","last_change":"2024-03-14T12:13:50.568041+0000","last_active":"2024-03-14T12:13:51.570680+0000","last_peered":"2024-03-14T12:13:51.570680+0000","last_clean":"2024-03-14T12:13:51.570680+0000","last_became_active":"2024-03-14T12:13:50.567698+0000","last_became_peered":"2024-03-14T12:13:50.567698+0000","last_unstale":"2024-03-14T12:13:51.570680+0000","last_undegraded":"2024-03-14T12:13:51.570680+0000","last_fullsized":"2024-03-14T12:13:51.570680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573917+0000","last_change":"2024-03-14T12:13:51.572247+0000","last_active":"2024-03-14T12:13:52.573917+0000","last_peered":"2024-03-14T12:13:52.573917+0000","last_clean":"2024-03-14T12:13:52.573917+0000","last_became_active":"2024-03-14T12:13:51.572000+0000","last_became_peered":"2024-03-14T12:13:51.572000+0000","last_unstale":"2024-03-14T12:13:52.573917+0000","last_undegraded":"2024-03-14T12:13:52.573917+0000","last_fullsized":"2024-03-14T12:13:52.573917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580774+0000","last_change":"2024-03-14T12:13:50.562695+0000","last_active":"2024-03-14T12:13:52.580774+0000","last_peered":"2024-03-14T12:13:52.580774+0000","last_clean":"2024-03-14T12:13:52.580774+0000","last_became_active":"2024-03-14T12:13:50.562419+0000","last_became_peered":"2024-03-14T12:13:50.562419+0000","last_unstale":"2024-03-14T12:13:52.580774+0000","last_undegraded":"2024-03-14T12:13:52.580774+0000","last_fullsized":"2024-03-14T12:13:52.580774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580029+0000","last_change":"2024-03-14T12:13:51.578692+0000","last_active":"2024-03-14T12:13:52.580029+0000","last_peered":"2024-03-14T12:13:52.580029+0000","last_clean":"2024-03-14T12:13:52.580029+0000","last_became_active":"2024-03-14T12:13:51.578375+0000","last_became_peered":"2024-03-14T12:13:51.578375+0000","last_unstale":"2024-03-14T12:13:52.580029+0000","last_undegraded":"2024-03-14T12:13:52.580029+0000","last_fullsized":"2024-03-14T12:13:52.580029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573848+0000","last_change":"2024-03-14T12:13:50.568583+0000","last_active":"2024-03-14T12:13:52.573848+0000","last_peered":"2024-03-14T12:13:52.573848+0000","last_clean":"2024-03-14T12:13:52.573848+0000","last_became_active":"2024-03-14T12:13:50.568267+0000","last_became_peered":"2024-03-14T12:13:50.568267+0000","last_unstale":"2024-03-14T12:13:52.573848+0000","last_undegraded":"2024-03-14T12:13:52.573848+0000","last_fullsized":"2024-03-14T12:13:52.573848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580655+0000","last_change":"2024-03-14T12:13:51.579205+0000","last_active":"2024-03-14T12:13:52.580655+0000","last_peered":"2024-03-14T12:13:52.580655+0000","last_clean":"2024-03-14T12:13:52.580655+0000","last_became_active":"2024-03-14T12:13:51.578901+0000","last_became_peered":"2024-03-14T12:13:51.578901+0000","last_unstale":"2024-03-14T12:13:52.580655+0000","last_undegraded":"2024-03-14T12:13:52.580655+0000","last_fullsized":"2024-03-14T12:13:52.580655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.577780+0000","last_change":"2024-03-14T12:13:50.571020+0000","last_active":"2024-03-14T12:13:51.577780+0000","last_peered":"2024-03-14T12:13:51.577780+0000","last_clean":"2024-03-14T12:13:51.577780+0000","last_became_active":"2024-03-14T12:13:50.570744+0000","last_became_peered":"2024-03-14T12:13:50.570744+0000","last_unstale":"2024-03-14T12:13:51.577780+0000","last_undegraded":"2024-03-14T12:13:51.577780+0000","last_fullsized":"2024-03-14T12:13:51.577780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573864+0000","last_change":"2024-03-14T12:13:51.574187+0000","last_active":"2024-03-14T12:13:52.573864+0000","last_peered":"2024-03-14T12:13:52.573864+0000","last_clean":"2024-03-14T12:13:52.573864+0000","last_became_active":"2024-03-14T12:13:51.573916+0000","last_became_peered":"2024-03-14T12:13:51.573916+0000","last_unstale":"2024-03-14T12:13:52.573864+0000","last_undegraded":"2024-03-14T12:13:52.573864+0000","last_fullsized":"2024-03-14T12:13:52.573864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580829+0000","last_change":"2024-03-14T12:13:50.569678+0000","last_active":"2024-03-14T12:13:52.580829+0000","last_peered":"2024-03-14T12:13:52.580829+0000","last_clean":"2024-03-14T12:13:52.580829+0000","last_became_active":"2024-03-14T12:13:50.569377+0000","last_became_peered":"2024-03-14T12:13:50.569377+0000","last_unstale":"2024-03-14T12:13:52.580829+0000","last_undegraded":"2024-03-14T12:13:52.580829+0000","last_fullsized":"2024-03-14T12:13:52.580829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.582019+0000","last_change":"2024-03-14T12:13:51.579167+0000","last_active":"2024-03-14T12:13:51.582019+0000","last_peered":"2024-03-14T12:13:51.582019+0000","last_clean":"2024-03-14T12:13:51.582019+0000","last_became_active":"2024-03-14T12:13:51.578971+0000","last_became_peered":"2024-03-14T12:13:51.578971+0000","last_unstale":"2024-03-14T12:13:51.582019+0000","last_undegraded":"2024-03-14T12:13:51.582019+0000","last_fullsized":"2024-03-14T12:13:51.582019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582080+0000","last_change":"2024-03-14T12:13:50.570044+0000","last_active":"2024-03-14T12:13:52.582080+0000","last_peered":"2024-03-14T12:13:52.582080+0000","last_clean":"2024-03-14T12:13:52.582080+0000","last_became_active":"2024-03-14T12:13:50.569759+0000","last_became_peered":"2024-03-14T12:13:50.569759+0000","last_unstale":"2024-03-14T12:13:52.582080+0000","last_undegraded":"2024-03-14T12:13:52.582080+0000","last_fullsized":"2024-03-14T12:13:52.582080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777595+0000","last_change":"2024-03-14T12:13:51.581169+0000","last_active":"2024-03-14T12:13:52.777595+0000","last_peered":"2024-03-14T12:13:52.777595+0000","last_clean":"2024-03-14T12:13:52.777595+0000","last_became_active":"2024-03-14T12:13:51.580757+0000","last_became_peered":"2024-03-14T12:13:51.580757+0000","last_unstale":"2024-03-14T12:13:52.777595+0000","last_undegraded":"2024-03-14T12:13:52.777595+0000","last_fullsized":"2024-03-14T12:13:52.777595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.574249+0000","last_change":"2024-03-14T12:13:50.571439+0000","last_active":"2024-03-14T12:13:51.574249+0000","last_peered":"2024-03-14T12:13:51.574249+0000","last_clean":"2024-03-14T12:13:51.574249+0000","last_became_active":"2024-03-14T12:13:50.571184+0000","last_became_peered":"2024-03-14T12:13:50.571184+0000","last_unstale":"2024-03-14T12:13:51.574249+0000","last_undegraded":"2024-03-14T12:13:51.574249+0000","last_fullsized":"2024-03-14T12:13:51.574249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579859+0000","last_change":"2024-03-14T12:13:51.573261+0000","last_active":"2024-03-14T12:13:52.579859+0000","last_peered":"2024-03-14T12:13:52.579859+0000","last_clean":"2024-03-14T12:13:52.579859+0000","last_became_active":"2024-03-14T12:13:51.572866+0000","last_became_peered":"2024-03-14T12:13:51.572866+0000","last_unstale":"2024-03-14T12:13:52.579859+0000","last_undegraded":"2024-03-14T12:13:52.579859+0000","last_fullsized":"2024-03-14T12:13:52.579859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:39.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573916+0000","last_change":"2024-03-14T12:13:50.570820+0000","last_active":"2024-03-14T12:13:52.573916+0000","last_peered":"2024-03-14T12:13:52.573916+0000","last_clean":"2024-03-14T12:13:52.573916+0000","last_became_active":"2024-03-14T12:13:50.570423+0000","last_became_peered":"2024-03-14T12:13:50.570423+0000","last_unstale":"2024-03-14T12:13:52.573916+0000","last_undegraded":"2024-03-14T12:13:52.573916+0000","last_fullsized":"2024-03-14T12:13:52.573916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.576872+0000","last_change":"2024-03-14T12:13:51.575973+0000","last_active":"2024-03-14T12:13:51.576872+0000","last_peered":"2024-03-14T12:13:51.576872+0000","last_clean":"2024-03-14T12:13:51.576872+0000","last_became_active":"2024-03-14T12:13:51.575695+0000","last_became_peered":"2024-03-14T12:13:51.575695+0000","last_unstale":"2024-03-14T12:13:51.576872+0000","last_undegraded":"2024-03-14T12:13:51.576872+0000","last_fullsized":"2024-03-14T12:13:51.576872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:22.285709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573877+0000","last_change":"2024-03-14T12:13:50.565398+0000","last_active":"2024-03-14T12:13:52.573877+0000","last_peered":"2024-03-14T12:13:52.573877+0000","last_clean":"2024-03-14T12:13:52.573877+0000","last_became_active":"2024-03-14T12:13:50.565109+0000","last_became_peered":"2024-03-14T12:13:50.565109+0000","last_unstale":"2024-03-14T12:13:52.573877+0000","last_undegraded":"2024-03-14T12:13:52.573877+0000","last_fullsized":"2024-03-14T12:13:52.573877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.578089+0000","last_change":"2024-03-14T12:13:51.576916+0000","last_active":"2024-03-14T12:13:51.578089+0000","last_peered":"2024-03-14T12:13:51.578089+0000","last_clean":"2024-03-14T12:13:51.578089+0000","last_became_active":"2024-03-14T12:13:51.576639+0000","last_became_peered":"2024-03-14T12:13:51.576639+0000","last_unstale":"2024-03-14T12:13:51.578089+0000","last_undegraded":"2024-03-14T12:13:51.578089+0000","last_fullsized":"2024-03-14T12:13:51.578089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:55.327329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596200+0000","last_change":"2024-03-14T12:13:50.575585+0000","last_active":"2024-03-14T12:13:51.596200+0000","last_peered":"2024-03-14T12:13:51.596200+0000","last_clean":"2024-03-14T12:13:51.596200+0000","last_became_active":"2024-03-14T12:13:50.575288+0000","last_became_peered":"2024-03-14T12:13:50.575288+0000","last_unstale":"2024-03-14T12:13:51.596200+0000","last_undegraded":"2024-03-14T12:13:51.596200+0000","last_fullsized":"2024-03-14T12:13:51.596200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581882+0000","last_change":"2024-03-14T12:13:51.588022+0000","last_active":"2024-03-14T12:13:52.581882+0000","last_peered":"2024-03-14T12:13:52.581882+0000","last_clean":"2024-03-14T12:13:52.581882+0000","last_became_active":"2024-03-14T12:13:51.587766+0000","last_became_peered":"2024-03-14T12:13:51.587766+0000","last_unstale":"2024-03-14T12:13:52.581882+0000","last_undegraded":"2024-03-14T12:13:52.581882+0000","last_fullsized":"2024-03-14T12:13:52.581882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580894+0000","last_change":"2024-03-14T12:13:51.572504+0000","last_active":"2024-03-14T12:13:52.580894+0000","last_peered":"2024-03-14T12:13:52.580894+0000","last_clean":"2024-03-14T12:13:52.580894+0000","last_became_active":"2024-03-14T12:13:51.572219+0000","last_became_peered":"2024-03-14T12:13:51.572219+0000","last_unstale":"2024-03-14T12:13:52.580894+0000","last_undegraded":"2024-03-14T12:13:52.580894+0000","last_fullsized":"2024-03-14T12:13:52.580894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.595216+0000","last_change":"2024-03-14T12:13:50.572648+0000","last_active":"2024-03-14T12:13:51.595216+0000","last_peered":"2024-03-14T12:13:51.595216+0000","last_clean":"2024-03-14T12:13:51.595216+0000","last_became_active":"2024-03-14T12:13:50.572409+0000","last_became_peered":"2024-03-14T12:13:50.572409+0000","last_unstale":"2024-03-14T12:13:51.595216+0000","last_undegraded":"2024-03-14T12:13:51.595216+0000","last_fullsized":"2024-03-14T12:13:51.595216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579831+0000","last_change":"2024-03-14T12:13:51.580823+0000","last_active":"2024-03-14T12:13:52.579831+0000","last_peered":"2024-03-14T12:13:52.579831+0000","last_clean":"2024-03-14T12:13:52.579831+0000","last_became_active":"2024-03-14T12:13:51.574625+0000","last_became_peered":"2024-03-14T12:13:51.574625+0000","last_unstale":"2024-03-14T12:13:52.579831+0000","last_undegraded":"2024-03-14T12:13:52.579831+0000","last_fullsized":"2024-03-14T12:13:52.579831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596356+0000","last_change":"2024-03-14T12:13:50.572321+0000","last_active":"2024-03-14T12:13:51.596356+0000","last_peered":"2024-03-14T12:13:51.596356+0000","last_clean":"2024-03-14T12:13:51.596356+0000","last_became_active":"2024-03-14T12:13:50.572001+0000","last_became_peered":"2024-03-14T12:13:50.572001+0000","last_unstale":"2024-03-14T12:13:51.596356+0000","last_undegraded":"2024-03-14T12:13:51.596356+0000","last_fullsized":"2024-03-14T12:13:51.596356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777630+0000","last_change":"2024-03-14T12:13:51.575693+0000","last_active":"2024-03-14T12:13:52.777630+0000","last_peered":"2024-03-14T12:13:52.777630+0000","last_clean":"2024-03-14T12:13:52.777630+0000","last_became_active":"2024-03-14T12:13:51.574493+0000","last_became_peered":"2024-03-14T12:13:51.574493+0000","last_unstale":"2024-03-14T12:13:52.777630+0000","last_undegraded":"2024-03-14T12:13:52.777630+0000","last_fullsized":"2024-03-14T12:13:52.777630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.592745+0000","last_change":"2024-03-14T12:13:50.565913+0000","last_active":"2024-03-14T12:13:51.592745+0000","last_peered":"2024-03-14T12:13:51.592745+0000","last_clean":"2024-03-14T12:13:51.592745+0000","last_became_active":"2024-03-14T12:13:50.565652+0000","last_became_peered":"2024-03-14T12:13:50.565652+0000","last_unstale":"2024-03-14T12:13:51.592745+0000","last_undegraded":"2024-03-14T12:13:51.592745+0000","last_fullsized":"2024-03-14T12:13:51.592745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.582658+0000","last_change":"2024-03-14T12:13:51.581638+0000","last_active":"2024-03-14T12:13:51.582658+0000","last_peered":"2024-03-14T12:13:51.582658+0000","last_clean":"2024-03-14T12:13:51.582658+0000","last_became_active":"2024-03-14T12:13:51.581373+0000","last_became_peered":"2024-03-14T12:13:51.581373+0000","last_unstale":"2024-03-14T12:13:51.582658+0000","last_undegraded":"2024-03-14T12:13:51.582658+0000","last_fullsized":"2024-03-14T12:13:51.582658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777698+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:52.777698+0000","last_peered":"2024-03-14T12:13:52.777698+0000","last_clean":"2024-03-14T12:13:52.777698+0000","last_became_active":"2024-03-14T12:13:50.569816+0000","last_became_peered":"2024-03-14T12:13:50.569816+0000","last_unstale":"2024-03-14T12:13:52.777698+0000","last_undegraded":"2024-03-14T12:13:52.777698+0000","last_fullsized":"2024-03-14T12:13:52.777698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:13:55.760 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-14T12:13:55.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b49953640 1 Processor -- start 2024-03-14T12:13:55.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b49953640 1 -- start start 2024-03-14T12:13:55.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b49953640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b44161770 0x7f9b440d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b49953640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b440b1eb0 0x7f9b440d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b49953640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440cf630 con 0x7f9b440b3270 2024-03-14T12:13:55.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b49953640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b440d2ea0 con 0x7f9b44161770 2024-03-14T12:13:55.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b49953640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440cfed0 con 0x7f9b440b1eb0 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b42ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b44161770 0x7f9b440d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b42ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b44161770 0x7f9b440d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48562/0 (socket says 172.21.15.82:48562) 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b42ffd640 1 -- 172.21.15.82:0/2928178927 learned_addr learned my addr 172.21.15.82:0/2928178927 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.895+0000 7f9b427fc640 1 --1- 172.21.15.82:0/2928178927 >> v1:172.21.15.82:6789/0 conn(0x7f9b440b3270 0x7f9b440d1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34516/0 (socket says 172.21.15.82:34516) 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.896+0000 7f9b437fe640 1 --2- 172.21.15.82:0/2928178927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b440b1eb0 0x7f9b440d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.896+0000 7f9b42ffd640 1 -- 172.21.15.82:0/2928178927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b440b1eb0 msgr2=0x7f9b440d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.896+0000 7f9b42ffd640 1 --2- 172.21.15.82:0/2928178927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b440b1eb0 0x7f9b440d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.896+0000 7f9b42ffd640 1 -- 172.21.15.82:0/2928178927 >> v1:172.21.15.82:6789/0 conn(0x7f9b440b3270 legacy=0x7f9b440d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.896+0000 7f9b42ffd640 1 -- 172.21.15.82:0/2928178927 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440cf910 con 0x7f9b44161770 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.896+0000 7f9b42ffd640 1 --2- 172.21.15.82:0/2928178927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44161770 0x7f9b440d07d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009e50 tx=0x7f9b3402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306c08d8c7c24eb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b41ffb640 1 -- 172.21.15.82:0/2928178927 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3402cb00 con 0x7f9b44161770 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b41ffb640 1 -- 172.21.15.82:0/2928178927 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3402cca0 con 0x7f9b44161770 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b41ffb640 1 -- 172.21.15.82:0/2928178927 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34034610 con 0x7f9b44161770 2024-03-14T12:13:55.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 -- 172.21.15.82:0/2928178927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44161770 msgr2=0x7f9b440d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 --2- 172.21.15.82:0/2928178927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44161770 0x7f9b440d07d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009e50 tx=0x7f9b3402bf60 comp rx=0 tx=0).stop 2024-03-14T12:13:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 -- 172.21.15.82:0/2928178927 shutdown_connections 2024-03-14T12:13:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 --2- 172.21.15.82:0/2928178927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b440b1eb0 0x7f9b440d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 --2- 172.21.15.82:0/2928178927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b44161770 0x7f9b440d07d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 -- 172.21.15.82:0/2928178927 >> 172.21.15.82:0/2928178927 conn(0x7f9b4409d1d0 msgr2=0x7f9b440d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 -- 172.21.15.82:0/2928178927 shutdown_connections 2024-03-14T12:13:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.897+0000 7f9b49953640 1 -- 172.21.15.82:0/2928178927 wait complete. 2024-03-14T12:13:55.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.898+0000 7f9b49953640 1 Processor -- start 2024-03-14T12:13:55.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b49953640 1 -- start start 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b49953640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b440b1eb0 0x7f9b440b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b49953640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b440b3270 0x7f9b441501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b49953640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b44161770 0x7f9b440ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b49953640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440b2e10 con 0x7f9b440b3270 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b49953640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440b2c90 con 0x7f9b440b1eb0 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b49953640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440b2f90 con 0x7f9b44161770 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b427fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b440b3270 0x7f9b441501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b427fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b440b3270 0x7f9b441501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48564/0 (socket says 172.21.15.82:48564) 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.899+0000 7f9b427fc640 1 -- 172.21.15.82:0/529272796 learned_addr learned my addr 172.21.15.82:0/529272796 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b427fc640 1 -- 172.21.15.82:0/529272796 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b44161770 msgr2=0x7f9b440ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b427fc640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b44161770 0x7f9b440ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b427fc640 1 -- 172.21.15.82:0/529272796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b440b1eb0 msgr2=0x7f9b440b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b427fc640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b440b1eb0 0x7f9b440b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b427fc640 1 -- 172.21.15.82:0/529272796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440b6e70 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b427fc640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b440b3270 0x7f9b441501a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c3e0 tx=0x7f9b2c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b3bfff640 1 -- 172.21.15.82:0/529272796 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0074e0 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b3bfff640 1 -- 172.21.15.82:0/529272796 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c007680 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b440b7150 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b3bfff640 1 -- 172.21.15.82:0/529272796 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0049e0 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.900+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b440b7710 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.901+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4404f050 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.901+0000 7f9b3bfff640 1 -- 172.21.15.82:0/529272796 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9b2c01a020 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.901+0000 7f9b3bfff640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b2406bbd0 0x7f9b2406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.902+0000 7f9b3bfff640 1 -- 172.21.15.82:0/529272796 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b2c08a300 con 0x7f9b440b3270 2024-03-14T12:13:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.902+0000 7f9b42ffd640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b2406bbd0 0x7f9b2406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:55.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.902+0000 7f9b42ffd640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b2406bbd0 0x7f9b2406e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b34035000 tx=0x7f9b3402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:55.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:55.906+0000 7f9b3bfff640 1 -- 172.21.15.82:0/529272796 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9b2c010300 con 0x7f9b440b3270 2024-03-14T12:13:56.077 INFO:tasks.ceph.mds.b.smithi164.stderr:2024-03-14T12:13:56.078+0000 7fe469e74640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:13:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.144+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9b4406c240 con 0x7f9b440b3270 2024-03-14T12:13:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.145+0000 7f9b3bfff640 1 -- 172.21.15.82:0/529272796 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1011 (secure 0 0 0) 0x7f9b2c058d60 con 0x7f9b440b3270 2024-03-14T12:13:56.146 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:56.147 INFO:teuthology.orchestra.run.smithi082.stdout:{"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1710418418,"num_in_osds":8,"osd_in_since":1710418409,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224317440,"bytes_avail":772869795840,"bytes_total":773094113280,"write_bytes_sec":2408,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4306}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-03-14T12:13:27.375235+0000","services":{}},"progress_events":{}} 2024-03-14T12:13:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b2406bbd0 msgr2=0x7f9b2406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b2406bbd0 0x7f9b2406e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b34035000 tx=0x7f9b3402e000 comp rx=0 tx=0).stop 2024-03-14T12:13:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b440b3270 msgr2=0x7f9b441501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b440b3270 0x7f9b441501a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00c3e0 tx=0x7f9b2c00c8a0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 shutdown_connections 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9b2406bbd0 0x7f9b2406e090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b44161770 0x7f9b440ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b440b3270 0x7f9b441501a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 --2- 172.21.15.82:0/529272796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b440b1eb0 0x7f9b440b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.146+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 >> 172.21.15.82:0/529272796 conn(0x7f9b4409d1d0 msgr2=0x7f9b4406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.147+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 shutdown_connections 2024-03-14T12:13:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.147+0000 7f9b49953640 1 -- 172.21.15.82:0/529272796 wait complete. 2024-03-14T12:13:56.158 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-14T12:13:56.158 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:13:56.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.258+0000 7f95c6f59640 1 Processor -- start 2024-03-14T12:13:56.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c6f59640 1 -- start start 2024-03-14T12:13:56.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c6f59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 0x7f95c00ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c6f59640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95c0144ac0 0x7f95c0144f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c6f59640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00ce8b0 con 0x7f95c015f7a0 2024-03-14T12:13:56.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c6f59640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c00cf150 con 0x7f95c0144ac0 2024-03-14T12:13:56.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c6f59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00ceb90 con 0x7f95c00cdd60 2024-03-14T12:13:56.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c4cf5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f95c015f7a0 0x7f95c015fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34528/0 (socket says 172.21.15.82:34528) 2024-03-14T12:13:56.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.259+0000 7f95c4cf5640 1 -- 172.21.15.82:0/272897806 learned_addr learned my addr 172.21.15.82:0/272897806 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c54f6640 1 --2- 172.21.15.82:0/272897806 >> v2:172.21.15.164:3300/0 conn(0x7f95c0144ac0 0x7f95c0144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bffff640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 0x7f95c00ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473116396 0 0) 0x7f95c00ce8b0 con 0x7f95c015f7a0 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b0003520 con 0x7f95c015f7a0 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3726435244 0 0) 0x7f95b0003520 con 0x7f95c015f7a0 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 msgr2=0x7f95c00ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 0x7f95c00ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 >> v2:172.21.15.164:3300/0 conn(0x7f95c0144ac0 msgr2=0x7f95c0144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 --2- 172.21.15.82:0/272897806 >> v2:172.21.15.164:3300/0 conn(0x7f95c0144ac0 0x7f95c0144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00cee70 con 0x7f95c015f7a0 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f95b4001670 con 0x7f95c015f7a0 2024-03-14T12:13:56.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 >> v1:172.21.15.82:6789/0 conn(0x7f95c015f7a0 legacy=0x7f95c015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b00039b0 0x7f95b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b0004800 0x7f95b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 0x7f95b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00cf150 con 0x7f95b00039b0 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00ceb90 con 0x7f95b0004800 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b000b9f0 con 0x7f95c00cdd60 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c4cf5640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b00039b0 0x7f95b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95bffff640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b0004800 0x7f95b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c54f6640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 0x7f95b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c4cf5640 1 -- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 msgr2=0x7f95b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c4cf5640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 0x7f95b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c4cf5640 1 -- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b0004800 msgr2=0x7f95b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c4cf5640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b0004800 0x7f95b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c4cf5640 1 -- 172.21.15.82:0/272897806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0003520 con 0x7f95b00039b0 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.260+0000 7f95c54f6640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c00cdd60 0x7f95b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.261+0000 7f95c4cf5640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b00039b0 0x7f95b0003dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f95b4007fd0 tx=0x7f95b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322086e2226d5a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.261+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4007d50 con 0x7f95b00039b0 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.261+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b4030070 con 0x7f95b00039b0 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.261+0000 7f95bf7fe640 1 -- 172.21.15.82:0/272897806 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b402cd20 con 0x7f95b00039b0 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.262+0000 7f95c6f59640 1 -- 172.21.15.82:0/272897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b00039b0 msgr2=0x7f95b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.262+0000 7f95c6f59640 1 --2- 172.21.15.82:0/272897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b00039b0 0x7f95b0003dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f95b4007fd0 tx=0x7f95b0005830 comp rx=0 tx=0).stop 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.262+0000 7f95c54f6640 1 -- 172.21.15.82:0/272897806 reap_dead start 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.262+0000 7f95c6f59640 1 -- 172.21.15.82:0/272897806 shutdown_connections 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.262+0000 7f95c6f59640 1 -- 172.21.15.82:0/272897806 >> 172.21.15.82:0/272897806 conn(0x7f95c00b1c70 msgr2=0x7f95c00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.262+0000 7f95c6f59640 1 -- 172.21.15.82:0/272897806 shutdown_connections 2024-03-14T12:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.262+0000 7f95c6f59640 1 -- 172.21.15.82:0/272897806 wait complete. 2024-03-14T12:13:56.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.263+0000 7f95c6f59640 1 Processor -- start 2024-03-14T12:13:56.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.263+0000 7f95c6f59640 1 -- start start 2024-03-14T12:13:56.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c6f59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95b00039b0 0x7f95c01585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c6f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95c0132dd0 0x7f95c015e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c6f59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c0133250 0x7f95c00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c6f59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00cc670 con 0x7f95c0132dd0 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c6f59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00cc4f0 con 0x7f95c0133250 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c6f59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0070630 con 0x7f95b00039b0 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95c0132dd0 0x7f95c015e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95c0132dd0 0x7f95c015e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48582/0 (socket says 172.21.15.82:48582) 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 -- 172.21.15.82:0/108390301 learned_addr learned my addr 172.21.15.82:0/108390301 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c4cf5640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95b00039b0 0x7f95c01585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95c54f6640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c0133250 0x7f95c00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 -- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95b00039b0 msgr2=0x7f95c01585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95b00039b0 0x7f95c01585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 -- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c0133250 msgr2=0x7f95c00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c0133250 0x7f95c00709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 -- 172.21.15.82:0/108390301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c015f2b0 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.264+0000 7f95bffff640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95c0132dd0 0x7f95c015e4d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0118e0 tx=0x7f95ac011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.265+0000 7f95bd7fa640 1 -- 172.21.15.82:0/108390301 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00faf0 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.265+0000 7f95bd7fa640 1 -- 172.21.15.82:0/108390301 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac00fc90 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.265+0000 7f95bd7fa640 1 -- 172.21.15.82:0/108390301 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac014440 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.265+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c015efd0 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.265+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c00c4c30 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.266+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c0140b30 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.266+0000 7f95bd7fa640 1 -- 172.21.15.82:0/108390301 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f95ac012040 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.266+0000 7f95bd7fa640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f95a406bbd0 0x7f95a406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.266+0000 7f95bd7fa640 1 -- 172.21.15.82:0/108390301 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95ac08b000 con 0x7f95c0132dd0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.266+0000 7f95c4cf5640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f95a406bbd0 0x7f95a406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.267+0000 7f95c4cf5640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f95a406bbd0 0x7f95a406e090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95b4034000 tx=0x7f95b4007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:56.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.271+0000 7f95bd7fa640 1 -- 172.21.15.82:0/108390301 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f95c0140b30 con 0x7f95c0132dd0 2024-03-14T12:13:56.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.442+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95c004f050 con 0x7f95a406bbd0 2024-03-14T12:13:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.447+0000 7f95bd7fa640 1 -- 172.21.15.82:0/108390301 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251923 (secure 0 0 0) 0x7f95c004f050 con 0x7f95a406bbd0 2024-03-14T12:13:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:13:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.450+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f95a406bbd0 msgr2=0x7f95a406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.450+0000 7f95c6f59640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f95a406bbd0 0x7f95a406e090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95b4034000 tx=0x7f95b4007000 comp rx=0 tx=0).stop 2024-03-14T12:13:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95c0132dd0 msgr2=0x7f95c015e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95c0132dd0 0x7f95c015e4d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0118e0 tx=0x7f95ac011da0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 shutdown_connections 2024-03-14T12:13:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c0133250 0x7f95c00709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95c0132dd0 0x7f95c015e4d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f95a406bbd0 0x7f95a406e090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 --2- 172.21.15.82:0/108390301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95b00039b0 0x7f95c01585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 >> 172.21.15.82:0/108390301 conn(0x7f95c00b1c70 msgr2=0x7f95c00cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 shutdown_connections 2024-03-14T12:13:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:56.451+0000 7f95c6f59640 1 -- 172.21.15.82:0/108390301 wait complete. 2024-03-14T12:13:56.462 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-14T12:13:55.498766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219060,"kb_used_data":2872,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755660,"statfs":{"total":773094113280,"available":772869795840,"internally_reserved":0,"allocated":2940928,"data_stored":1765196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952078"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.588354+0000","last_change":"2024-03-14T12:13:51.582275+0000","last_active":"2024-03-14T12:13:51.588354+0000","last_peered":"2024-03-14T12:13:51.588354+0000","last_clean":"2024-03-14T12:13:51.588354+0000","last_became_active":"2024-03-14T12:13:51.581999+0000","last_became_peered":"2024-03-14T12:13:51.581999+0000","last_unstale":"2024-03-14T12:13:51.588354+0000","last_undegraded":"2024-03-14T12:13:51.588354+0000","last_fullsized":"2024-03-14T12:13:51.588354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574007+0000","last_change":"2024-03-14T12:13:51.575382+0000","last_active":"2024-03-14T12:13:52.574007+0000","last_peered":"2024-03-14T12:13:52.574007+0000","last_clean":"2024-03-14T12:13:52.574007+0000","last_became_active":"2024-03-14T12:13:51.575139+0000","last_became_peered":"2024-03-14T12:13:51.575139+0000","last_unstale":"2024-03-14T12:13:52.574007+0000","last_undegraded":"2024-03-14T12:13:52.574007+0000","last_fullsized":"2024-03-14T12:13:52.574007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777473+0000","last_change":"2024-03-14T12:13:51.574924+0000","last_active":"2024-03-14T12:13:52.777473+0000","last_peered":"2024-03-14T12:13:52.777473+0000","last_clean":"2024-03-14T12:13:52.777473+0000","last_became_active":"2024-03-14T12:13:51.574299+0000","last_became_peered":"2024-03-14T12:13:51.574299+0000","last_unstale":"2024-03-14T12:13:52.777473+0000","last_undegraded":"2024-03-14T12:13:52.777473+0000","last_fullsized":"2024-03-14T12:13:52.777473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579676+0000","last_change":"2024-03-14T12:13:51.581446+0000","last_active":"2024-03-14T12:13:52.579676+0000","last_peered":"2024-03-14T12:13:52.579676+0000","last_clean":"2024-03-14T12:13:52.579676+0000","last_became_active":"2024-03-14T12:13:51.581192+0000","last_became_peered":"2024-03-14T12:13:51.581192+0000","last_unstale":"2024-03-14T12:13:52.579676+0000","last_undegraded":"2024-03-14T12:13:52.579676+0000","last_fullsized":"2024-03-14T12:13:52.579676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.593758+0000","last_change":"2024-03-14T12:13:51.587466+0000","last_active":"2024-03-14T12:13:51.593758+0000","last_peered":"2024-03-14T12:13:51.593758+0000","last_clean":"2024-03-14T12:13:51.593758+0000","last_became_active":"2024-03-14T12:13:51.585375+0000","last_became_peered":"2024-03-14T12:13:51.585375+0000","last_unstale":"2024-03-14T12:13:51.593758+0000","last_undegraded":"2024-03-14T12:13:51.593758+0000","last_fullsized":"2024-03-14T12:13:51.593758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:21:17.245054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580498+0000","last_change":"2024-03-14T12:13:51.580794+0000","last_active":"2024-03-14T12:13:52.580498+0000","last_peered":"2024-03-14T12:13:52.580498+0000","last_clean":"2024-03-14T12:13:52.580498+0000","last_became_active":"2024-03-14T12:13:51.580511+0000","last_became_peered":"2024-03-14T12:13:51.580511+0000","last_unstale":"2024-03-14T12:13:52.580498+0000","last_undegraded":"2024-03-14T12:13:52.580498+0000","last_fullsized":"2024-03-14T12:13:52.580498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596271+0000","last_change":"2024-03-14T12:13:51.588949+0000","last_active":"2024-03-14T12:13:51.596271+0000","last_peered":"2024-03-14T12:13:51.596271+0000","last_clean":"2024-03-14T12:13:51.596271+0000","last_became_active":"2024-03-14T12:13:51.588671+0000","last_became_peered":"2024-03-14T12:13:51.588671+0000","last_unstale":"2024-03-14T12:13:51.596271+0000","last_undegraded":"2024-03-14T12:13:51.596271+0000","last_fullsized":"2024-03-14T12:13:51.596271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:01.290201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.582077+0000","last_change":"2024-03-14T12:13:51.579815+0000","last_active":"2024-03-14T12:13:51.582077+0000","last_peered":"2024-03-14T12:13:51.582077+0000","last_clean":"2024-03-14T12:13:51.582077+0000","last_became_active":"2024-03-14T12:13:51.579580+0000","last_became_peered":"2024-03-14T12:13:51.579580+0000","last_unstale":"2024-03-14T12:13:51.582077+0000","last_undegraded":"2024-03-14T12:13:51.582077+0000","last_fullsized":"2024-03-14T12:13:51.582077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.598887+0000","last_change":"2024-03-14T12:13:51.593442+0000","last_active":"2024-03-14T12:13:51.598887+0000","last_peered":"2024-03-14T12:13:51.598887+0000","last_clean":"2024-03-14T12:13:51.598887+0000","last_became_active":"2024-03-14T12:13:51.593151+0000","last_became_peered":"2024-03-14T12:13:51.593151+0000","last_unstale":"2024-03-14T12:13:51.598887+0000","last_undegraded":"2024-03-14T12:13:51.598887+0000","last_fullsized":"2024-03-14T12:13:51.598887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582259+0000","last_change":"2024-03-14T12:13:51.584838+0000","last_active":"2024-03-14T12:13:52.582259+0000","last_peered":"2024-03-14T12:13:52.582259+0000","last_clean":"2024-03-14T12:13:52.582259+0000","last_became_active":"2024-03-14T12:13:51.584501+0000","last_became_peered":"2024-03-14T12:13:51.584501+0000","last_unstale":"2024-03-14T12:13:52.582259+0000","last_undegraded":"2024-03-14T12:13:52.582259+0000","last_fullsized":"2024-03-14T12:13:52.582259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581960+0000","last_change":"2024-03-14T12:13:51.584374+0000","last_active":"2024-03-14T12:13:52.581960+0000","last_peered":"2024-03-14T12:13:52.581960+0000","last_clean":"2024-03-14T12:13:52.581960+0000","last_became_active":"2024-03-14T12:13:51.584094+0000","last_became_peered":"2024-03-14T12:13:51.584094+0000","last_unstale":"2024-03-14T12:13:52.581960+0000","last_undegraded":"2024-03-14T12:13:52.581960+0000","last_fullsized":"2024-03-14T12:13:52.581960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580082+0000","last_change":"2024-03-14T12:13:51.583802+0000","last_active":"2024-03-14T12:13:52.580082+0000","last_peered":"2024-03-14T12:13:52.580082+0000","last_clean":"2024-03-14T12:13:52.580082+0000","last_became_active":"2024-03-14T12:13:51.583284+0000","last_became_peered":"2024-03-14T12:13:51.583284+0000","last_unstale":"2024-03-14T12:13:52.580082+0000","last_undegraded":"2024-03-14T12:13:52.580082+0000","last_fullsized":"2024-03-14T12:13:52.580082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.580263+0000","last_change":"2024-03-14T12:13:51.578671+0000","last_active":"2024-03-14T12:13:51.580263+0000","last_peered":"2024-03-14T12:13:51.580263+0000","last_clean":"2024-03-14T12:13:51.580263+0000","last_became_active":"2024-03-14T12:13:51.578427+0000","last_became_peered":"2024-03-14T12:13:51.578427+0000","last_unstale":"2024-03-14T12:13:51.580263+0000","last_undegraded":"2024-03-14T12:13:51.580263+0000","last_fullsized":"2024-03-14T12:13:51.580263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:39.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777400+0000","last_change":"2024-03-14T12:13:51.581289+0000","last_active":"2024-03-14T12:13:52.777400+0000","last_peered":"2024-03-14T12:13:52.777400+0000","last_clean":"2024-03-14T12:13:52.777400+0000","last_became_active":"2024-03-14T12:13:51.580645+0000","last_became_peered":"2024-03-14T12:13:51.580645+0000","last_unstale":"2024-03-14T12:13:52.777400+0000","last_undegraded":"2024-03-14T12:13:52.777400+0000","last_fullsized":"2024-03-14T12:13:52.777400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:01.290201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573796+0000","last_change":"2024-03-14T12:13:51.571032+0000","last_active":"2024-03-14T12:13:52.573796+0000","last_peered":"2024-03-14T12:13:52.573796+0000","last_clean":"2024-03-14T12:13:52.573796+0000","last_became_active":"2024-03-14T12:13:51.570723+0000","last_became_peered":"2024-03-14T12:13:51.570723+0000","last_unstale":"2024-03-14T12:13:52.573796+0000","last_undegraded":"2024-03-14T12:13:52.573796+0000","last_fullsized":"2024-03-14T12:13:52.573796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580831+0000","last_change":"2024-03-14T12:13:51.582495+0000","last_active":"2024-03-14T12:13:52.580831+0000","last_peered":"2024-03-14T12:13:52.580831+0000","last_clean":"2024-03-14T12:13:52.580831+0000","last_became_active":"2024-03-14T12:13:51.582170+0000","last_became_peered":"2024-03-14T12:13:51.582170+0000","last_unstale":"2024-03-14T12:13:52.580831+0000","last_undegraded":"2024-03-14T12:13:52.580831+0000","last_fullsized":"2024-03-14T12:13:52.580831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580861+0000","last_change":"2024-03-14T12:13:51.582330+0000","last_active":"2024-03-14T12:13:52.580861+0000","last_peered":"2024-03-14T12:13:52.580861+0000","last_clean":"2024-03-14T12:13:52.580861+0000","last_became_active":"2024-03-14T12:13:51.581920+0000","last_became_peered":"2024-03-14T12:13:51.581920+0000","last_unstale":"2024-03-14T12:13:52.580861+0000","last_undegraded":"2024-03-14T12:13:52.580861+0000","last_fullsized":"2024-03-14T12:13:52.580861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.583893+0000","last_change":"2024-03-14T12:13:51.582708+0000","last_active":"2024-03-14T12:13:51.583893+0000","last_peered":"2024-03-14T12:13:51.583893+0000","last_clean":"2024-03-14T12:13:51.583893+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:51.583893+0000","last_undegraded":"2024-03-14T12:13:51.583893+0000","last_fullsized":"2024-03-14T12:13:51.583893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.593704+0000","last_change":"2024-03-14T12:13:51.587758+0000","last_active":"2024-03-14T12:13:51.593704+0000","last_peered":"2024-03-14T12:13:51.593704+0000","last_clean":"2024-03-14T12:13:51.593704+0000","last_became_active":"2024-03-14T12:13:51.586310+0000","last_became_peered":"2024-03-14T12:13:51.586310+0000","last_unstale":"2024-03-14T12:13:51.593704+0000","last_undegraded":"2024-03-14T12:13:51.593704+0000","last_fullsized":"2024-03-14T12:13:51.593704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582057+0000","last_change":"2024-03-14T12:13:51.589443+0000","last_active":"2024-03-14T12:13:52.582057+0000","last_peered":"2024-03-14T12:13:52.582057+0000","last_clean":"2024-03-14T12:13:52.582057+0000","last_became_active":"2024-03-14T12:13:51.589174+0000","last_became_peered":"2024-03-14T12:13:51.589174+0000","last_unstale":"2024-03-14T12:13:52.582057+0000","last_undegraded":"2024-03-14T12:13:52.582057+0000","last_fullsized":"2024-03-14T12:13:52.582057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777419+0000","last_change":"2024-03-14T12:13:51.580813+0000","last_active":"2024-03-14T12:13:52.777419+0000","last_peered":"2024-03-14T12:13:52.777419+0000","last_clean":"2024-03-14T12:13:52.777419+0000","last_became_active":"2024-03-14T12:13:51.580334+0000","last_became_peered":"2024-03-14T12:13:51.580334+0000","last_unstale":"2024-03-14T12:13:52.777419+0000","last_undegraded":"2024-03-14T12:13:52.777419+0000","last_fullsized":"2024-03-14T12:13:52.777419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580443+0000","last_change":"2024-03-14T12:13:51.583228+0000","last_active":"2024-03-14T12:13:52.580443+0000","last_peered":"2024-03-14T12:13:52.580443+0000","last_clean":"2024-03-14T12:13:52.580443+0000","last_became_active":"2024-03-14T12:13:51.582937+0000","last_became_peered":"2024-03-14T12:13:51.582937+0000","last_unstale":"2024-03-14T12:13:52.580443+0000","last_undegraded":"2024-03-14T12:13:52.580443+0000","last_fullsized":"2024-03-14T12:13:52.580443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.584037+0000","last_change":"2024-03-14T12:13:51.582656+0000","last_active":"2024-03-14T12:13:51.584037+0000","last_peered":"2024-03-14T12:13:51.584037+0000","last_clean":"2024-03-14T12:13:51.584037+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:51.584037+0000","last_undegraded":"2024-03-14T12:13:51.584037+0000","last_fullsized":"2024-03-14T12:13:51.584037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580802+0000","last_change":"2024-03-14T12:13:51.579810+0000","last_active":"2024-03-14T12:13:52.580802+0000","last_peered":"2024-03-14T12:13:52.580802+0000","last_clean":"2024-03-14T12:13:52.580802+0000","last_became_active":"2024-03-14T12:13:51.579523+0000","last_became_peered":"2024-03-14T12:13:51.579523+0000","last_unstale":"2024-03-14T12:13:52.580802+0000","last_undegraded":"2024-03-14T12:13:52.580802+0000","last_fullsized":"2024-03-14T12:13:52.580802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.584031+0000","last_change":"2024-03-14T12:13:51.582442+0000","last_active":"2024-03-14T12:13:51.584031+0000","last_peered":"2024-03-14T12:13:51.584031+0000","last_clean":"2024-03-14T12:13:51.584031+0000","last_became_active":"2024-03-14T12:13:51.582175+0000","last_became_peered":"2024-03-14T12:13:51.582175+0000","last_unstale":"2024-03-14T12:13:51.584031+0000","last_undegraded":"2024-03-14T12:13:51.584031+0000","last_fullsized":"2024-03-14T12:13:51.584031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:41:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582355+0000","last_change":"2024-03-14T12:13:51.590516+0000","last_active":"2024-03-14T12:13:52.582355+0000","last_peered":"2024-03-14T12:13:52.582355+0000","last_clean":"2024-03-14T12:13:52.582355+0000","last_became_active":"2024-03-14T12:13:51.590223+0000","last_became_peered":"2024-03-14T12:13:51.590223+0000","last_unstale":"2024-03-14T12:13:52.582355+0000","last_undegraded":"2024-03-14T12:13:52.582355+0000","last_fullsized":"2024-03-14T12:13:52.582355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573995+0000","last_change":"2024-03-14T12:13:51.574721+0000","last_active":"2024-03-14T12:13:52.573995+0000","last_peered":"2024-03-14T12:13:52.573995+0000","last_clean":"2024-03-14T12:13:52.573995+0000","last_became_active":"2024-03-14T12:13:51.574443+0000","last_became_peered":"2024-03-14T12:13:51.574443+0000","last_unstale":"2024-03-14T12:13:52.573995+0000","last_undegraded":"2024-03-14T12:13:52.573995+0000","last_fullsized":"2024-03-14T12:13:52.573995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.584024+0000","last_change":"2024-03-14T12:13:51.582640+0000","last_active":"2024-03-14T12:13:51.584024+0000","last_peered":"2024-03-14T12:13:51.584024+0000","last_clean":"2024-03-14T12:13:51.584024+0000","last_became_active":"2024-03-14T12:13:51.582355+0000","last_became_peered":"2024-03-14T12:13:51.582355+0000","last_unstale":"2024-03-14T12:13:51.584024+0000","last_undegraded":"2024-03-14T12:13:51.584024+0000","last_fullsized":"2024-03-14T12:13:51.584024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:21:17.245054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777477+0000","last_change":"2024-03-14T12:13:51.577794+0000","last_active":"2024-03-14T12:13:52.777477+0000","last_peered":"2024-03-14T12:13:52.777477+0000","last_clean":"2024-03-14T12:13:52.777477+0000","last_became_active":"2024-03-14T12:13:51.577518+0000","last_became_peered":"2024-03-14T12:13:51.577518+0000","last_unstale":"2024-03-14T12:13:52.777477+0000","last_undegraded":"2024-03-14T12:13:52.777477+0000","last_fullsized":"2024-03-14T12:13:52.777477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580629+0000","last_change":"2024-03-14T12:13:51.580193+0000","last_active":"2024-03-14T12:13:52.580629+0000","last_peered":"2024-03-14T12:13:52.580629+0000","last_clean":"2024-03-14T12:13:52.580629+0000","last_became_active":"2024-03-14T12:13:51.577667+0000","last_became_peered":"2024-03-14T12:13:51.577667+0000","last_unstale":"2024-03-14T12:13:52.580629+0000","last_undegraded":"2024-03-14T12:13:52.580629+0000","last_fullsized":"2024-03-14T12:13:52.580629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.581932+0000","last_change":"2024-03-14T12:13:51.579889+0000","last_active":"2024-03-14T12:13:51.581932+0000","last_peered":"2024-03-14T12:13:51.581932+0000","last_clean":"2024-03-14T12:13:51.581932+0000","last_became_active":"2024-03-14T12:13:51.579585+0000","last_became_peered":"2024-03-14T12:13:51.579585+0000","last_unstale":"2024-03-14T12:13:51.581932+0000","last_undegraded":"2024-03-14T12:13:51.581932+0000","last_fullsized":"2024-03-14T12:13:51.581932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:55.327329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581817+0000","last_change":"2024-03-14T12:13:51.577340+0000","last_active":"2024-03-14T12:13:52.581817+0000","last_peered":"2024-03-14T12:13:52.581817+0000","last_clean":"2024-03-14T12:13:52.581817+0000","last_became_active":"2024-03-14T12:13:51.577080+0000","last_became_peered":"2024-03-14T12:13:51.577080+0000","last_unstale":"2024-03-14T12:13:52.581817+0000","last_undegraded":"2024-03-14T12:13:52.581817+0000","last_fullsized":"2024-03-14T12:13:52.581817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579599+0000","last_change":"2024-03-14T12:13:51.573447+0000","last_active":"2024-03-14T12:13:52.579599+0000","last_peered":"2024-03-14T12:13:52.579599+0000","last_clean":"2024-03-14T12:13:52.579599+0000","last_became_active":"2024-03-14T12:13:51.573137+0000","last_became_peered":"2024-03-14T12:13:51.573137+0000","last_unstale":"2024-03-14T12:13:52.579599+0000","last_undegraded":"2024-03-14T12:13:52.579599+0000","last_fullsized":"2024-03-14T12:13:52.579599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596461+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:51.596461+0000","last_peered":"2024-03-14T12:13:51.596461+0000","last_clean":"2024-03-14T12:13:51.596461+0000","last_became_active":"2024-03-14T12:13:50.569748+0000","last_became_peered":"2024-03-14T12:13:50.569748+0000","last_unstale":"2024-03-14T12:13:51.596461+0000","last_undegraded":"2024-03-14T12:13:51.596461+0000","last_fullsized":"2024-03-14T12:13:51.596461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777628+0000","last_change":"2024-03-14T12:13:51.581108+0000","last_active":"2024-03-14T12:13:52.777628+0000","last_peered":"2024-03-14T12:13:52.777628+0000","last_clean":"2024-03-14T12:13:52.777628+0000","last_became_active":"2024-03-14T12:13:51.580829+0000","last_became_peered":"2024-03-14T12:13:51.580829+0000","last_unstale":"2024-03-14T12:13:52.777628+0000","last_undegraded":"2024-03-14T12:13:52.777628+0000","last_fullsized":"2024-03-14T12:13:52.777628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581951+0000","last_change":"2024-03-14T12:13:50.565826+0000","last_active":"2024-03-14T12:13:52.581951+0000","last_peered":"2024-03-14T12:13:52.581951+0000","last_clean":"2024-03-14T12:13:52.581951+0000","last_became_active":"2024-03-14T12:13:50.565553+0000","last_became_peered":"2024-03-14T12:13:50.565553+0000","last_unstale":"2024-03-14T12:13:52.581951+0000","last_undegraded":"2024-03-14T12:13:52.581951+0000","last_fullsized":"2024-03-14T12:13:52.581951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.568911+0000","last_change":"2024-03-14T12:13:50.563462+0000","last_active":"2024-03-14T12:13:51.568911+0000","last_peered":"2024-03-14T12:13:51.568911+0000","last_clean":"2024-03-14T12:13:51.568911+0000","last_became_active":"2024-03-14T12:13:50.563198+0000","last_became_peered":"2024-03-14T12:13:50.563198+0000","last_unstale":"2024-03-14T12:13:51.568911+0000","last_undegraded":"2024-03-14T12:13:51.568911+0000","last_fullsized":"2024-03-14T12:13:51.568911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.578083+0000","last_change":"2024-03-14T12:13:51.577103+0000","last_active":"2024-03-14T12:13:51.578083+0000","last_peered":"2024-03-14T12:13:51.578083+0000","last_clean":"2024-03-14T12:13:51.578083+0000","last_became_active":"2024-03-14T12:13:51.576602+0000","last_became_peered":"2024-03-14T12:13:51.576602+0000","last_unstale":"2024-03-14T12:13:51.578083+0000","last_undegraded":"2024-03-14T12:13:51.578083+0000","last_fullsized":"2024-03-14T12:13:51.578083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580881+0000","last_change":"2024-03-14T12:13:50.559389+0000","last_active":"2024-03-14T12:13:52.580881+0000","last_peered":"2024-03-14T12:13:52.580881+0000","last_clean":"2024-03-14T12:13:52.580881+0000","last_became_active":"2024-03-14T12:13:50.559100+0000","last_became_peered":"2024-03-14T12:13:50.559100+0000","last_unstale":"2024-03-14T12:13:52.580881+0000","last_undegraded":"2024-03-14T12:13:52.580881+0000","last_fullsized":"2024-03-14T12:13:52.580881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573924+0000","last_change":"2024-03-14T12:13:51.576591+0000","last_active":"2024-03-14T12:13:52.573924+0000","last_peered":"2024-03-14T12:13:52.573924+0000","last_clean":"2024-03-14T12:13:52.573924+0000","last_became_active":"2024-03-14T12:13:51.576232+0000","last_became_peered":"2024-03-14T12:13:51.576232+0000","last_unstale":"2024-03-14T12:13:52.573924+0000","last_undegraded":"2024-03-14T12:13:52.573924+0000","last_fullsized":"2024-03-14T12:13:52.573924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581963+0000","last_change":"2024-03-14T12:13:50.568392+0000","last_active":"2024-03-14T12:13:52.581963+0000","last_peered":"2024-03-14T12:13:52.581963+0000","last_clean":"2024-03-14T12:13:52.581963+0000","last_became_active":"2024-03-14T12:13:50.568127+0000","last_became_peered":"2024-03-14T12:13:50.568127+0000","last_unstale":"2024-03-14T12:13:52.581963+0000","last_undegraded":"2024-03-14T12:13:52.581963+0000","last_fullsized":"2024-03-14T12:13:52.581963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580126+0000","last_change":"2024-03-14T12:13:51.584429+0000","last_active":"2024-03-14T12:13:52.580126+0000","last_peered":"2024-03-14T12:13:52.580126+0000","last_clean":"2024-03-14T12:13:52.580126+0000","last_became_active":"2024-03-14T12:13:51.584098+0000","last_became_peered":"2024-03-14T12:13:51.584098+0000","last_unstale":"2024-03-14T12:13:52.580126+0000","last_undegraded":"2024-03-14T12:13:52.580126+0000","last_fullsized":"2024-03-14T12:13:52.580126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580241+0000","last_change":"2024-03-14T12:13:50.567672+0000","last_active":"2024-03-14T12:13:52.580241+0000","last_peered":"2024-03-14T12:13:52.580241+0000","last_clean":"2024-03-14T12:13:52.580241+0000","last_became_active":"2024-03-14T12:13:50.567366+0000","last_became_peered":"2024-03-14T12:13:50.567366+0000","last_unstale":"2024-03-14T12:13:52.580241+0000","last_undegraded":"2024-03-14T12:13:52.580241+0000","last_fullsized":"2024-03-14T12:13:52.580241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777722+0000","last_change":"2024-03-14T12:13:51.575322+0000","last_active":"2024-03-14T12:13:52.777722+0000","last_peered":"2024-03-14T12:13:52.777722+0000","last_clean":"2024-03-14T12:13:52.777722+0000","last_became_active":"2024-03-14T12:13:51.574694+0000","last_became_peered":"2024-03-14T12:13:51.574694+0000","last_unstale":"2024-03-14T12:13:52.777722+0000","last_undegraded":"2024-03-14T12:13:52.777722+0000","last_fullsized":"2024-03-14T12:13:52.777722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573967+0000","last_change":"2024-03-14T12:13:50.567057+0000","last_active":"2024-03-14T12:13:52.573967+0000","last_peered":"2024-03-14T12:13:52.573967+0000","last_clean":"2024-03-14T12:13:52.573967+0000","last_became_active":"2024-03-14T12:13:50.566800+0000","last_became_peered":"2024-03-14T12:13:50.566800+0000","last_unstale":"2024-03-14T12:13:52.573967+0000","last_undegraded":"2024-03-14T12:13:52.573967+0000","last_fullsized":"2024-03-14T12:13:52.573967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580236+0000","last_change":"2024-03-14T12:13:51.585716+0000","last_active":"2024-03-14T12:13:52.580236+0000","last_peered":"2024-03-14T12:13:52.580236+0000","last_clean":"2024-03-14T12:13:52.580236+0000","last_became_active":"2024-03-14T12:13:51.585226+0000","last_became_peered":"2024-03-14T12:13:51.585226+0000","last_unstale":"2024-03-14T12:13:52.580236+0000","last_undegraded":"2024-03-14T12:13:52.580236+0000","last_fullsized":"2024-03-14T12:13:52.580236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582041+0000","last_change":"2024-03-14T12:13:50.562666+0000","last_active":"2024-03-14T12:13:52.582041+0000","last_peered":"2024-03-14T12:13:52.582041+0000","last_clean":"2024-03-14T12:13:52.582041+0000","last_became_active":"2024-03-14T12:13:50.562367+0000","last_became_peered":"2024-03-14T12:13:50.562367+0000","last_unstale":"2024-03-14T12:13:52.582041+0000","last_undegraded":"2024-03-14T12:13:52.582041+0000","last_fullsized":"2024-03-14T12:13:52.582041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.593676+0000","last_change":"2024-03-14T12:13:51.587183+0000","last_active":"2024-03-14T12:13:51.593676+0000","last_peered":"2024-03-14T12:13:51.593676+0000","last_clean":"2024-03-14T12:13:51.593676+0000","last_became_active":"2024-03-14T12:13:51.585685+0000","last_became_peered":"2024-03-14T12:13:51.585685+0000","last_unstale":"2024-03-14T12:13:51.593676+0000","last_undegraded":"2024-03-14T12:13:51.593676+0000","last_fullsized":"2024-03-14T12:13:51.593676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:41:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777610+0000","last_change":"2024-03-14T12:13:50.557053+0000","last_active":"2024-03-14T12:13:52.777610+0000","last_peered":"2024-03-14T12:13:52.777610+0000","last_clean":"2024-03-14T12:13:52.777610+0000","last_became_active":"2024-03-14T12:13:50.556737+0000","last_became_peered":"2024-03-14T12:13:50.556737+0000","last_unstale":"2024-03-14T12:13:52.777610+0000","last_undegraded":"2024-03-14T12:13:52.777610+0000","last_fullsized":"2024-03-14T12:13:52.777610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777653+0000","last_change":"2024-03-14T12:13:51.581665+0000","last_active":"2024-03-14T12:13:52.777653+0000","last_peered":"2024-03-14T12:13:52.777653+0000","last_clean":"2024-03-14T12:13:52.777653+0000","last_became_active":"2024-03-14T12:13:51.580980+0000","last_became_peered":"2024-03-14T12:13:51.580980+0000","last_unstale":"2024-03-14T12:13:52.777653+0000","last_undegraded":"2024-03-14T12:13:52.777653+0000","last_fullsized":"2024-03-14T12:13:52.777653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580677+0000","last_change":"2024-03-14T12:13:50.566403+0000","last_active":"2024-03-14T12:13:52.580677+0000","last_peered":"2024-03-14T12:13:52.580677+0000","last_clean":"2024-03-14T12:13:52.580677+0000","last_became_active":"2024-03-14T12:13:50.565782+0000","last_became_peered":"2024-03-14T12:13:50.565782+0000","last_unstale":"2024-03-14T12:13:52.580677+0000","last_undegraded":"2024-03-14T12:13:52.580677+0000","last_fullsized":"2024-03-14T12:13:52.580677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596562+0000","last_change":"2024-03-14T12:13:51.592512+0000","last_active":"2024-03-14T12:13:51.596562+0000","last_peered":"2024-03-14T12:13:51.596562+0000","last_clean":"2024-03-14T12:13:51.596562+0000","last_became_active":"2024-03-14T12:13:51.592273+0000","last_became_peered":"2024-03-14T12:13:51.592273+0000","last_unstale":"2024-03-14T12:13:51.596562+0000","last_undegraded":"2024-03-14T12:13:51.596562+0000","last_fullsized":"2024-03-14T12:13:51.596562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581835+0000","last_change":"2024-03-14T12:13:50.564239+0000","last_active":"2024-03-14T12:13:52.581835+0000","last_peered":"2024-03-14T12:13:52.581835+0000","last_clean":"2024-03-14T12:13:52.581835+0000","last_became_active":"2024-03-14T12:13:50.563920+0000","last_became_peered":"2024-03-14T12:13:50.563920+0000","last_unstale":"2024-03-14T12:13:52.581835+0000","last_undegraded":"2024-03-14T12:13:52.581835+0000","last_fullsized":"2024-03-14T12:13:52.581835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581784+0000","last_change":"2024-03-14T12:13:51.576682+0000","last_active":"2024-03-14T12:13:52.581784+0000","last_peered":"2024-03-14T12:13:52.581784+0000","last_clean":"2024-03-14T12:13:52.581784+0000","last_became_active":"2024-03-14T12:13:51.576420+0000","last_became_peered":"2024-03-14T12:13:51.576420+0000","last_unstale":"2024-03-14T12:13:52.581784+0000","last_undegraded":"2024-03-14T12:13:52.581784+0000","last_fullsized":"2024-03-14T12:13:52.581784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.592862+0000","last_change":"2024-03-14T12:13:50.564352+0000","last_active":"2024-03-14T12:13:51.592862+0000","last_peered":"2024-03-14T12:13:51.592862+0000","last_clean":"2024-03-14T12:13:51.592862+0000","last_became_active":"2024-03-14T12:13:50.563979+0000","last_became_peered":"2024-03-14T12:13:50.563979+0000","last_unstale":"2024-03-14T12:13:51.592862+0000","last_undegraded":"2024-03-14T12:13:51.592862+0000","last_fullsized":"2024-03-14T12:13:51.592862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596570+0000","last_change":"2024-03-14T12:13:51.592382+0000","last_active":"2024-03-14T12:13:51.596570+0000","last_peered":"2024-03-14T12:13:51.596570+0000","last_clean":"2024-03-14T12:13:51.596570+0000","last_became_active":"2024-03-14T12:13:51.592109+0000","last_became_peered":"2024-03-14T12:13:51.592109+0000","last_unstale":"2024-03-14T12:13:51.596570+0000","last_undegraded":"2024-03-14T12:13:51.596570+0000","last_fullsized":"2024-03-14T12:13:51.596570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579929+0000","last_change":"2024-03-14T12:13:50.565493+0000","last_active":"2024-03-14T12:13:52.579929+0000","last_peered":"2024-03-14T12:13:52.579929+0000","last_clean":"2024-03-14T12:13:52.579929+0000","last_became_active":"2024-03-14T12:13:50.565247+0000","last_became_peered":"2024-03-14T12:13:50.565247+0000","last_unstale":"2024-03-14T12:13:52.579929+0000","last_undegraded":"2024-03-14T12:13:52.579929+0000","last_fullsized":"2024-03-14T12:13:52.579929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:39:10.529585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596442+0000","last_change":"2024-03-14T12:13:51.591007+0000","last_active":"2024-03-14T12:13:51.596442+0000","last_peered":"2024-03-14T12:13:51.596442+0000","last_clean":"2024-03-14T12:13:51.596442+0000","last_became_active":"2024-03-14T12:13:51.590750+0000","last_became_peered":"2024-03-14T12:13:51.590750+0000","last_unstale":"2024-03-14T12:13:51.596442+0000","last_undegraded":"2024-03-14T12:13:51.596442+0000","last_fullsized":"2024-03-14T12:13:51.596442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.589972+0000","last_change":"2024-03-14T12:13:50.568050+0000","last_active":"2024-03-14T12:13:51.589972+0000","last_peered":"2024-03-14T12:13:51.589972+0000","last_clean":"2024-03-14T12:13:51.589972+0000","last_became_active":"2024-03-14T12:13:50.567713+0000","last_became_peered":"2024-03-14T12:13:50.567713+0000","last_unstale":"2024-03-14T12:13:51.589972+0000","last_undegraded":"2024-03-14T12:13:51.589972+0000","last_fullsized":"2024-03-14T12:13:51.589972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596589+0000","last_change":"2024-03-14T12:13:51.592562+0000","last_active":"2024-03-14T12:13:51.596589+0000","last_peered":"2024-03-14T12:13:51.596589+0000","last_clean":"2024-03-14T12:13:51.596589+0000","last_became_active":"2024-03-14T12:13:51.592255+0000","last_became_peered":"2024-03-14T12:13:51.592255+0000","last_unstale":"2024-03-14T12:13:51.596589+0000","last_undegraded":"2024-03-14T12:13:51.596589+0000","last_fullsized":"2024-03-14T12:13:51.596589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574041+0000","last_change":"2024-03-14T12:13:50.566339+0000","last_active":"2024-03-14T12:13:52.574041+0000","last_peered":"2024-03-14T12:13:52.574041+0000","last_clean":"2024-03-14T12:13:52.574041+0000","last_became_active":"2024-03-14T12:13:50.566058+0000","last_became_peered":"2024-03-14T12:13:50.566058+0000","last_unstale":"2024-03-14T12:13:52.574041+0000","last_undegraded":"2024-03-14T12:13:52.574041+0000","last_fullsized":"2024-03-14T12:13:52.574041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579993+0000","last_change":"2024-03-14T12:13:51.581453+0000","last_active":"2024-03-14T12:13:52.579993+0000","last_peered":"2024-03-14T12:13:52.579993+0000","last_clean":"2024-03-14T12:13:52.579993+0000","last_became_active":"2024-03-14T12:13:51.581168+0000","last_became_peered":"2024-03-14T12:13:51.581168+0000","last_unstale":"2024-03-14T12:13:52.579993+0000","last_undegraded":"2024-03-14T12:13:52.579993+0000","last_fullsized":"2024-03-14T12:13:52.579993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.570338+0000","last_change":"2024-03-14T12:13:50.568172+0000","last_active":"2024-03-14T12:13:51.570338+0000","last_peered":"2024-03-14T12:13:51.570338+0000","last_clean":"2024-03-14T12:13:51.570338+0000","last_became_active":"2024-03-14T12:13:50.567825+0000","last_became_peered":"2024-03-14T12:13:50.567825+0000","last_unstale":"2024-03-14T12:13:51.570338+0000","last_undegraded":"2024-03-14T12:13:51.570338+0000","last_fullsized":"2024-03-14T12:13:51.570338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.598902+0000","last_change":"2024-03-14T12:13:51.593995+0000","last_active":"2024-03-14T12:13:51.598902+0000","last_peered":"2024-03-14T12:13:51.598902+0000","last_clean":"2024-03-14T12:13:51.598902+0000","last_became_active":"2024-03-14T12:13:51.593473+0000","last_became_peered":"2024-03-14T12:13:51.593473+0000","last_unstale":"2024-03-14T12:13:51.598902+0000","last_undegraded":"2024-03-14T12:13:51.598902+0000","last_fullsized":"2024-03-14T12:13:51.598902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.575200+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:13:51.575200+0000","last_peered":"2024-03-14T12:13:51.575200+0000","last_clean":"2024-03-14T12:13:51.575200+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:13:51.575200+0000","last_undegraded":"2024-03-14T12:13:51.575200+0000","last_fullsized":"2024-03-14T12:13:51.575200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580336+0000","last_change":"2024-03-14T12:13:50.565229+0000","last_active":"2024-03-14T12:13:52.580336+0000","last_peered":"2024-03-14T12:13:52.580336+0000","last_clean":"2024-03-14T12:13:52.580336+0000","last_became_active":"2024-03-14T12:13:50.564952+0000","last_became_peered":"2024-03-14T12:13:50.564952+0000","last_unstale":"2024-03-14T12:13:52.580336+0000","last_undegraded":"2024-03-14T12:13:52.580336+0000","last_fullsized":"2024-03-14T12:13:52.580336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.576826+0000","last_change":"2024-03-14T12:13:51.573189+0000","last_active":"2024-03-14T12:13:51.576826+0000","last_peered":"2024-03-14T12:13:51.576826+0000","last_clean":"2024-03-14T12:13:51.576826+0000","last_became_active":"2024-03-14T12:13:51.572900+0000","last_became_peered":"2024-03-14T12:13:51.572900+0000","last_unstale":"2024-03-14T12:13:51.576826+0000","last_undegraded":"2024-03-14T12:13:51.576826+0000","last_fullsized":"2024-03-14T12:13:51.576826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:31.510461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.575144+0000","last_change":"2024-03-14T12:13:50.571689+0000","last_active":"2024-03-14T12:13:51.575144+0000","last_peered":"2024-03-14T12:13:51.575144+0000","last_clean":"2024-03-14T12:13:51.575144+0000","last_became_active":"2024-03-14T12:13:50.571292+0000","last_became_peered":"2024-03-14T12:13:50.571292+0000","last_unstale":"2024-03-14T12:13:51.575144+0000","last_undegraded":"2024-03-14T12:13:51.575144+0000","last_fullsized":"2024-03-14T12:13:51.575144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777395+0000","last_change":"2024-03-14T12:13:51.582845+0000","last_active":"2024-03-14T12:13:52.777395+0000","last_peered":"2024-03-14T12:13:52.777395+0000","last_clean":"2024-03-14T12:13:52.777395+0000","last_became_active":"2024-03-14T12:13:51.582580+0000","last_became_peered":"2024-03-14T12:13:51.582580+0000","last_unstale":"2024-03-14T12:13:52.777395+0000","last_undegraded":"2024-03-14T12:13:52.777395+0000","last_fullsized":"2024-03-14T12:13:52.777395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.570680+0000","last_change":"2024-03-14T12:13:50.568041+0000","last_active":"2024-03-14T12:13:51.570680+0000","last_peered":"2024-03-14T12:13:51.570680+0000","last_clean":"2024-03-14T12:13:51.570680+0000","last_became_active":"2024-03-14T12:13:50.567698+0000","last_became_peered":"2024-03-14T12:13:50.567698+0000","last_unstale":"2024-03-14T12:13:51.570680+0000","last_undegraded":"2024-03-14T12:13:51.570680+0000","last_fullsized":"2024-03-14T12:13:51.570680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573917+0000","last_change":"2024-03-14T12:13:51.572247+0000","last_active":"2024-03-14T12:13:52.573917+0000","last_peered":"2024-03-14T12:13:52.573917+0000","last_clean":"2024-03-14T12:13:52.573917+0000","last_became_active":"2024-03-14T12:13:51.572000+0000","last_became_peered":"2024-03-14T12:13:51.572000+0000","last_unstale":"2024-03-14T12:13:52.573917+0000","last_undegraded":"2024-03-14T12:13:52.573917+0000","last_fullsized":"2024-03-14T12:13:52.573917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580774+0000","last_change":"2024-03-14T12:13:50.562695+0000","last_active":"2024-03-14T12:13:52.580774+0000","last_peered":"2024-03-14T12:13:52.580774+0000","last_clean":"2024-03-14T12:13:52.580774+0000","last_became_active":"2024-03-14T12:13:50.562419+0000","last_became_peered":"2024-03-14T12:13:50.562419+0000","last_unstale":"2024-03-14T12:13:52.580774+0000","last_undegraded":"2024-03-14T12:13:52.580774+0000","last_fullsized":"2024-03-14T12:13:52.580774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580029+0000","last_change":"2024-03-14T12:13:51.578692+0000","last_active":"2024-03-14T12:13:52.580029+0000","last_peered":"2024-03-14T12:13:52.580029+0000","last_clean":"2024-03-14T12:13:52.580029+0000","last_became_active":"2024-03-14T12:13:51.578375+0000","last_became_peered":"2024-03-14T12:13:51.578375+0000","last_unstale":"2024-03-14T12:13:52.580029+0000","last_undegraded":"2024-03-14T12:13:52.580029+0000","last_fullsized":"2024-03-14T12:13:52.580029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573848+0000","last_change":"2024-03-14T12:13:50.568583+0000","last_active":"2024-03-14T12:13:52.573848+0000","last_peered":"2024-03-14T12:13:52.573848+0000","last_clean":"2024-03-14T12:13:52.573848+0000","last_became_active":"2024-03-14T12:13:50.568267+0000","last_became_peered":"2024-03-14T12:13:50.568267+0000","last_unstale":"2024-03-14T12:13:52.573848+0000","last_undegraded":"2024-03-14T12:13:52.573848+0000","last_fullsized":"2024-03-14T12:13:52.573848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580655+0000","last_change":"2024-03-14T12:13:51.579205+0000","last_active":"2024-03-14T12:13:52.580655+0000","last_peered":"2024-03-14T12:13:52.580655+0000","last_clean":"2024-03-14T12:13:52.580655+0000","last_became_active":"2024-03-14T12:13:51.578901+0000","last_became_peered":"2024-03-14T12:13:51.578901+0000","last_unstale":"2024-03-14T12:13:52.580655+0000","last_undegraded":"2024-03-14T12:13:52.580655+0000","last_fullsized":"2024-03-14T12:13:52.580655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.577780+0000","last_change":"2024-03-14T12:13:50.571020+0000","last_active":"2024-03-14T12:13:51.577780+0000","last_peered":"2024-03-14T12:13:51.577780+0000","last_clean":"2024-03-14T12:13:51.577780+0000","last_became_active":"2024-03-14T12:13:50.570744+0000","last_became_peered":"2024-03-14T12:13:50.570744+0000","last_unstale":"2024-03-14T12:13:51.577780+0000","last_undegraded":"2024-03-14T12:13:51.577780+0000","last_fullsized":"2024-03-14T12:13:51.577780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573864+0000","last_change":"2024-03-14T12:13:51.574187+0000","last_active":"2024-03-14T12:13:52.573864+0000","last_peered":"2024-03-14T12:13:52.573864+0000","last_clean":"2024-03-14T12:13:52.573864+0000","last_became_active":"2024-03-14T12:13:51.573916+0000","last_became_peered":"2024-03-14T12:13:51.573916+0000","last_unstale":"2024-03-14T12:13:52.573864+0000","last_undegraded":"2024-03-14T12:13:52.573864+0000","last_fullsized":"2024-03-14T12:13:52.573864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580829+0000","last_change":"2024-03-14T12:13:50.569678+0000","last_active":"2024-03-14T12:13:52.580829+0000","last_peered":"2024-03-14T12:13:52.580829+0000","last_clean":"2024-03-14T12:13:52.580829+0000","last_became_active":"2024-03-14T12:13:50.569377+0000","last_became_peered":"2024-03-14T12:13:50.569377+0000","last_unstale":"2024-03-14T12:13:52.580829+0000","last_undegraded":"2024-03-14T12:13:52.580829+0000","last_fullsized":"2024-03-14T12:13:52.580829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.582019+0000","last_change":"2024-03-14T12:13:51.579167+0000","last_active":"2024-03-14T12:13:51.582019+0000","last_peered":"2024-03-14T12:13:51.582019+0000","last_clean":"2024-03-14T12:13:51.582019+0000","last_became_active":"2024-03-14T12:13:51.578971+0000","last_became_peered":"2024-03-14T12:13:51.578971+0000","last_unstale":"2024-03-14T12:13:51.582019+0000","last_undegraded":"2024-03-14T12:13:51.582019+0000","last_fullsized":"2024-03-14T12:13:51.582019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582080+0000","last_change":"2024-03-14T12:13:50.570044+0000","last_active":"2024-03-14T12:13:52.582080+0000","last_peered":"2024-03-14T12:13:52.582080+0000","last_clean":"2024-03-14T12:13:52.582080+0000","last_became_active":"2024-03-14T12:13:50.569759+0000","last_became_peered":"2024-03-14T12:13:50.569759+0000","last_unstale":"2024-03-14T12:13:52.582080+0000","last_undegraded":"2024-03-14T12:13:52.582080+0000","last_fullsized":"2024-03-14T12:13:52.582080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777595+0000","last_change":"2024-03-14T12:13:51.581169+0000","last_active":"2024-03-14T12:13:52.777595+0000","last_peered":"2024-03-14T12:13:52.777595+0000","last_clean":"2024-03-14T12:13:52.777595+0000","last_became_active":"2024-03-14T12:13:51.580757+0000","last_became_peered":"2024-03-14T12:13:51.580757+0000","last_unstale":"2024-03-14T12:13:52.777595+0000","last_undegraded":"2024-03-14T12:13:52.777595+0000","last_fullsized":"2024-03-14T12:13:52.777595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.574249+0000","last_change":"2024-03-14T12:13:50.571439+0000","last_active":"2024-03-14T12:13:51.574249+0000","last_peered":"2024-03-14T12:13:51.574249+0000","last_clean":"2024-03-14T12:13:51.574249+0000","last_became_active":"2024-03-14T12:13:50.571184+0000","last_became_peered":"2024-03-14T12:13:50.571184+0000","last_unstale":"2024-03-14T12:13:51.574249+0000","last_undegraded":"2024-03-14T12:13:51.574249+0000","last_fullsized":"2024-03-14T12:13:51.574249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579859+0000","last_change":"2024-03-14T12:13:51.573261+0000","last_active":"2024-03-14T12:13:52.579859+0000","last_peered":"2024-03-14T12:13:52.579859+0000","last_clean":"2024-03-14T12:13:52.579859+0000","last_became_active":"2024-03-14T12:13:51.572866+0000","last_became_peered":"2024-03-14T12:13:51.572866+0000","last_unstale":"2024-03-14T12:13:52.579859+0000","last_undegraded":"2024-03-14T12:13:52.579859+0000","last_fullsized":"2024-03-14T12:13:52.579859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:39.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573916+0000","last_change":"2024-03-14T12:13:50.570820+0000","last_active":"2024-03-14T12:13:52.573916+0000","last_peered":"2024-03-14T12:13:52.573916+0000","last_clean":"2024-03-14T12:13:52.573916+0000","last_became_active":"2024-03-14T12:13:50.570423+0000","last_became_peered":"2024-03-14T12:13:50.570423+0000","last_unstale":"2024-03-14T12:13:52.573916+0000","last_undegraded":"2024-03-14T12:13:52.573916+0000","last_fullsized":"2024-03-14T12:13:52.573916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.576872+0000","last_change":"2024-03-14T12:13:51.575973+0000","last_active":"2024-03-14T12:13:51.576872+0000","last_peered":"2024-03-14T12:13:51.576872+0000","last_clean":"2024-03-14T12:13:51.576872+0000","last_became_active":"2024-03-14T12:13:51.575695+0000","last_became_peered":"2024-03-14T12:13:51.575695+0000","last_unstale":"2024-03-14T12:13:51.576872+0000","last_undegraded":"2024-03-14T12:13:51.576872+0000","last_fullsized":"2024-03-14T12:13:51.576872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:22.285709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573877+0000","last_change":"2024-03-14T12:13:50.565398+0000","last_active":"2024-03-14T12:13:52.573877+0000","last_peered":"2024-03-14T12:13:52.573877+0000","last_clean":"2024-03-14T12:13:52.573877+0000","last_became_active":"2024-03-14T12:13:50.565109+0000","last_became_peered":"2024-03-14T12:13:50.565109+0000","last_unstale":"2024-03-14T12:13:52.573877+0000","last_undegraded":"2024-03-14T12:13:52.573877+0000","last_fullsized":"2024-03-14T12:13:52.573877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.578089+0000","last_change":"2024-03-14T12:13:51.576916+0000","last_active":"2024-03-14T12:13:51.578089+0000","last_peered":"2024-03-14T12:13:51.578089+0000","last_clean":"2024-03-14T12:13:51.578089+0000","last_became_active":"2024-03-14T12:13:51.576639+0000","last_became_peered":"2024-03-14T12:13:51.576639+0000","last_unstale":"2024-03-14T12:13:51.578089+0000","last_undegraded":"2024-03-14T12:13:51.578089+0000","last_fullsized":"2024-03-14T12:13:51.578089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:55.327329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596200+0000","last_change":"2024-03-14T12:13:50.575585+0000","last_active":"2024-03-14T12:13:51.596200+0000","last_peered":"2024-03-14T12:13:51.596200+0000","last_clean":"2024-03-14T12:13:51.596200+0000","last_became_active":"2024-03-14T12:13:50.575288+0000","last_became_peered":"2024-03-14T12:13:50.575288+0000","last_unstale":"2024-03-14T12:13:51.596200+0000","last_undegraded":"2024-03-14T12:13:51.596200+0000","last_fullsized":"2024-03-14T12:13:51.596200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581882+0000","last_change":"2024-03-14T12:13:51.588022+0000","last_active":"2024-03-14T12:13:52.581882+0000","last_peered":"2024-03-14T12:13:52.581882+0000","last_clean":"2024-03-14T12:13:52.581882+0000","last_became_active":"2024-03-14T12:13:51.587766+0000","last_became_peered":"2024-03-14T12:13:51.587766+0000","last_unstale":"2024-03-14T12:13:52.581882+0000","last_undegraded":"2024-03-14T12:13:52.581882+0000","last_fullsized":"2024-03-14T12:13:52.581882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580894+0000","last_change":"2024-03-14T12:13:51.572504+0000","last_active":"2024-03-14T12:13:52.580894+0000","last_peered":"2024-03-14T12:13:52.580894+0000","last_clean":"2024-03-14T12:13:52.580894+0000","last_became_active":"2024-03-14T12:13:51.572219+0000","last_became_peered":"2024-03-14T12:13:51.572219+0000","last_unstale":"2024-03-14T12:13:52.580894+0000","last_undegraded":"2024-03-14T12:13:52.580894+0000","last_fullsized":"2024-03-14T12:13:52.580894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.595216+0000","last_change":"2024-03-14T12:13:50.572648+0000","last_active":"2024-03-14T12:13:51.595216+0000","last_peered":"2024-03-14T12:13:51.595216+0000","last_clean":"2024-03-14T12:13:51.595216+0000","last_became_active":"2024-03-14T12:13:50.572409+0000","last_became_peered":"2024-03-14T12:13:50.572409+0000","last_unstale":"2024-03-14T12:13:51.595216+0000","last_undegraded":"2024-03-14T12:13:51.595216+0000","last_fullsized":"2024-03-14T12:13:51.595216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579831+0000","last_change":"2024-03-14T12:13:51.580823+0000","last_active":"2024-03-14T12:13:52.579831+0000","last_peered":"2024-03-14T12:13:52.579831+0000","last_clean":"2024-03-14T12:13:52.579831+0000","last_became_active":"2024-03-14T12:13:51.574625+0000","last_became_peered":"2024-03-14T12:13:51.574625+0000","last_unstale":"2024-03-14T12:13:52.579831+0000","last_undegraded":"2024-03-14T12:13:52.579831+0000","last_fullsized":"2024-03-14T12:13:52.579831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.596356+0000","last_change":"2024-03-14T12:13:50.572321+0000","last_active":"2024-03-14T12:13:51.596356+0000","last_peered":"2024-03-14T12:13:51.596356+0000","last_clean":"2024-03-14T12:13:51.596356+0000","last_became_active":"2024-03-14T12:13:50.572001+0000","last_became_peered":"2024-03-14T12:13:50.572001+0000","last_unstale":"2024-03-14T12:13:51.596356+0000","last_undegraded":"2024-03-14T12:13:51.596356+0000","last_fullsized":"2024-03-14T12:13:51.596356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777630+0000","last_change":"2024-03-14T12:13:51.575693+0000","last_active":"2024-03-14T12:13:52.777630+0000","last_peered":"2024-03-14T12:13:52.777630+0000","last_clean":"2024-03-14T12:13:52.777630+0000","last_became_active":"2024-03-14T12:13:51.574493+0000","last_became_peered":"2024-03-14T12:13:51.574493+0000","last_unstale":"2024-03-14T12:13:52.777630+0000","last_undegraded":"2024-03-14T12:13:52.777630+0000","last_fullsized":"2024-03-14T12:13:52.777630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.592745+0000","last_change":"2024-03-14T12:13:50.565913+0000","last_active":"2024-03-14T12:13:51.592745+0000","last_peered":"2024-03-14T12:13:51.592745+0000","last_clean":"2024-03-14T12:13:51.592745+0000","last_became_active":"2024-03-14T12:13:50.565652+0000","last_became_peered":"2024-03-14T12:13:50.565652+0000","last_unstale":"2024-03-14T12:13:51.592745+0000","last_undegraded":"2024-03-14T12:13:51.592745+0000","last_fullsized":"2024-03-14T12:13:51.592745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T12:13:51.582658+0000","last_change":"2024-03-14T12:13:51.581638+0000","last_active":"2024-03-14T12:13:51.582658+0000","last_peered":"2024-03-14T12:13:51.582658+0000","last_clean":"2024-03-14T12:13:51.582658+0000","last_became_active":"2024-03-14T12:13:51.581373+0000","last_became_peered":"2024-03-14T12:13:51.581373+0000","last_unstale":"2024-03-14T12:13:51.582658+0000","last_undegraded":"2024-03-14T12:13:51.582658+0000","last_fullsized":"2024-03-14T12:13:51.582658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777698+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:52.777698+0000","last_peered":"2024-03-14T12:13:52.777698+0000","last_clean":"2024-03-14T12:13:52.777698+0000","last_became_active":"2024-03-14T12:13:50.569816+0000","last_became_peered":"2024-03-14T12:13:50.569816+0000","last_unstale":"2024-03-14T12:13:52.777698+0000","last_undegraded":"2024-03-14T12:13:52.777698+0000","last_fullsized":"2024-03-14T12:13:52.777698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:13:59.466 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:13:59.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.579+0000 7f4287e7e640 1 Processor -- start 2024-03-14T12:13:59.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.579+0000 7f4287e7e640 1 -- start start 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.580+0000 7f4287e7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42800b4220 0x7f4280161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.580+0000 7f4287e7e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428006fa90 con 0x7f4280162010 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.580+0000 7f4287e7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428006be20 con 0x7f428015e150 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.580+0000 7f4287e7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428006c750 con 0x7f42800b4220 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.580+0000 7f428641b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4280162010 0x7f428006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34532/0 (socket says 172.21.15.82:34532) 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.580+0000 7f428641b640 1 -- 172.21.15.82:0/20281395 learned_addr learned my addr 172.21.15.82:0/20281395 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.580+0000 7f4285419640 1 --2- 172.21.15.82:0/20281395 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42800b4220 0x7f4280161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552200839 0 0) 0x7f428006fa90 con 0x7f4280162010 2024-03-14T12:13:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f426c003520 con 0x7f4280162010 2024-03-14T12:13:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661499699 0 0) 0x7f428006be20 con 0x7f428015e150 2024-03-14T12:13:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f428006fa90 con 0x7f428015e150 2024-03-14T12:13:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805496775 0 0) 0x7f426c003520 con 0x7f4280162010 2024-03-14T12:13:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4285419640 1 -- 172.21.15.82:0/20281395 >> v1:172.21.15.164:6789/0 conn(0x7f428015e150 legacy=0x7f428006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4285419640 1 -- 172.21.15.82:0/20281395 >> v1:172.21.15.82:6789/0 conn(0x7f4280162010 legacy=0x7f428006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.581+0000 7f4285419640 1 -- 172.21.15.82:0/20281395 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428006c100 con 0x7f42800b4220 2024-03-14T12:13:59.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.582+0000 7f4285419640 1 --2- 172.21.15.82:0/20281395 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42800b4220 0x7f4280161ad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42700097d0 tx=0x7f427002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1026cff587a30faf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.582+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270037070 con 0x7f42800b4220 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.582+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4270030510 con 0x7f42800b4220 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.582+0000 7f4284c18640 1 -- 172.21.15.82:0/20281395 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780019f0 con 0x7f42800b4220 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.582+0000 7f4287e7e640 1 -- 172.21.15.82:0/20281395 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42800b4220 msgr2=0x7f4280161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.582+0000 7f4287e7e640 1 --2- 172.21.15.82:0/20281395 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42800b4220 0x7f4280161ad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42700097d0 tx=0x7f427002b800 comp rx=0 tx=0).stop 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.583+0000 7f4287e7e640 1 -- 172.21.15.82:0/20281395 shutdown_connections 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.583+0000 7f4287e7e640 1 --2- 172.21.15.82:0/20281395 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42800b4220 0x7f4280161ad0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.583+0000 7f4287e7e640 1 -- 172.21.15.82:0/20281395 >> 172.21.15.82:0/20281395 conn(0x7f42800b3670 msgr2=0x7f42800b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.583+0000 7f4287e7e640 1 -- 172.21.15.82:0/20281395 shutdown_connections 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.583+0000 7f4287e7e640 1 -- 172.21.15.82:0/20281395 wait complete. 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.584+0000 7f4287e7e640 1 Processor -- start 2024-03-14T12:13:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.584+0000 7f4287e7e640 1 -- start start 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42800b4220 0x7f42800ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428015e150 0x7f42800c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4280162010 0x7f42800b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428006d470 con 0x7f428015e150 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428006d2f0 con 0x7f42800b4220 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428006d5f0 con 0x7f4280162010 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428015e150 0x7f42800c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428015e150 0x7f42800c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48588/0 (socket says 172.21.15.82:48588) 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 -- 172.21.15.82:0/436136402 learned_addr learned my addr 172.21.15.82:0/436136402 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285c1a640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42800b4220 0x7f42800ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f428641b640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4280162010 0x7f42800b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 -- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4280162010 msgr2=0x7f42800b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4280162010 0x7f42800b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 -- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42800b4220 msgr2=0x7f42800ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42800b4220 0x7f42800ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 -- 172.21.15.82:0/436136402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42800ced30 con 0x7f428015e150 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f428641b640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4280162010 0x7f42800b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4285419640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428015e150 0x7f42800c7e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f427002b840 tx=0x7f42700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f42767fc640 1 -- 172.21.15.82:0/436136402 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270041020 con 0x7f428015e150 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f42767fc640 1 -- 172.21.15.82:0/436136402 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427003c870 con 0x7f428015e150 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42800c6d80 con 0x7f428015e150 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f42767fc640 1 -- 172.21.15.82:0/436136402 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270037040 con 0x7f428015e150 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.585+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42800c7340 con 0x7f428015e150 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.586+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428004f050 con 0x7f428015e150 2024-03-14T12:13:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.587+0000 7f42767fc640 1 -- 172.21.15.82:0/436136402 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4270044020 con 0x7f428015e150 2024-03-14T12:13:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.587+0000 7f42767fc640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f426406bbd0 0x7f426406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.587+0000 7f42767fc640 1 -- 172.21.15.82:0/436136402 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42700ac3f0 con 0x7f428015e150 2024-03-14T12:13:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.587+0000 7f4285c1a640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f426406bbd0 0x7f426406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.588+0000 7f4285c1a640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f426406bbd0 0x7f426406e090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f427c0011e0 tx=0x7f427c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.591+0000 7f42767fc640 1 -- 172.21.15.82:0/436136402 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f427004b030 con 0x7f428015e150 2024-03-14T12:13:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.761+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42800b7ae0 con 0x7f426406bbd0 2024-03-14T12:13:59.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.767+0000 7f42767fc640 1 -- 172.21.15.82:0/436136402 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251860 (secure 0 0 0) 0x7f42800b7ae0 con 0x7f426406bbd0 2024-03-14T12:13:59.771 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:13:59.774 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f426406bbd0 msgr2=0x7f426406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f426406bbd0 0x7f426406e090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f427c0011e0 tx=0x7f427c009040 comp rx=0 tx=0).stop 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428015e150 msgr2=0x7f42800c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428015e150 0x7f42800c7e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f427002b840 tx=0x7f42700046c0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 shutdown_connections 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f426406bbd0 0x7f426406e090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4280162010 0x7f42800b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428015e150 0x7f42800c7e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 --2- 172.21.15.82:0/436136402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42800b4220 0x7f42800ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 >> 172.21.15.82:0/436136402 conn(0x7f42800b3670 msgr2=0x7f42800b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.770+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 shutdown_connections 2024-03-14T12:13:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.771+0000 7f4287e7e640 1 -- 172.21.15.82:0/436136402 wait complete. 2024-03-14T12:13:59.781 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-03-14T12:13:59.499561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001431"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478859+0000","last_change":"2024-03-14T12:13:51.582275+0000","last_active":"2024-03-14T12:13:54.478859+0000","last_peered":"2024-03-14T12:13:54.478859+0000","last_clean":"2024-03-14T12:13:54.478859+0000","last_became_active":"2024-03-14T12:13:51.581999+0000","last_became_peered":"2024-03-14T12:13:51.581999+0000","last_unstale":"2024-03-14T12:13:54.478859+0000","last_undegraded":"2024-03-14T12:13:54.478859+0000","last_fullsized":"2024-03-14T12:13:54.478859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574007+0000","last_change":"2024-03-14T12:13:51.575382+0000","last_active":"2024-03-14T12:13:52.574007+0000","last_peered":"2024-03-14T12:13:52.574007+0000","last_clean":"2024-03-14T12:13:52.574007+0000","last_became_active":"2024-03-14T12:13:51.575139+0000","last_became_peered":"2024-03-14T12:13:51.575139+0000","last_unstale":"2024-03-14T12:13:52.574007+0000","last_undegraded":"2024-03-14T12:13:52.574007+0000","last_fullsized":"2024-03-14T12:13:52.574007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777473+0000","last_change":"2024-03-14T12:13:51.574924+0000","last_active":"2024-03-14T12:13:52.777473+0000","last_peered":"2024-03-14T12:13:52.777473+0000","last_clean":"2024-03-14T12:13:52.777473+0000","last_became_active":"2024-03-14T12:13:51.574299+0000","last_became_peered":"2024-03-14T12:13:51.574299+0000","last_unstale":"2024-03-14T12:13:52.777473+0000","last_undegraded":"2024-03-14T12:13:52.777473+0000","last_fullsized":"2024-03-14T12:13:52.777473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579676+0000","last_change":"2024-03-14T12:13:51.581446+0000","last_active":"2024-03-14T12:13:52.579676+0000","last_peered":"2024-03-14T12:13:52.579676+0000","last_clean":"2024-03-14T12:13:52.579676+0000","last_became_active":"2024-03-14T12:13:51.581192+0000","last_became_peered":"2024-03-14T12:13:51.581192+0000","last_unstale":"2024-03-14T12:13:52.579676+0000","last_undegraded":"2024-03-14T12:13:52.579676+0000","last_fullsized":"2024-03-14T12:13:52.579676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478459+0000","last_change":"2024-03-14T12:13:51.587466+0000","last_active":"2024-03-14T12:13:54.478459+0000","last_peered":"2024-03-14T12:13:54.478459+0000","last_clean":"2024-03-14T12:13:54.478459+0000","last_became_active":"2024-03-14T12:13:51.585375+0000","last_became_peered":"2024-03-14T12:13:51.585375+0000","last_unstale":"2024-03-14T12:13:54.478459+0000","last_undegraded":"2024-03-14T12:13:54.478459+0000","last_fullsized":"2024-03-14T12:13:54.478459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580498+0000","last_change":"2024-03-14T12:13:51.580794+0000","last_active":"2024-03-14T12:13:52.580498+0000","last_peered":"2024-03-14T12:13:52.580498+0000","last_clean":"2024-03-14T12:13:52.580498+0000","last_became_active":"2024-03-14T12:13:51.580511+0000","last_became_peered":"2024-03-14T12:13:51.580511+0000","last_unstale":"2024-03-14T12:13:52.580498+0000","last_undegraded":"2024-03-14T12:13:52.580498+0000","last_fullsized":"2024-03-14T12:13:52.580498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478351+0000","last_change":"2024-03-14T12:13:51.588949+0000","last_active":"2024-03-14T12:13:54.478351+0000","last_peered":"2024-03-14T12:13:54.478351+0000","last_clean":"2024-03-14T12:13:54.478351+0000","last_became_active":"2024-03-14T12:13:51.588671+0000","last_became_peered":"2024-03-14T12:13:51.588671+0000","last_unstale":"2024-03-14T12:13:54.478351+0000","last_undegraded":"2024-03-14T12:13:54.478351+0000","last_fullsized":"2024-03-14T12:13:54.478351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477849+0000","last_change":"2024-03-14T12:13:51.579815+0000","last_active":"2024-03-14T12:13:54.477849+0000","last_peered":"2024-03-14T12:13:54.477849+0000","last_clean":"2024-03-14T12:13:54.477849+0000","last_became_active":"2024-03-14T12:13:51.579580+0000","last_became_peered":"2024-03-14T12:13:51.579580+0000","last_unstale":"2024-03-14T12:13:54.477849+0000","last_undegraded":"2024-03-14T12:13:54.477849+0000","last_fullsized":"2024-03-14T12:13:54.477849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478866+0000","last_change":"2024-03-14T12:13:51.593442+0000","last_active":"2024-03-14T12:13:54.478866+0000","last_peered":"2024-03-14T12:13:54.478866+0000","last_clean":"2024-03-14T12:13:54.478866+0000","last_became_active":"2024-03-14T12:13:51.593151+0000","last_became_peered":"2024-03-14T12:13:51.593151+0000","last_unstale":"2024-03-14T12:13:54.478866+0000","last_undegraded":"2024-03-14T12:13:54.478866+0000","last_fullsized":"2024-03-14T12:13:54.478866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582259+0000","last_change":"2024-03-14T12:13:51.584838+0000","last_active":"2024-03-14T12:13:52.582259+0000","last_peered":"2024-03-14T12:13:52.582259+0000","last_clean":"2024-03-14T12:13:52.582259+0000","last_became_active":"2024-03-14T12:13:51.584501+0000","last_became_peered":"2024-03-14T12:13:51.584501+0000","last_unstale":"2024-03-14T12:13:52.582259+0000","last_undegraded":"2024-03-14T12:13:52.582259+0000","last_fullsized":"2024-03-14T12:13:52.582259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581960+0000","last_change":"2024-03-14T12:13:51.584374+0000","last_active":"2024-03-14T12:13:52.581960+0000","last_peered":"2024-03-14T12:13:52.581960+0000","last_clean":"2024-03-14T12:13:52.581960+0000","last_became_active":"2024-03-14T12:13:51.584094+0000","last_became_peered":"2024-03-14T12:13:51.584094+0000","last_unstale":"2024-03-14T12:13:52.581960+0000","last_undegraded":"2024-03-14T12:13:52.581960+0000","last_fullsized":"2024-03-14T12:13:52.581960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580082+0000","last_change":"2024-03-14T12:13:51.583802+0000","last_active":"2024-03-14T12:13:52.580082+0000","last_peered":"2024-03-14T12:13:52.580082+0000","last_clean":"2024-03-14T12:13:52.580082+0000","last_became_active":"2024-03-14T12:13:51.583284+0000","last_became_peered":"2024-03-14T12:13:51.583284+0000","last_unstale":"2024-03-14T12:13:52.580082+0000","last_undegraded":"2024-03-14T12:13:52.580082+0000","last_fullsized":"2024-03-14T12:13:52.580082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478086+0000","last_change":"2024-03-14T12:13:51.578671+0000","last_active":"2024-03-14T12:13:54.478086+0000","last_peered":"2024-03-14T12:13:54.478086+0000","last_clean":"2024-03-14T12:13:54.478086+0000","last_became_active":"2024-03-14T12:13:51.578427+0000","last_became_peered":"2024-03-14T12:13:51.578427+0000","last_unstale":"2024-03-14T12:13:54.478086+0000","last_undegraded":"2024-03-14T12:13:54.478086+0000","last_fullsized":"2024-03-14T12:13:54.478086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777400+0000","last_change":"2024-03-14T12:13:51.581289+0000","last_active":"2024-03-14T12:13:52.777400+0000","last_peered":"2024-03-14T12:13:52.777400+0000","last_clean":"2024-03-14T12:13:52.777400+0000","last_became_active":"2024-03-14T12:13:51.580645+0000","last_became_peered":"2024-03-14T12:13:51.580645+0000","last_unstale":"2024-03-14T12:13:52.777400+0000","last_undegraded":"2024-03-14T12:13:52.777400+0000","last_fullsized":"2024-03-14T12:13:52.777400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:01.290201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573796+0000","last_change":"2024-03-14T12:13:51.571032+0000","last_active":"2024-03-14T12:13:52.573796+0000","last_peered":"2024-03-14T12:13:52.573796+0000","last_clean":"2024-03-14T12:13:52.573796+0000","last_became_active":"2024-03-14T12:13:51.570723+0000","last_became_peered":"2024-03-14T12:13:51.570723+0000","last_unstale":"2024-03-14T12:13:52.573796+0000","last_undegraded":"2024-03-14T12:13:52.573796+0000","last_fullsized":"2024-03-14T12:13:52.573796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580831+0000","last_change":"2024-03-14T12:13:51.582495+0000","last_active":"2024-03-14T12:13:52.580831+0000","last_peered":"2024-03-14T12:13:52.580831+0000","last_clean":"2024-03-14T12:13:52.580831+0000","last_became_active":"2024-03-14T12:13:51.582170+0000","last_became_peered":"2024-03-14T12:13:51.582170+0000","last_unstale":"2024-03-14T12:13:52.580831+0000","last_undegraded":"2024-03-14T12:13:52.580831+0000","last_fullsized":"2024-03-14T12:13:52.580831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580861+0000","last_change":"2024-03-14T12:13:51.582330+0000","last_active":"2024-03-14T12:13:52.580861+0000","last_peered":"2024-03-14T12:13:52.580861+0000","last_clean":"2024-03-14T12:13:52.580861+0000","last_became_active":"2024-03-14T12:13:51.581920+0000","last_became_peered":"2024-03-14T12:13:51.581920+0000","last_unstale":"2024-03-14T12:13:52.580861+0000","last_undegraded":"2024-03-14T12:13:52.580861+0000","last_fullsized":"2024-03-14T12:13:52.580861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477706+0000","last_change":"2024-03-14T12:13:51.582708+0000","last_active":"2024-03-14T12:13:54.477706+0000","last_peered":"2024-03-14T12:13:54.477706+0000","last_clean":"2024-03-14T12:13:54.477706+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:54.477706+0000","last_undegraded":"2024-03-14T12:13:54.477706+0000","last_fullsized":"2024-03-14T12:13:54.477706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.479134+0000","last_change":"2024-03-14T12:13:51.587758+0000","last_active":"2024-03-14T12:13:54.479134+0000","last_peered":"2024-03-14T12:13:54.479134+0000","last_clean":"2024-03-14T12:13:54.479134+0000","last_became_active":"2024-03-14T12:13:51.586310+0000","last_became_peered":"2024-03-14T12:13:51.586310+0000","last_unstale":"2024-03-14T12:13:54.479134+0000","last_undegraded":"2024-03-14T12:13:54.479134+0000","last_fullsized":"2024-03-14T12:13:54.479134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:57:08.624764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582057+0000","last_change":"2024-03-14T12:13:51.589443+0000","last_active":"2024-03-14T12:13:52.582057+0000","last_peered":"2024-03-14T12:13:52.582057+0000","last_clean":"2024-03-14T12:13:52.582057+0000","last_became_active":"2024-03-14T12:13:51.589174+0000","last_became_peered":"2024-03-14T12:13:51.589174+0000","last_unstale":"2024-03-14T12:13:52.582057+0000","last_undegraded":"2024-03-14T12:13:52.582057+0000","last_fullsized":"2024-03-14T12:13:52.582057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777419+0000","last_change":"2024-03-14T12:13:51.580813+0000","last_active":"2024-03-14T12:13:52.777419+0000","last_peered":"2024-03-14T12:13:52.777419+0000","last_clean":"2024-03-14T12:13:52.777419+0000","last_became_active":"2024-03-14T12:13:51.580334+0000","last_became_peered":"2024-03-14T12:13:51.580334+0000","last_unstale":"2024-03-14T12:13:52.777419+0000","last_undegraded":"2024-03-14T12:13:52.777419+0000","last_fullsized":"2024-03-14T12:13:52.777419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580443+0000","last_change":"2024-03-14T12:13:51.583228+0000","last_active":"2024-03-14T12:13:52.580443+0000","last_peered":"2024-03-14T12:13:52.580443+0000","last_clean":"2024-03-14T12:13:52.580443+0000","last_became_active":"2024-03-14T12:13:51.582937+0000","last_became_peered":"2024-03-14T12:13:51.582937+0000","last_unstale":"2024-03-14T12:13:52.580443+0000","last_undegraded":"2024-03-14T12:13:52.580443+0000","last_fullsized":"2024-03-14T12:13:52.580443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477840+0000","last_change":"2024-03-14T12:13:51.582656+0000","last_active":"2024-03-14T12:13:54.477840+0000","last_peered":"2024-03-14T12:13:54.477840+0000","last_clean":"2024-03-14T12:13:54.477840+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:54.477840+0000","last_undegraded":"2024-03-14T12:13:54.477840+0000","last_fullsized":"2024-03-14T12:13:54.477840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580802+0000","last_change":"2024-03-14T12:13:51.579810+0000","last_active":"2024-03-14T12:13:52.580802+0000","last_peered":"2024-03-14T12:13:52.580802+0000","last_clean":"2024-03-14T12:13:52.580802+0000","last_became_active":"2024-03-14T12:13:51.579523+0000","last_became_peered":"2024-03-14T12:13:51.579523+0000","last_unstale":"2024-03-14T12:13:52.580802+0000","last_undegraded":"2024-03-14T12:13:52.580802+0000","last_fullsized":"2024-03-14T12:13:52.580802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478115+0000","last_change":"2024-03-14T12:13:51.582442+0000","last_active":"2024-03-14T12:13:54.478115+0000","last_peered":"2024-03-14T12:13:54.478115+0000","last_clean":"2024-03-14T12:13:54.478115+0000","last_became_active":"2024-03-14T12:13:51.582175+0000","last_became_peered":"2024-03-14T12:13:51.582175+0000","last_unstale":"2024-03-14T12:13:54.478115+0000","last_undegraded":"2024-03-14T12:13:54.478115+0000","last_fullsized":"2024-03-14T12:13:54.478115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582355+0000","last_change":"2024-03-14T12:13:51.590516+0000","last_active":"2024-03-14T12:13:52.582355+0000","last_peered":"2024-03-14T12:13:52.582355+0000","last_clean":"2024-03-14T12:13:52.582355+0000","last_became_active":"2024-03-14T12:13:51.590223+0000","last_became_peered":"2024-03-14T12:13:51.590223+0000","last_unstale":"2024-03-14T12:13:52.582355+0000","last_undegraded":"2024-03-14T12:13:52.582355+0000","last_fullsized":"2024-03-14T12:13:52.582355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573995+0000","last_change":"2024-03-14T12:13:51.574721+0000","last_active":"2024-03-14T12:13:52.573995+0000","last_peered":"2024-03-14T12:13:52.573995+0000","last_clean":"2024-03-14T12:13:52.573995+0000","last_became_active":"2024-03-14T12:13:51.574443+0000","last_became_peered":"2024-03-14T12:13:51.574443+0000","last_unstale":"2024-03-14T12:13:52.573995+0000","last_undegraded":"2024-03-14T12:13:52.573995+0000","last_fullsized":"2024-03-14T12:13:52.573995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478558+0000","last_change":"2024-03-14T12:13:51.582640+0000","last_active":"2024-03-14T12:13:54.478558+0000","last_peered":"2024-03-14T12:13:54.478558+0000","last_clean":"2024-03-14T12:13:54.478558+0000","last_became_active":"2024-03-14T12:13:51.582355+0000","last_became_peered":"2024-03-14T12:13:51.582355+0000","last_unstale":"2024-03-14T12:13:54.478558+0000","last_undegraded":"2024-03-14T12:13:54.478558+0000","last_fullsized":"2024-03-14T12:13:54.478558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777477+0000","last_change":"2024-03-14T12:13:51.577794+0000","last_active":"2024-03-14T12:13:52.777477+0000","last_peered":"2024-03-14T12:13:52.777477+0000","last_clean":"2024-03-14T12:13:52.777477+0000","last_became_active":"2024-03-14T12:13:51.577518+0000","last_became_peered":"2024-03-14T12:13:51.577518+0000","last_unstale":"2024-03-14T12:13:52.777477+0000","last_undegraded":"2024-03-14T12:13:52.777477+0000","last_fullsized":"2024-03-14T12:13:52.777477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580629+0000","last_change":"2024-03-14T12:13:51.580193+0000","last_active":"2024-03-14T12:13:52.580629+0000","last_peered":"2024-03-14T12:13:52.580629+0000","last_clean":"2024-03-14T12:13:52.580629+0000","last_became_active":"2024-03-14T12:13:51.577667+0000","last_became_peered":"2024-03-14T12:13:51.577667+0000","last_unstale":"2024-03-14T12:13:52.580629+0000","last_undegraded":"2024-03-14T12:13:52.580629+0000","last_fullsized":"2024-03-14T12:13:52.580629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477630+0000","last_change":"2024-03-14T12:13:51.579889+0000","last_active":"2024-03-14T12:13:54.477630+0000","last_peered":"2024-03-14T12:13:54.477630+0000","last_clean":"2024-03-14T12:13:54.477630+0000","last_became_active":"2024-03-14T12:13:51.579585+0000","last_became_peered":"2024-03-14T12:13:51.579585+0000","last_unstale":"2024-03-14T12:13:54.477630+0000","last_undegraded":"2024-03-14T12:13:54.477630+0000","last_fullsized":"2024-03-14T12:13:54.477630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581817+0000","last_change":"2024-03-14T12:13:51.577340+0000","last_active":"2024-03-14T12:13:52.581817+0000","last_peered":"2024-03-14T12:13:52.581817+0000","last_clean":"2024-03-14T12:13:52.581817+0000","last_became_active":"2024-03-14T12:13:51.577080+0000","last_became_peered":"2024-03-14T12:13:51.577080+0000","last_unstale":"2024-03-14T12:13:52.581817+0000","last_undegraded":"2024-03-14T12:13:52.581817+0000","last_fullsized":"2024-03-14T12:13:52.581817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579599+0000","last_change":"2024-03-14T12:13:51.573447+0000","last_active":"2024-03-14T12:13:52.579599+0000","last_peered":"2024-03-14T12:13:52.579599+0000","last_clean":"2024-03-14T12:13:52.579599+0000","last_became_active":"2024-03-14T12:13:51.573137+0000","last_became_peered":"2024-03-14T12:13:51.573137+0000","last_unstale":"2024-03-14T12:13:52.579599+0000","last_undegraded":"2024-03-14T12:13:52.579599+0000","last_fullsized":"2024-03-14T12:13:52.579599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478302+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:54.478302+0000","last_peered":"2024-03-14T12:13:54.478302+0000","last_clean":"2024-03-14T12:13:54.478302+0000","last_became_active":"2024-03-14T12:13:50.569748+0000","last_became_peered":"2024-03-14T12:13:50.569748+0000","last_unstale":"2024-03-14T12:13:54.478302+0000","last_undegraded":"2024-03-14T12:13:54.478302+0000","last_fullsized":"2024-03-14T12:13:54.478302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777628+0000","last_change":"2024-03-14T12:13:51.581108+0000","last_active":"2024-03-14T12:13:52.777628+0000","last_peered":"2024-03-14T12:13:52.777628+0000","last_clean":"2024-03-14T12:13:52.777628+0000","last_became_active":"2024-03-14T12:13:51.580829+0000","last_became_peered":"2024-03-14T12:13:51.580829+0000","last_unstale":"2024-03-14T12:13:52.777628+0000","last_undegraded":"2024-03-14T12:13:52.777628+0000","last_fullsized":"2024-03-14T12:13:52.777628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581951+0000","last_change":"2024-03-14T12:13:50.565826+0000","last_active":"2024-03-14T12:13:52.581951+0000","last_peered":"2024-03-14T12:13:52.581951+0000","last_clean":"2024-03-14T12:13:52.581951+0000","last_became_active":"2024-03-14T12:13:50.565553+0000","last_became_peered":"2024-03-14T12:13:50.565553+0000","last_unstale":"2024-03-14T12:13:52.581951+0000","last_undegraded":"2024-03-14T12:13:52.581951+0000","last_fullsized":"2024-03-14T12:13:52.581951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477885+0000","last_change":"2024-03-14T12:13:50.563462+0000","last_active":"2024-03-14T12:13:54.477885+0000","last_peered":"2024-03-14T12:13:54.477885+0000","last_clean":"2024-03-14T12:13:54.477885+0000","last_became_active":"2024-03-14T12:13:50.563198+0000","last_became_peered":"2024-03-14T12:13:50.563198+0000","last_unstale":"2024-03-14T12:13:54.477885+0000","last_undegraded":"2024-03-14T12:13:54.477885+0000","last_fullsized":"2024-03-14T12:13:54.477885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477930+0000","last_change":"2024-03-14T12:13:51.577103+0000","last_active":"2024-03-14T12:13:54.477930+0000","last_peered":"2024-03-14T12:13:54.477930+0000","last_clean":"2024-03-14T12:13:54.477930+0000","last_became_active":"2024-03-14T12:13:51.576602+0000","last_became_peered":"2024-03-14T12:13:51.576602+0000","last_unstale":"2024-03-14T12:13:54.477930+0000","last_undegraded":"2024-03-14T12:13:54.477930+0000","last_fullsized":"2024-03-14T12:13:54.477930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580881+0000","last_change":"2024-03-14T12:13:50.559389+0000","last_active":"2024-03-14T12:13:52.580881+0000","last_peered":"2024-03-14T12:13:52.580881+0000","last_clean":"2024-03-14T12:13:52.580881+0000","last_became_active":"2024-03-14T12:13:50.559100+0000","last_became_peered":"2024-03-14T12:13:50.559100+0000","last_unstale":"2024-03-14T12:13:52.580881+0000","last_undegraded":"2024-03-14T12:13:52.580881+0000","last_fullsized":"2024-03-14T12:13:52.580881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573924+0000","last_change":"2024-03-14T12:13:51.576591+0000","last_active":"2024-03-14T12:13:52.573924+0000","last_peered":"2024-03-14T12:13:52.573924+0000","last_clean":"2024-03-14T12:13:52.573924+0000","last_became_active":"2024-03-14T12:13:51.576232+0000","last_became_peered":"2024-03-14T12:13:51.576232+0000","last_unstale":"2024-03-14T12:13:52.573924+0000","last_undegraded":"2024-03-14T12:13:52.573924+0000","last_fullsized":"2024-03-14T12:13:52.573924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581963+0000","last_change":"2024-03-14T12:13:50.568392+0000","last_active":"2024-03-14T12:13:52.581963+0000","last_peered":"2024-03-14T12:13:52.581963+0000","last_clean":"2024-03-14T12:13:52.581963+0000","last_became_active":"2024-03-14T12:13:50.568127+0000","last_became_peered":"2024-03-14T12:13:50.568127+0000","last_unstale":"2024-03-14T12:13:52.581963+0000","last_undegraded":"2024-03-14T12:13:52.581963+0000","last_fullsized":"2024-03-14T12:13:52.581963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580126+0000","last_change":"2024-03-14T12:13:51.584429+0000","last_active":"2024-03-14T12:13:52.580126+0000","last_peered":"2024-03-14T12:13:52.580126+0000","last_clean":"2024-03-14T12:13:52.580126+0000","last_became_active":"2024-03-14T12:13:51.584098+0000","last_became_peered":"2024-03-14T12:13:51.584098+0000","last_unstale":"2024-03-14T12:13:52.580126+0000","last_undegraded":"2024-03-14T12:13:52.580126+0000","last_fullsized":"2024-03-14T12:13:52.580126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580241+0000","last_change":"2024-03-14T12:13:50.567672+0000","last_active":"2024-03-14T12:13:52.580241+0000","last_peered":"2024-03-14T12:13:52.580241+0000","last_clean":"2024-03-14T12:13:52.580241+0000","last_became_active":"2024-03-14T12:13:50.567366+0000","last_became_peered":"2024-03-14T12:13:50.567366+0000","last_unstale":"2024-03-14T12:13:52.580241+0000","last_undegraded":"2024-03-14T12:13:52.580241+0000","last_fullsized":"2024-03-14T12:13:52.580241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777722+0000","last_change":"2024-03-14T12:13:51.575322+0000","last_active":"2024-03-14T12:13:52.777722+0000","last_peered":"2024-03-14T12:13:52.777722+0000","last_clean":"2024-03-14T12:13:52.777722+0000","last_became_active":"2024-03-14T12:13:51.574694+0000","last_became_peered":"2024-03-14T12:13:51.574694+0000","last_unstale":"2024-03-14T12:13:52.777722+0000","last_undegraded":"2024-03-14T12:13:52.777722+0000","last_fullsized":"2024-03-14T12:13:52.777722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573967+0000","last_change":"2024-03-14T12:13:50.567057+0000","last_active":"2024-03-14T12:13:52.573967+0000","last_peered":"2024-03-14T12:13:52.573967+0000","last_clean":"2024-03-14T12:13:52.573967+0000","last_became_active":"2024-03-14T12:13:50.566800+0000","last_became_peered":"2024-03-14T12:13:50.566800+0000","last_unstale":"2024-03-14T12:13:52.573967+0000","last_undegraded":"2024-03-14T12:13:52.573967+0000","last_fullsized":"2024-03-14T12:13:52.573967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580236+0000","last_change":"2024-03-14T12:13:51.585716+0000","last_active":"2024-03-14T12:13:52.580236+0000","last_peered":"2024-03-14T12:13:52.580236+0000","last_clean":"2024-03-14T12:13:52.580236+0000","last_became_active":"2024-03-14T12:13:51.585226+0000","last_became_peered":"2024-03-14T12:13:51.585226+0000","last_unstale":"2024-03-14T12:13:52.580236+0000","last_undegraded":"2024-03-14T12:13:52.580236+0000","last_fullsized":"2024-03-14T12:13:52.580236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582041+0000","last_change":"2024-03-14T12:13:50.562666+0000","last_active":"2024-03-14T12:13:52.582041+0000","last_peered":"2024-03-14T12:13:52.582041+0000","last_clean":"2024-03-14T12:13:52.582041+0000","last_became_active":"2024-03-14T12:13:50.562367+0000","last_became_peered":"2024-03-14T12:13:50.562367+0000","last_unstale":"2024-03-14T12:13:52.582041+0000","last_undegraded":"2024-03-14T12:13:52.582041+0000","last_fullsized":"2024-03-14T12:13:52.582041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478606+0000","last_change":"2024-03-14T12:13:51.587183+0000","last_active":"2024-03-14T12:13:54.478606+0000","last_peered":"2024-03-14T12:13:54.478606+0000","last_clean":"2024-03-14T12:13:54.478606+0000","last_became_active":"2024-03-14T12:13:51.585685+0000","last_became_peered":"2024-03-14T12:13:51.585685+0000","last_unstale":"2024-03-14T12:13:54.478606+0000","last_undegraded":"2024-03-14T12:13:54.478606+0000","last_fullsized":"2024-03-14T12:13:54.478606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777610+0000","last_change":"2024-03-14T12:13:50.557053+0000","last_active":"2024-03-14T12:13:52.777610+0000","last_peered":"2024-03-14T12:13:52.777610+0000","last_clean":"2024-03-14T12:13:52.777610+0000","last_became_active":"2024-03-14T12:13:50.556737+0000","last_became_peered":"2024-03-14T12:13:50.556737+0000","last_unstale":"2024-03-14T12:13:52.777610+0000","last_undegraded":"2024-03-14T12:13:52.777610+0000","last_fullsized":"2024-03-14T12:13:52.777610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777653+0000","last_change":"2024-03-14T12:13:51.581665+0000","last_active":"2024-03-14T12:13:52.777653+0000","last_peered":"2024-03-14T12:13:52.777653+0000","last_clean":"2024-03-14T12:13:52.777653+0000","last_became_active":"2024-03-14T12:13:51.580980+0000","last_became_peered":"2024-03-14T12:13:51.580980+0000","last_unstale":"2024-03-14T12:13:52.777653+0000","last_undegraded":"2024-03-14T12:13:52.777653+0000","last_fullsized":"2024-03-14T12:13:52.777653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580677+0000","last_change":"2024-03-14T12:13:50.566403+0000","last_active":"2024-03-14T12:13:52.580677+0000","last_peered":"2024-03-14T12:13:52.580677+0000","last_clean":"2024-03-14T12:13:52.580677+0000","last_became_active":"2024-03-14T12:13:50.565782+0000","last_became_peered":"2024-03-14T12:13:50.565782+0000","last_unstale":"2024-03-14T12:13:52.580677+0000","last_undegraded":"2024-03-14T12:13:52.580677+0000","last_fullsized":"2024-03-14T12:13:52.580677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478689+0000","last_change":"2024-03-14T12:13:51.592512+0000","last_active":"2024-03-14T12:13:54.478689+0000","last_peered":"2024-03-14T12:13:54.478689+0000","last_clean":"2024-03-14T12:13:54.478689+0000","last_became_active":"2024-03-14T12:13:51.592273+0000","last_became_peered":"2024-03-14T12:13:51.592273+0000","last_unstale":"2024-03-14T12:13:54.478689+0000","last_undegraded":"2024-03-14T12:13:54.478689+0000","last_fullsized":"2024-03-14T12:13:54.478689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581835+0000","last_change":"2024-03-14T12:13:50.564239+0000","last_active":"2024-03-14T12:13:52.581835+0000","last_peered":"2024-03-14T12:13:52.581835+0000","last_clean":"2024-03-14T12:13:52.581835+0000","last_became_active":"2024-03-14T12:13:50.563920+0000","last_became_peered":"2024-03-14T12:13:50.563920+0000","last_unstale":"2024-03-14T12:13:52.581835+0000","last_undegraded":"2024-03-14T12:13:52.581835+0000","last_fullsized":"2024-03-14T12:13:52.581835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581784+0000","last_change":"2024-03-14T12:13:51.576682+0000","last_active":"2024-03-14T12:13:52.581784+0000","last_peered":"2024-03-14T12:13:52.581784+0000","last_clean":"2024-03-14T12:13:52.581784+0000","last_became_active":"2024-03-14T12:13:51.576420+0000","last_became_peered":"2024-03-14T12:13:51.576420+0000","last_unstale":"2024-03-14T12:13:52.581784+0000","last_undegraded":"2024-03-14T12:13:52.581784+0000","last_fullsized":"2024-03-14T12:13:52.581784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477551+0000","last_change":"2024-03-14T12:13:50.564352+0000","last_active":"2024-03-14T12:13:54.477551+0000","last_peered":"2024-03-14T12:13:54.477551+0000","last_clean":"2024-03-14T12:13:54.477551+0000","last_became_active":"2024-03-14T12:13:50.563979+0000","last_became_peered":"2024-03-14T12:13:50.563979+0000","last_unstale":"2024-03-14T12:13:54.477551+0000","last_undegraded":"2024-03-14T12:13:54.477551+0000","last_fullsized":"2024-03-14T12:13:54.477551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478395+0000","last_change":"2024-03-14T12:13:51.592382+0000","last_active":"2024-03-14T12:13:54.478395+0000","last_peered":"2024-03-14T12:13:54.478395+0000","last_clean":"2024-03-14T12:13:54.478395+0000","last_became_active":"2024-03-14T12:13:51.592109+0000","last_became_peered":"2024-03-14T12:13:51.592109+0000","last_unstale":"2024-03-14T12:13:54.478395+0000","last_undegraded":"2024-03-14T12:13:54.478395+0000","last_fullsized":"2024-03-14T12:13:54.478395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579929+0000","last_change":"2024-03-14T12:13:50.565493+0000","last_active":"2024-03-14T12:13:52.579929+0000","last_peered":"2024-03-14T12:13:52.579929+0000","last_clean":"2024-03-14T12:13:52.579929+0000","last_became_active":"2024-03-14T12:13:50.565247+0000","last_became_peered":"2024-03-14T12:13:50.565247+0000","last_unstale":"2024-03-14T12:13:52.579929+0000","last_undegraded":"2024-03-14T12:13:52.579929+0000","last_fullsized":"2024-03-14T12:13:52.579929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:39:10.529585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478754+0000","last_change":"2024-03-14T12:13:51.591007+0000","last_active":"2024-03-14T12:13:54.478754+0000","last_peered":"2024-03-14T12:13:54.478754+0000","last_clean":"2024-03-14T12:13:54.478754+0000","last_became_active":"2024-03-14T12:13:51.590750+0000","last_became_peered":"2024-03-14T12:13:51.590750+0000","last_unstale":"2024-03-14T12:13:54.478754+0000","last_undegraded":"2024-03-14T12:13:54.478754+0000","last_fullsized":"2024-03-14T12:13:54.478754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477792+0000","last_change":"2024-03-14T12:13:50.568050+0000","last_active":"2024-03-14T12:13:54.477792+0000","last_peered":"2024-03-14T12:13:54.477792+0000","last_clean":"2024-03-14T12:13:54.477792+0000","last_became_active":"2024-03-14T12:13:50.567713+0000","last_became_peered":"2024-03-14T12:13:50.567713+0000","last_unstale":"2024-03-14T12:13:54.477792+0000","last_undegraded":"2024-03-14T12:13:54.477792+0000","last_fullsized":"2024-03-14T12:13:54.477792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478989+0000","last_change":"2024-03-14T12:13:51.592562+0000","last_active":"2024-03-14T12:13:54.478989+0000","last_peered":"2024-03-14T12:13:54.478989+0000","last_clean":"2024-03-14T12:13:54.478989+0000","last_became_active":"2024-03-14T12:13:51.592255+0000","last_became_peered":"2024-03-14T12:13:51.592255+0000","last_unstale":"2024-03-14T12:13:54.478989+0000","last_undegraded":"2024-03-14T12:13:54.478989+0000","last_fullsized":"2024-03-14T12:13:54.478989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574041+0000","last_change":"2024-03-14T12:13:50.566339+0000","last_active":"2024-03-14T12:13:52.574041+0000","last_peered":"2024-03-14T12:13:52.574041+0000","last_clean":"2024-03-14T12:13:52.574041+0000","last_became_active":"2024-03-14T12:13:50.566058+0000","last_became_peered":"2024-03-14T12:13:50.566058+0000","last_unstale":"2024-03-14T12:13:52.574041+0000","last_undegraded":"2024-03-14T12:13:52.574041+0000","last_fullsized":"2024-03-14T12:13:52.574041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579993+0000","last_change":"2024-03-14T12:13:51.581453+0000","last_active":"2024-03-14T12:13:52.579993+0000","last_peered":"2024-03-14T12:13:52.579993+0000","last_clean":"2024-03-14T12:13:52.579993+0000","last_became_active":"2024-03-14T12:13:51.581168+0000","last_became_peered":"2024-03-14T12:13:51.581168+0000","last_unstale":"2024-03-14T12:13:52.579993+0000","last_undegraded":"2024-03-14T12:13:52.579993+0000","last_fullsized":"2024-03-14T12:13:52.579993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477730+0000","last_change":"2024-03-14T12:13:50.568172+0000","last_active":"2024-03-14T12:13:54.477730+0000","last_peered":"2024-03-14T12:13:54.477730+0000","last_clean":"2024-03-14T12:13:54.477730+0000","last_became_active":"2024-03-14T12:13:50.567825+0000","last_became_peered":"2024-03-14T12:13:50.567825+0000","last_unstale":"2024-03-14T12:13:54.477730+0000","last_undegraded":"2024-03-14T12:13:54.477730+0000","last_fullsized":"2024-03-14T12:13:54.477730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478581+0000","last_change":"2024-03-14T12:13:51.593995+0000","last_active":"2024-03-14T12:13:54.478581+0000","last_peered":"2024-03-14T12:13:54.478581+0000","last_clean":"2024-03-14T12:13:54.478581+0000","last_became_active":"2024-03-14T12:13:51.593473+0000","last_became_peered":"2024-03-14T12:13:51.593473+0000","last_unstale":"2024-03-14T12:13:54.478581+0000","last_undegraded":"2024-03-14T12:13:54.478581+0000","last_fullsized":"2024-03-14T12:13:54.478581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478363+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:13:54.478363+0000","last_peered":"2024-03-14T12:13:54.478363+0000","last_clean":"2024-03-14T12:13:54.478363+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:13:54.478363+0000","last_undegraded":"2024-03-14T12:13:54.478363+0000","last_fullsized":"2024-03-14T12:13:54.478363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580336+0000","last_change":"2024-03-14T12:13:50.565229+0000","last_active":"2024-03-14T12:13:52.580336+0000","last_peered":"2024-03-14T12:13:52.580336+0000","last_clean":"2024-03-14T12:13:52.580336+0000","last_became_active":"2024-03-14T12:13:50.564952+0000","last_became_peered":"2024-03-14T12:13:50.564952+0000","last_unstale":"2024-03-14T12:13:52.580336+0000","last_undegraded":"2024-03-14T12:13:52.580336+0000","last_fullsized":"2024-03-14T12:13:52.580336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478072+0000","last_change":"2024-03-14T12:13:51.573189+0000","last_active":"2024-03-14T12:13:54.478072+0000","last_peered":"2024-03-14T12:13:54.478072+0000","last_clean":"2024-03-14T12:13:54.478072+0000","last_became_active":"2024-03-14T12:13:51.572900+0000","last_became_peered":"2024-03-14T12:13:51.572900+0000","last_unstale":"2024-03-14T12:13:54.478072+0000","last_undegraded":"2024-03-14T12:13:54.478072+0000","last_fullsized":"2024-03-14T12:13:54.478072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478221+0000","last_change":"2024-03-14T12:13:50.571689+0000","last_active":"2024-03-14T12:13:54.478221+0000","last_peered":"2024-03-14T12:13:54.478221+0000","last_clean":"2024-03-14T12:13:54.478221+0000","last_became_active":"2024-03-14T12:13:50.571292+0000","last_became_peered":"2024-03-14T12:13:50.571292+0000","last_unstale":"2024-03-14T12:13:54.478221+0000","last_undegraded":"2024-03-14T12:13:54.478221+0000","last_fullsized":"2024-03-14T12:13:54.478221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777395+0000","last_change":"2024-03-14T12:13:51.582845+0000","last_active":"2024-03-14T12:13:52.777395+0000","last_peered":"2024-03-14T12:13:52.777395+0000","last_clean":"2024-03-14T12:13:52.777395+0000","last_became_active":"2024-03-14T12:13:51.582580+0000","last_became_peered":"2024-03-14T12:13:51.582580+0000","last_unstale":"2024-03-14T12:13:52.777395+0000","last_undegraded":"2024-03-14T12:13:52.777395+0000","last_fullsized":"2024-03-14T12:13:52.777395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477686+0000","last_change":"2024-03-14T12:13:50.568041+0000","last_active":"2024-03-14T12:13:54.477686+0000","last_peered":"2024-03-14T12:13:54.477686+0000","last_clean":"2024-03-14T12:13:54.477686+0000","last_became_active":"2024-03-14T12:13:50.567698+0000","last_became_peered":"2024-03-14T12:13:50.567698+0000","last_unstale":"2024-03-14T12:13:54.477686+0000","last_undegraded":"2024-03-14T12:13:54.477686+0000","last_fullsized":"2024-03-14T12:13:54.477686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573917+0000","last_change":"2024-03-14T12:13:51.572247+0000","last_active":"2024-03-14T12:13:52.573917+0000","last_peered":"2024-03-14T12:13:52.573917+0000","last_clean":"2024-03-14T12:13:52.573917+0000","last_became_active":"2024-03-14T12:13:51.572000+0000","last_became_peered":"2024-03-14T12:13:51.572000+0000","last_unstale":"2024-03-14T12:13:52.573917+0000","last_undegraded":"2024-03-14T12:13:52.573917+0000","last_fullsized":"2024-03-14T12:13:52.573917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580774+0000","last_change":"2024-03-14T12:13:50.562695+0000","last_active":"2024-03-14T12:13:52.580774+0000","last_peered":"2024-03-14T12:13:52.580774+0000","last_clean":"2024-03-14T12:13:52.580774+0000","last_became_active":"2024-03-14T12:13:50.562419+0000","last_became_peered":"2024-03-14T12:13:50.562419+0000","last_unstale":"2024-03-14T12:13:52.580774+0000","last_undegraded":"2024-03-14T12:13:52.580774+0000","last_fullsized":"2024-03-14T12:13:52.580774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580029+0000","last_change":"2024-03-14T12:13:51.578692+0000","last_active":"2024-03-14T12:13:52.580029+0000","last_peered":"2024-03-14T12:13:52.580029+0000","last_clean":"2024-03-14T12:13:52.580029+0000","last_became_active":"2024-03-14T12:13:51.578375+0000","last_became_peered":"2024-03-14T12:13:51.578375+0000","last_unstale":"2024-03-14T12:13:52.580029+0000","last_undegraded":"2024-03-14T12:13:52.580029+0000","last_fullsized":"2024-03-14T12:13:52.580029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573848+0000","last_change":"2024-03-14T12:13:50.568583+0000","last_active":"2024-03-14T12:13:52.573848+0000","last_peered":"2024-03-14T12:13:52.573848+0000","last_clean":"2024-03-14T12:13:52.573848+0000","last_became_active":"2024-03-14T12:13:50.568267+0000","last_became_peered":"2024-03-14T12:13:50.568267+0000","last_unstale":"2024-03-14T12:13:52.573848+0000","last_undegraded":"2024-03-14T12:13:52.573848+0000","last_fullsized":"2024-03-14T12:13:52.573848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580655+0000","last_change":"2024-03-14T12:13:51.579205+0000","last_active":"2024-03-14T12:13:52.580655+0000","last_peered":"2024-03-14T12:13:52.580655+0000","last_clean":"2024-03-14T12:13:52.580655+0000","last_became_active":"2024-03-14T12:13:51.578901+0000","last_became_peered":"2024-03-14T12:13:51.578901+0000","last_unstale":"2024-03-14T12:13:52.580655+0000","last_undegraded":"2024-03-14T12:13:52.580655+0000","last_fullsized":"2024-03-14T12:13:52.580655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.479092+0000","last_change":"2024-03-14T12:13:50.571020+0000","last_active":"2024-03-14T12:13:54.479092+0000","last_peered":"2024-03-14T12:13:54.479092+0000","last_clean":"2024-03-14T12:13:54.479092+0000","last_became_active":"2024-03-14T12:13:50.570744+0000","last_became_peered":"2024-03-14T12:13:50.570744+0000","last_unstale":"2024-03-14T12:13:54.479092+0000","last_undegraded":"2024-03-14T12:13:54.479092+0000","last_fullsized":"2024-03-14T12:13:54.479092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573864+0000","last_change":"2024-03-14T12:13:51.574187+0000","last_active":"2024-03-14T12:13:52.573864+0000","last_peered":"2024-03-14T12:13:52.573864+0000","last_clean":"2024-03-14T12:13:52.573864+0000","last_became_active":"2024-03-14T12:13:51.573916+0000","last_became_peered":"2024-03-14T12:13:51.573916+0000","last_unstale":"2024-03-14T12:13:52.573864+0000","last_undegraded":"2024-03-14T12:13:52.573864+0000","last_fullsized":"2024-03-14T12:13:52.573864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580829+0000","last_change":"2024-03-14T12:13:50.569678+0000","last_active":"2024-03-14T12:13:52.580829+0000","last_peered":"2024-03-14T12:13:52.580829+0000","last_clean":"2024-03-14T12:13:52.580829+0000","last_became_active":"2024-03-14T12:13:50.569377+0000","last_became_peered":"2024-03-14T12:13:50.569377+0000","last_unstale":"2024-03-14T12:13:52.580829+0000","last_undegraded":"2024-03-14T12:13:52.580829+0000","last_fullsized":"2024-03-14T12:13:52.580829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478237+0000","last_change":"2024-03-14T12:13:51.579167+0000","last_active":"2024-03-14T12:13:54.478237+0000","last_peered":"2024-03-14T12:13:54.478237+0000","last_clean":"2024-03-14T12:13:54.478237+0000","last_became_active":"2024-03-14T12:13:51.578971+0000","last_became_peered":"2024-03-14T12:13:51.578971+0000","last_unstale":"2024-03-14T12:13:54.478237+0000","last_undegraded":"2024-03-14T12:13:54.478237+0000","last_fullsized":"2024-03-14T12:13:54.478237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582080+0000","last_change":"2024-03-14T12:13:50.570044+0000","last_active":"2024-03-14T12:13:52.582080+0000","last_peered":"2024-03-14T12:13:52.582080+0000","last_clean":"2024-03-14T12:13:52.582080+0000","last_became_active":"2024-03-14T12:13:50.569759+0000","last_became_peered":"2024-03-14T12:13:50.569759+0000","last_unstale":"2024-03-14T12:13:52.582080+0000","last_undegraded":"2024-03-14T12:13:52.582080+0000","last_fullsized":"2024-03-14T12:13:52.582080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777595+0000","last_change":"2024-03-14T12:13:51.581169+0000","last_active":"2024-03-14T12:13:52.777595+0000","last_peered":"2024-03-14T12:13:52.777595+0000","last_clean":"2024-03-14T12:13:52.777595+0000","last_became_active":"2024-03-14T12:13:51.580757+0000","last_became_peered":"2024-03-14T12:13:51.580757+0000","last_unstale":"2024-03-14T12:13:52.777595+0000","last_undegraded":"2024-03-14T12:13:52.777595+0000","last_fullsized":"2024-03-14T12:13:52.777595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478220+0000","last_change":"2024-03-14T12:13:50.571439+0000","last_active":"2024-03-14T12:13:54.478220+0000","last_peered":"2024-03-14T12:13:54.478220+0000","last_clean":"2024-03-14T12:13:54.478220+0000","last_became_active":"2024-03-14T12:13:50.571184+0000","last_became_peered":"2024-03-14T12:13:50.571184+0000","last_unstale":"2024-03-14T12:13:54.478220+0000","last_undegraded":"2024-03-14T12:13:54.478220+0000","last_fullsized":"2024-03-14T12:13:54.478220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579859+0000","last_change":"2024-03-14T12:13:51.573261+0000","last_active":"2024-03-14T12:13:52.579859+0000","last_peered":"2024-03-14T12:13:52.579859+0000","last_clean":"2024-03-14T12:13:52.579859+0000","last_became_active":"2024-03-14T12:13:51.572866+0000","last_became_peered":"2024-03-14T12:13:51.572866+0000","last_unstale":"2024-03-14T12:13:52.579859+0000","last_undegraded":"2024-03-14T12:13:52.579859+0000","last_fullsized":"2024-03-14T12:13:52.579859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:39.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573916+0000","last_change":"2024-03-14T12:13:50.570820+0000","last_active":"2024-03-14T12:13:52.573916+0000","last_peered":"2024-03-14T12:13:52.573916+0000","last_clean":"2024-03-14T12:13:52.573916+0000","last_became_active":"2024-03-14T12:13:50.570423+0000","last_became_peered":"2024-03-14T12:13:50.570423+0000","last_unstale":"2024-03-14T12:13:52.573916+0000","last_undegraded":"2024-03-14T12:13:52.573916+0000","last_fullsized":"2024-03-14T12:13:52.573916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477998+0000","last_change":"2024-03-14T12:13:51.575973+0000","last_active":"2024-03-14T12:13:54.477998+0000","last_peered":"2024-03-14T12:13:54.477998+0000","last_clean":"2024-03-14T12:13:54.477998+0000","last_became_active":"2024-03-14T12:13:51.575695+0000","last_became_peered":"2024-03-14T12:13:51.575695+0000","last_unstale":"2024-03-14T12:13:54.477998+0000","last_undegraded":"2024-03-14T12:13:54.477998+0000","last_fullsized":"2024-03-14T12:13:54.477998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573877+0000","last_change":"2024-03-14T12:13:50.565398+0000","last_active":"2024-03-14T12:13:52.573877+0000","last_peered":"2024-03-14T12:13:52.573877+0000","last_clean":"2024-03-14T12:13:52.573877+0000","last_became_active":"2024-03-14T12:13:50.565109+0000","last_became_peered":"2024-03-14T12:13:50.565109+0000","last_unstale":"2024-03-14T12:13:52.573877+0000","last_undegraded":"2024-03-14T12:13:52.573877+0000","last_fullsized":"2024-03-14T12:13:52.573877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478222+0000","last_change":"2024-03-14T12:13:51.576916+0000","last_active":"2024-03-14T12:13:54.478222+0000","last_peered":"2024-03-14T12:13:54.478222+0000","last_clean":"2024-03-14T12:13:54.478222+0000","last_became_active":"2024-03-14T12:13:51.576639+0000","last_became_peered":"2024-03-14T12:13:51.576639+0000","last_unstale":"2024-03-14T12:13:54.478222+0000","last_undegraded":"2024-03-14T12:13:54.478222+0000","last_fullsized":"2024-03-14T12:13:54.478222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478704+0000","last_change":"2024-03-14T12:13:50.575585+0000","last_active":"2024-03-14T12:13:54.478704+0000","last_peered":"2024-03-14T12:13:54.478704+0000","last_clean":"2024-03-14T12:13:54.478704+0000","last_became_active":"2024-03-14T12:13:50.575288+0000","last_became_peered":"2024-03-14T12:13:50.575288+0000","last_unstale":"2024-03-14T12:13:54.478704+0000","last_undegraded":"2024-03-14T12:13:54.478704+0000","last_fullsized":"2024-03-14T12:13:54.478704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581882+0000","last_change":"2024-03-14T12:13:51.588022+0000","last_active":"2024-03-14T12:13:52.581882+0000","last_peered":"2024-03-14T12:13:52.581882+0000","last_clean":"2024-03-14T12:13:52.581882+0000","last_became_active":"2024-03-14T12:13:51.587766+0000","last_became_peered":"2024-03-14T12:13:51.587766+0000","last_unstale":"2024-03-14T12:13:52.581882+0000","last_undegraded":"2024-03-14T12:13:52.581882+0000","last_fullsized":"2024-03-14T12:13:52.581882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580894+0000","last_change":"2024-03-14T12:13:51.572504+0000","last_active":"2024-03-14T12:13:52.580894+0000","last_peered":"2024-03-14T12:13:52.580894+0000","last_clean":"2024-03-14T12:13:52.580894+0000","last_became_active":"2024-03-14T12:13:51.572219+0000","last_became_peered":"2024-03-14T12:13:51.572219+0000","last_unstale":"2024-03-14T12:13:52.580894+0000","last_undegraded":"2024-03-14T12:13:52.580894+0000","last_fullsized":"2024-03-14T12:13:52.580894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478474+0000","last_change":"2024-03-14T12:13:50.572648+0000","last_active":"2024-03-14T12:13:54.478474+0000","last_peered":"2024-03-14T12:13:54.478474+0000","last_clean":"2024-03-14T12:13:54.478474+0000","last_became_active":"2024-03-14T12:13:50.572409+0000","last_became_peered":"2024-03-14T12:13:50.572409+0000","last_unstale":"2024-03-14T12:13:54.478474+0000","last_undegraded":"2024-03-14T12:13:54.478474+0000","last_fullsized":"2024-03-14T12:13:54.478474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579831+0000","last_change":"2024-03-14T12:13:51.580823+0000","last_active":"2024-03-14T12:13:52.579831+0000","last_peered":"2024-03-14T12:13:52.579831+0000","last_clean":"2024-03-14T12:13:52.579831+0000","last_became_active":"2024-03-14T12:13:51.574625+0000","last_became_peered":"2024-03-14T12:13:51.574625+0000","last_unstale":"2024-03-14T12:13:52.579831+0000","last_undegraded":"2024-03-14T12:13:52.579831+0000","last_fullsized":"2024-03-14T12:13:52.579831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478728+0000","last_change":"2024-03-14T12:13:50.572321+0000","last_active":"2024-03-14T12:13:54.478728+0000","last_peered":"2024-03-14T12:13:54.478728+0000","last_clean":"2024-03-14T12:13:54.478728+0000","last_became_active":"2024-03-14T12:13:50.572001+0000","last_became_peered":"2024-03-14T12:13:50.572001+0000","last_unstale":"2024-03-14T12:13:54.478728+0000","last_undegraded":"2024-03-14T12:13:54.478728+0000","last_fullsized":"2024-03-14T12:13:54.478728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777630+0000","last_change":"2024-03-14T12:13:51.575693+0000","last_active":"2024-03-14T12:13:52.777630+0000","last_peered":"2024-03-14T12:13:52.777630+0000","last_clean":"2024-03-14T12:13:52.777630+0000","last_became_active":"2024-03-14T12:13:51.574493+0000","last_became_peered":"2024-03-14T12:13:51.574493+0000","last_unstale":"2024-03-14T12:13:52.777630+0000","last_undegraded":"2024-03-14T12:13:52.777630+0000","last_fullsized":"2024-03-14T12:13:52.777630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478473+0000","last_change":"2024-03-14T12:13:50.565913+0000","last_active":"2024-03-14T12:13:54.478473+0000","last_peered":"2024-03-14T12:13:54.478473+0000","last_clean":"2024-03-14T12:13:54.478473+0000","last_became_active":"2024-03-14T12:13:50.565652+0000","last_became_peered":"2024-03-14T12:13:50.565652+0000","last_unstale":"2024-03-14T12:13:54.478473+0000","last_undegraded":"2024-03-14T12:13:54.478473+0000","last_fullsized":"2024-03-14T12:13:54.478473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478392+0000","last_change":"2024-03-14T12:13:51.581638+0000","last_active":"2024-03-14T12:13:54.478392+0000","last_peered":"2024-03-14T12:13:54.478392+0000","last_clean":"2024-03-14T12:13:54.478392+0000","last_became_active":"2024-03-14T12:13:51.581373+0000","last_became_peered":"2024-03-14T12:13:51.581373+0000","last_unstale":"2024-03-14T12:13:54.478392+0000","last_undegraded":"2024-03-14T12:13:54.478392+0000","last_fullsized":"2024-03-14T12:13:54.478392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777698+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:52.777698+0000","last_peered":"2024-03-14T12:13:52.777698+0000","last_clean":"2024-03-14T12:13:52.777698+0000","last_became_active":"2024-03-14T12:13:50.569816+0000","last_became_peered":"2024-03-14T12:13:50.569816+0000","last_unstale":"2024-03-14T12:13:52.777698+0000","last_undegraded":"2024-03-14T12:13:52.777698+0000","last_fullsized":"2024-03-14T12:13:52.777698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:13:59.785 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T12:13:59.785 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-14T12:13:59.786 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-14T12:13:59.786 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-14T12:13:59.786 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-14T12:13:59.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.921+0000 7f3cbdb7e640 1 Processor -- start 2024-03-14T12:13:59.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.921+0000 7f3cbdb7e640 1 -- start start 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cbdb7e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3cb80cdb50 0x7f3cb80b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cbdb7e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb806ed00 con 0x7f3cb80ceb90 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cbdb7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb806efe0 con 0x7f3cb815e190 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cbdb7e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb80cc0f0 con 0x7f3cb80cdb50 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cb7fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3cb80cdb50 0x7f3cb80b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cb7fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3cb80cdb50 0x7f3cb80b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48590/0 (socket says 172.21.15.82:48590) 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cb7fff640 1 -- 172.21.15.82:0/3900141795 learned_addr learned my addr 172.21.15.82:0/3900141795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cb7fff640 1 -- 172.21.15.82:0/3900141795 >> v1:172.21.15.164:6789/0 conn(0x7f3cb815e190 legacy=0x7f3cb80ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.922+0000 7f3cb7fff640 1 -- 172.21.15.82:0/3900141795 >> v1:172.21.15.82:6789/0 conn(0x7f3cb80ceb90 legacy=0x7f3cb80cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.923+0000 7f3cb7fff640 1 -- 172.21.15.82:0/3900141795 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb80cc340 con 0x7f3cb80cdb50 2024-03-14T12:13:59.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.923+0000 7f3cb7fff640 1 --2- 172.21.15.82:0/3900141795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb80b4130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009e50 tx=0x7f3cb002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a87d1c0ab98c1a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:59.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.923+0000 7f3cb67fc640 1 -- 172.21.15.82:0/3900141795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb002c9f0 con 0x7f3cb80cdb50 2024-03-14T12:13:59.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.923+0000 7f3cb67fc640 1 -- 172.21.15.82:0/3900141795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb002cb90 con 0x7f3cb80cdb50 2024-03-14T12:13:59.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.923+0000 7f3cb67fc640 1 -- 172.21.15.82:0/3900141795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00343f0 con 0x7f3cb80cdb50 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.924+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/3900141795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 msgr2=0x7f3cb80b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.924+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/3900141795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb80b4130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009e50 tx=0x7f3cb002be50 comp rx=0 tx=0).stop 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.924+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/3900141795 shutdown_connections 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.924+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/3900141795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb80b4130 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.924+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/3900141795 >> 172.21.15.82:0/3900141795 conn(0x7f3cb80b3610 msgr2=0x7f3cb80b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.924+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/3900141795 shutdown_connections 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.924+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/3900141795 wait complete. 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.925+0000 7f3cbdb7e640 1 Processor -- start 2024-03-14T12:13:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.925+0000 7f3cbdb7e640 1 -- start start 2024-03-14T12:13:59.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cbdb7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cbdb7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3cb80ceb90 0x7f3cb81f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cbdb7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cb815e190 0x7f3cb80cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cbdb7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80d0710 con 0x7f3cb80cdb50 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cbdb7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80d0590 con 0x7f3cb815e190 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cbdb7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80d0890 con 0x7f3cb80ceb90 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48592/0 (socket says 172.21.15.82:48592) 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 -- 172.21.15.82:0/1229321988 learned_addr learned my addr 172.21.15.82:0/1229321988 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb7fff640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cb815e190 0x7f3cb80cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb6ffd640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3cb80ceb90 0x7f3cb81f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 -- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3cb80ceb90 msgr2=0x7f3cb81f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3cb80ceb90 0x7f3cb81f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 -- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cb815e190 msgr2=0x7f3cb80cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cb815e190 0x7f3cb80cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:13:59.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 -- 172.21.15.82:0/1229321988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb815d410 con 0x7f3cb80cdb50 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb7fff640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cb815e190 0x7f3cb80cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.926+0000 7f3cb77fe640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb815ced0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b420 tx=0x7f3ca800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.927+0000 7f3c9ffff640 1 -- 172.21.15.82:0/1229321988 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8011830 con 0x7f3cb80cdb50 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.927+0000 7f3c9ffff640 1 -- 172.21.15.82:0/1229321988 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8016070 con 0x7f3cb80cdb50 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.927+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb81f47c0 con 0x7f3cb80cdb50 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.927+0000 7f3c9ffff640 1 -- 172.21.15.82:0/1229321988 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8011c10 con 0x7f3cb80cdb50 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.927+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb80c6b00 con 0x7f3cb80cdb50 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.928+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb804f050 con 0x7f3cb80cdb50 2024-03-14T12:13:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.928+0000 7f3c9ffff640 1 -- 172.21.15.82:0/1229321988 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3ca801b460 con 0x7f3cb80cdb50 2024-03-14T12:13:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.928+0000 7f3c9ffff640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3ca406bbd0 0x7f3ca406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:13:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.928+0000 7f3cb6ffd640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3ca406bbd0 0x7f3ca406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:13:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.928+0000 7f3c9ffff640 1 -- 172.21.15.82:0/1229321988 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ca808a550 con 0x7f3cb80cdb50 2024-03-14T12:13:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.929+0000 7f3cb6ffd640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3ca406bbd0 0x7f3ca406e090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80b4f60 tx=0x7f3cac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:13:59.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:13:59.933+0000 7f3c9ffff640 1 -- 172.21.15.82:0/1229321988 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3ca8020020 con 0x7f3cb80cdb50 2024-03-14T12:14:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.149+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3cb80c7200 con 0x7f3cb80cdb50 2024-03-14T12:14:00.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.152+0000 7f3c9ffff640 1 -- 172.21.15.82:0/1229321988 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+172576 (secure 0 0 0) 0x7f3cb81f47c0 con 0x7f3cb80cdb50 2024-03-14T12:14:00.154 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:00.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3ca406bbd0 msgr2=0x7f3ca406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3ca406bbd0 0x7f3ca406e090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80b4f60 tx=0x7f3cac009040 comp rx=0 tx=0).stop 2024-03-14T12:14:00.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 msgr2=0x7f3cb815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb815ced0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b420 tx=0x7f3ca800b8e0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 shutdown_connections 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cb815e190 0x7f3cb80cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3ca406bbd0 0x7f3ca406e090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3cb80ceb90 0x7f3cb81f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 --2- 172.21.15.82:0/1229321988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb80cdb50 0x7f3cb815ced0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 >> 172.21.15.82:0/1229321988 conn(0x7f3cb80b3610 msgr2=0x7f3cb80b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 shutdown_connections 2024-03-14T12:14:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.154+0000 7f3cbdb7e640 1 -- 172.21.15.82:0/1229321988 wait complete. 2024-03-14T12:14:00.165 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"flags":0,"active_gid":4114,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":583731528},{"type":"v1","addr":"172.21.15.82:6833","nonce":583731528}]},"active_addr":"172.21.15.82:6833/583731528","active_change":"2024-03-14T12:13:31.487018+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4117,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2682774893}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2609924992}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2880376700}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1948132126}]}]} 2024-03-14T12:14:00.168 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-14T12:14:00.168 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-14T12:14:00.168 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.267+0000 7f38db481640 1 Processor -- start 2024-03-14T12:14:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.267+0000 7f38db481640 1 -- start start 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38db481640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f38d415f7a0 0x7f38d415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38db481640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d40ce210 0x7f38d41451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38db481640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40ce620 con 0x7f38d40cdd60 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38db481640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d40cf150 con 0x7f38d415f7a0 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38db481640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41600c0 con 0x7f38d40ce210 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d921d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f38d415f7a0 0x7f38d415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d921d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f38d415f7a0 0x7f38d415fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48606/0 (socket says 172.21.15.82:48606) 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d921d640 1 -- 172.21.15.82:0/2713121293 learned_addr learned my addr 172.21.15.82:0/2713121293 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d9a1e640 1 --2- 172.21.15.82:0/2713121293 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d40ce210 0x7f38d41451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d921d640 1 -- 172.21.15.82:0/2713121293 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d40ce210 msgr2=0x7f38d41451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d921d640 1 --2- 172.21.15.82:0/2713121293 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d40ce210 0x7f38d41451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d921d640 1 -- 172.21.15.82:0/2713121293 >> v1:172.21.15.164:6789/0 conn(0x7f38d40cdd60 legacy=0x7f38d4144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.268+0000 7f38d921d640 1 -- 172.21.15.82:0/2713121293 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40ce8b0 con 0x7f38d415f7a0 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.269+0000 7f38d921d640 1 --2- 172.21.15.82:0/2713121293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d415f7a0 0x7f38d415fb80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009870 tx=0x7f38d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de09d1a5352bbee server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.269+0000 7f38cbfff640 1 -- 172.21.15.82:0/2713121293 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0035020 con 0x7f38d415f7a0 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.269+0000 7f38cbfff640 1 -- 172.21.15.82:0/2713121293 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0031640 con 0x7f38d415f7a0 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.269+0000 7f38cbfff640 1 -- 172.21.15.82:0/2713121293 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d002d040 con 0x7f38d415f7a0 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 -- 172.21.15.82:0/2713121293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d415f7a0 msgr2=0x7f38d415fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 --2- 172.21.15.82:0/2713121293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d415f7a0 0x7f38d415fb80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009870 tx=0x7f38d002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 -- 172.21.15.82:0/2713121293 shutdown_connections 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 --2- 172.21.15.82:0/2713121293 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d40ce210 0x7f38d41451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 --2- 172.21.15.82:0/2713121293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d415f7a0 0x7f38d415fb80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 -- 172.21.15.82:0/2713121293 >> 172.21.15.82:0/2713121293 conn(0x7f38d40b1c70 msgr2=0x7f38d40cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 -- 172.21.15.82:0/2713121293 shutdown_connections 2024-03-14T12:14:00.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.270+0000 7f38db481640 1 -- 172.21.15.82:0/2713121293 wait complete. 2024-03-14T12:14:00.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.271+0000 7f38db481640 1 Processor -- start 2024-03-14T12:14:00.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.271+0000 7f38db481640 1 -- start start 2024-03-14T12:14:00.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38d40cdd60 0x7f38d4140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40ce210 0x7f38d41524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d415f7a0 0x7f38d41f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d406c4e0 con 0x7f38d40ce210 2024-03-14T12:14:00.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cee70 con 0x7f38d40cdd60 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cc4f0 con 0x7f38d415f7a0 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40ce210 0x7f38d41524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40ce210 0x7f38d41524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48610/0 (socket says 172.21.15.82:48610) 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 -- 172.21.15.82:0/2805412235 learned_addr learned my addr 172.21.15.82:0/2805412235 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d9a1e640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d415f7a0 0x7f38d41f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 -- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d415f7a0 msgr2=0x7f38d41f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d415f7a0 0x7f38d41f7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 -- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38d40cdd60 msgr2=0x7f38d4140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d921d640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38d40cdd60 0x7f38d4140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38d40cdd60 0x7f38d4140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 -- 172.21.15.82:0/2805412235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4154ff0 con 0x7f38d40ce210 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38d8a1c640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40ce210 0x7f38d41524c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c3f0 tx=0x7f38c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38c9ffb640 1 -- 172.21.15.82:0/2805412235 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40052d0 con 0x7f38d40ce210 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38c9ffb640 1 -- 172.21.15.82:0/2805412235 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c4005490 con 0x7f38d40ce210 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d41552d0 con 0x7f38d40ce210 2024-03-14T12:14:00.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38c9ffb640 1 -- 172.21.15.82:0/2805412235 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4007260 con 0x7f38d40ce210 2024-03-14T12:14:00.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.272+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d40b2be0 con 0x7f38d40ce210 2024-03-14T12:14:00.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.274+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d40b2de0 con 0x7f38d40ce210 2024-03-14T12:14:00.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.274+0000 7f38c9ffb640 1 -- 172.21.15.82:0/2805412235 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f38c4007400 con 0x7f38d40ce210 2024-03-14T12:14:00.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.274+0000 7f38c9ffb640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f38b806bbd0 0x7f38b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.274+0000 7f38d921d640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f38b806bbd0 0x7f38b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.274+0000 7f38c9ffb640 1 -- 172.21.15.82:0/2805412235 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38c4089f50 con 0x7f38d40ce210 2024-03-14T12:14:00.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.275+0000 7f38d921d640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f38b806bbd0 0x7f38b806e090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f38d0036000 tx=0x7f38d0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.278+0000 7f38c9ffb640 1 -- 172.21.15.82:0/2805412235 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f38d40b2de0 con 0x7f38d40ce210 2024-03-14T12:14:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.452+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38d404f050 con 0x7f38d40ce210 2024-03-14T12:14:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.453+0000 7f38c9ffb640 1 -- 172.21.15.82:0/2805412235 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f38c4058a40 con 0x7f38d40ce210 2024-03-14T12:14:00.453 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:00.453 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f38b806bbd0 msgr2=0x7f38b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f38b806bbd0 0x7f38b806e090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f38d0036000 tx=0x7f38d0008030 comp rx=0 tx=0).stop 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40ce210 msgr2=0x7f38d41524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40ce210 0x7f38d41524c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c3f0 tx=0x7f38c400c8b0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 shutdown_connections 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f38b806bbd0 0x7f38b806e090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38d415f7a0 0x7f38d41f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40ce210 0x7f38d41524c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 --2- 172.21.15.82:0/2805412235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38d40cdd60 0x7f38d4140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 >> 172.21.15.82:0/2805412235 conn(0x7f38d40b1c70 msgr2=0x7f38d41574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 shutdown_connections 2024-03-14T12:14:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.455+0000 7f38db481640 1 -- 172.21.15.82:0/2805412235 wait complete. 2024-03-14T12:14:00.466 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-14T12:14:00.467 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.565+0000 7f3f5639f640 1 Processor -- start 2024-03-14T12:14:00.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.565+0000 7f3f5639f640 1 -- start start 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f5639f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f5015ded0 0x7f3f500d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f5639f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f500b6c60 con 0x7f3f500d46e0 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f5639f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f500b6f40 con 0x7f3f500d28b0 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f5639f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f500b6a10 con 0x7f3f5015ded0 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f5539d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f5015ded0 0x7f3f500d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f5539d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f5015ded0 0x7f3f500d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48618/0 (socket says 172.21.15.82:48618) 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f5539d640 1 -- 172.21.15.82:0/1523281183 learned_addr learned my addr 172.21.15.82:0/1523281183 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.566+0000 7f3f54b9c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3f500d46e0 0x7f3f500d21a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34552/0 (socket says 172.21.15.82:34552) 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.567+0000 7f3f5539d640 1 -- 172.21.15.82:0/1523281183 >> v1:172.21.15.164:6789/0 conn(0x7f3f500d28b0 legacy=0x7f3f500b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.567+0000 7f3f5539d640 1 -- 172.21.15.82:0/1523281183 >> v1:172.21.15.82:6789/0 conn(0x7f3f500d46e0 legacy=0x7f3f500d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.567+0000 7f3f5539d640 1 -- 172.21.15.82:0/1523281183 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f500b7220 con 0x7f3f5015ded0 2024-03-14T12:14:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.567+0000 7f3f5539d640 1 --2- 172.21.15.82:0/1523281183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5015ded0 0x7f3f500d0c00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009e50 tx=0x7f3f4c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989d870739a446c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.567+0000 7f3f47fff640 1 -- 172.21.15.82:0/1523281183 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c02ca80 con 0x7f3f5015ded0 2024-03-14T12:14:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.567+0000 7f3f47fff640 1 -- 172.21.15.82:0/1523281183 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c02cc20 con 0x7f3f5015ded0 2024-03-14T12:14:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.567+0000 7f3f47fff640 1 -- 172.21.15.82:0/1523281183 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0343f0 con 0x7f3f5015ded0 2024-03-14T12:14:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.568+0000 7f3f5639f640 1 -- 172.21.15.82:0/1523281183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5015ded0 msgr2=0x7f3f500d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.568+0000 7f3f5639f640 1 --2- 172.21.15.82:0/1523281183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5015ded0 0x7f3f500d0c00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009e50 tx=0x7f3f4c02bee0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.568+0000 7f3f5639f640 1 -- 172.21.15.82:0/1523281183 shutdown_connections 2024-03-14T12:14:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.568+0000 7f3f5639f640 1 --2- 172.21.15.82:0/1523281183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5015ded0 0x7f3f500d0c00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.568+0000 7f3f5639f640 1 -- 172.21.15.82:0/1523281183 >> 172.21.15.82:0/1523281183 conn(0x7f3f500d3bc0 msgr2=0x7f3f500d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.568+0000 7f3f5639f640 1 -- 172.21.15.82:0/1523281183 shutdown_connections 2024-03-14T12:14:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.568+0000 7f3f5639f640 1 -- 172.21.15.82:0/1523281183 wait complete. 2024-03-14T12:14:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.569+0000 7f3f5639f640 1 Processor -- start 2024-03-14T12:14:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.569+0000 7f3f5639f640 1 -- start start 2024-03-14T12:14:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f5639f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f500d28b0 0x7f3f500c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f5639f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f500d46e0 0x7f3f50158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f5639f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f5015ded0 0x7f3f500b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f5639f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500d1d50 con 0x7f3f500d46e0 2024-03-14T12:14:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f5639f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500d1bd0 con 0x7f3f500d28b0 2024-03-14T12:14:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f5639f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5015e2b0 con 0x7f3f5015ded0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f500d46e0 0x7f3f50158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f500d46e0 0x7f3f50158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48632/0 (socket says 172.21.15.82:48632) 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 -- 172.21.15.82:0/3833492641 learned_addr learned my addr 172.21.15.82:0/3833492641 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f55b9e640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f5015ded0 0x7f3f500b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f5539d640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f500d28b0 0x7f3f500c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 -- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f5015ded0 msgr2=0x7f3f500b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f5015ded0 0x7f3f500b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 -- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f500d28b0 msgr2=0x7f3f500c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f500d28b0 0x7f3f500c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.570+0000 7f3f54b9c640 1 -- 172.21.15.82:0/3833492641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5015b210 con 0x7f3f500d46e0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.571+0000 7f3f54b9c640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f500d46e0 0x7f3f50158b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c440 tx=0x7f3f4000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.571+0000 7f3f45ffb640 1 -- 172.21.15.82:0/3833492641 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400074e0 con 0x7f3f500d46e0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.571+0000 7f3f45ffb640 1 -- 172.21.15.82:0/3833492641 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40007680 con 0x7f3f500d46e0 2024-03-14T12:14:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.571+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5015b4f0 con 0x7f3f500d46e0 2024-03-14T12:14:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.571+0000 7f3f45ffb640 1 -- 172.21.15.82:0/3833492641 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40004a20 con 0x7f3f500d46e0 2024-03-14T12:14:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.571+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5015bab0 con 0x7f3f500d46e0 2024-03-14T12:14:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.572+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f500d36b0 con 0x7f3f500d46e0 2024-03-14T12:14:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.572+0000 7f3f45ffb640 1 -- 172.21.15.82:0/3833492641 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3f4001a020 con 0x7f3f500d46e0 2024-03-14T12:14:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.573+0000 7f3f45ffb640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f3406bbd0 0x7f3f3406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.573+0000 7f3f45ffb640 1 -- 172.21.15.82:0/3833492641 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f4008a010 con 0x7f3f500d46e0 2024-03-14T12:14:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.573+0000 7f3f5539d640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f3406bbd0 0x7f3f3406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.573+0000 7f3f5539d640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f3406bbd0 0x7f3f3406e090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c035000 tx=0x7f3f4c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.577+0000 7f3f45ffb640 1 -- 172.21.15.82:0/3833492641 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3f500d36b0 con 0x7f3f500d46e0 2024-03-14T12:14:00.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.753+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f5015bc80 con 0x7f3f500d46e0 2024-03-14T12:14:00.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.754+0000 7f3f45ffb640 1 -- 172.21.15.82:0/3833492641 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f3f40058540 con 0x7f3f500d46e0 2024-03-14T12:14:00.754 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:00.754 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:00.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f3406bbd0 msgr2=0x7f3f3406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f3406bbd0 0x7f3f3406e090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c035000 tx=0x7f3f4c030000 comp rx=0 tx=0).stop 2024-03-14T12:14:00.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f500d46e0 msgr2=0x7f3f50158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f500d46e0 0x7f3f50158b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c440 tx=0x7f3f4000c900 comp rx=0 tx=0).stop 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 shutdown_connections 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f3406bbd0 0x7f3f3406e090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f5015ded0 0x7f3f500b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f500d46e0 0x7f3f50158b50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 --2- 172.21.15.82:0/3833492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f500d28b0 0x7f3f500c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 >> 172.21.15.82:0/3833492641 conn(0x7f3f500d3bc0 msgr2=0x7f3f5006ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 shutdown_connections 2024-03-14T12:14:00.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.756+0000 7f3f5639f640 1 -- 172.21.15.82:0/3833492641 wait complete. 2024-03-14T12:14:00.767 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-14T12:14:00.768 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-14T12:14:00.768 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-14T12:14:00.768 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-14T12:14:00.769 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-14T12:14:00.769 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-14T12:14:00.769 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-14T12:14:00.770 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-14T12:14:00.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.939+0000 7fc53a23d640 1 Processor -- start 2024-03-14T12:14:00.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.939+0000 7fc53a23d640 1 -- start start 2024-03-14T12:14:00.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.939+0000 7fc5183e6640 1 Processor -- start 2024-03-14T12:14:00.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f7e9f59e640 1 Processor -- start 2024-03-14T12:14:00.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.939+0000 7f6aa26a8640 1 Processor -- start 2024-03-14T12:14:00.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f6aa26a8640 1 -- start start 2024-03-14T12:14:00.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f33ff281640 1 Processor -- start 2024-03-14T12:14:00.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f7e9f59e640 1 -- start start 2024-03-14T12:14:00.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f0cdf59e640 1 Processor -- start 2024-03-14T12:14:00.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f0cdf59e640 1 -- start start 2024-03-14T12:14:00.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.939+0000 7fc5183e6640 1 -- start start 2024-03-14T12:14:00.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f6c6c1fa640 1 Processor -- start 2024-03-14T12:14:00.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7fc53a23d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc534151230 0x7fc5340cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7fc53a23d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc53415dde0 0x7fc53415e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7fc53a23d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5340b3da0 con 0x7fc5340b4b70 2024-03-14T12:14:00.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7fc53a23d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc534070d40 con 0x7fc534151230 2024-03-14T12:14:00.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7fc53a23d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53406c3b0 con 0x7fc53415dde0 2024-03-14T12:14:00.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc5183e6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc510162740 0x7fc51006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc5183e6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc510162d90 con 0x7fc51006c8f0 2024-03-14T12:14:00.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc5183e6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc510163070 con 0x7fc5101661d0 2024-03-14T12:14:00.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc5183e6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc510145610 con 0x7fc510162740 2024-03-14T12:14:00.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc516983640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc51006c8f0 0x7fc510070b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34566/0 (socket says 172.21.15.82:34566) 2024-03-14T12:14:00.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc516983640 1 -- 172.21.15.82:0/2445114942 learned_addr learned my addr 172.21.15.82:0/2445114942 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:00.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc515981640 1 --2- 172.21.15.82:0/2445114942 >> v2:172.21.15.164:3300/0 conn(0x7fc510162740 0x7fc51006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6aa26a8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6aa26a8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a9c06d3e0 0x7f6a9c06d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6aa26a8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c15a280 con 0x7f6a9c071320 2024-03-14T12:14:00.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6aa26a8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9c071ec0 con 0x7f6a9c06d3e0 2024-03-14T12:14:00.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6aa26a8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0720e0 con 0x7f6a9c06e620 2024-03-14T12:14:00.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6bc59e9640 1 Processor -- start 2024-03-14T12:14:00.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.940+0000 7f33ff281640 1 -- start start 2024-03-14T12:14:00.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f7e9f59e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e98005e90 0x7f7e980062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f7e9f59e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e98053720 con 0x7f7e98006810 2024-03-14T12:14:00.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f7e9f59e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e98053a00 con 0x7f7e980ead00 2024-03-14T12:14:00.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f7e9f59e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e98054330 con 0x7f7e98005e90 2024-03-14T12:14:00.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6bc59e9640 1 -- start start 2024-03-14T12:14:00.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f0cdf59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80eb050 0x7f0cd80552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cdf59e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0cd80eb430 0x7f0cd80eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc5337fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc534151230 0x7fc5340cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc532ffd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc5340b4b70 0x7fc534070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:54060/0 (socket says 172.21.15.82:54060) 2024-03-14T12:14:00.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7fc533fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc53415dde0 0x7fc53415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc532ffd640 1 -- 172.21.15.82:0/1922133520 learned_addr learned my addr 172.21.15.82:0/1922133520 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f7e9ed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7e98006810 0x7f7e98006c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34582/0 (socket says 172.21.15.82:34582) 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f7e9ed9d640 1 -- 172.21.15.82:0/4039229324 learned_addr learned my addr 172.21.15.82:0/4039229324 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f7e9dd9b640 1 --2- 172.21.15.82:0/4039229324 >> v2:172.21.15.82:3300/0 conn(0x7f7e98005e90 0x7f7e980062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33ff281640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33ff281640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f33f806b250 0x7f33f80b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33ff281640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80cc1e0 con 0x7f33f81620c0 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33ff281640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f8166e10 con 0x7f33f806b250 2024-03-14T12:14:00.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33ff281640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80b36f0 con 0x7f33f81624a0 2024-03-14T12:14:00.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cdf59e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd80387d0 con 0x7f0cd80e7a60 2024-03-14T12:14:00.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cdf59e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd8057e20 con 0x7f0cd80eb430 2024-03-14T12:14:00.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cdf59e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd8057990 con 0x7f0cd80eb050 2024-03-14T12:14:00.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6aa1ea7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a9c071320 0x7f6a9c0717b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34578/0 (socket says 172.21.15.82:34578) 2024-03-14T12:14:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6aa1ea7640 1 -- 172.21.15.82:0/3154737939 learned_addr learned my addr 172.21.15.82:0/3154737939 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.941+0000 7f6c6c1fa640 1 -- start start 2024-03-14T12:14:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6aa16a6640 1 --2- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33fd81e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f33f806b250 0x7f33f80b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c6c1fa640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c640d2390 0x7f6c640d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c6c1fa640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6c640c9520 0x7f6c640cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c6c1fa640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c640d2cb0 con 0x7f6c640d1b90 2024-03-14T12:14:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c6c1fa640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c640d4d30 con 0x7f6c640d2390 2024-03-14T12:14:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c6c1fa640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c640cfcf0 con 0x7f6c640c9520 2024-03-14T12:14:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc515981640 1 -- 172.21.15.82:0/2445114942 >> v1:172.21.15.164:6789/0 conn(0x7fc5101661d0 legacy=0x7fc5101665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc515981640 1 -- 172.21.15.82:0/2445114942 >> v1:172.21.15.82:6789/0 conn(0x7fc51006c8f0 legacy=0x7fc510070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc515981640 1 -- 172.21.15.82:0/2445114942 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510163350 con 0x7fc510162740 2024-03-14T12:14:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c69f96640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c640d2390 0x7f6c640d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc5327fc640 1 -- 172.21.15.82:0/1922133520 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042980403 0 0) 0x7fc5340b3da0 con 0x7fc5340b4b70 2024-03-14T12:14:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc5327fc640 1 -- 172.21.15.82:0/1922133520 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc51c005730 con 0x7fc5340b4b70 2024-03-14T12:14:00.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c69f96640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c640d2390 0x7f6c640d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48666/0 (socket says 172.21.15.82:48666) 2024-03-14T12:14:00.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6c69f96640 1 -- 172.21.15.82:0/1887616845 learned_addr learned my addr 172.21.15.82:0/1887616845 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f6aa0ea5640 1 --2- 172.21.15.82:0/3154737939 >> v2:172.21.15.164:3300/0 conn(0x7f6a9c06d3e0 0x7f6a9c06d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc533fff640 1 -- 172.21.15.82:0/1922133520 >> v2:172.21.15.82:3300/0 conn(0x7fc534151230 msgr2=0x7fc5340cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc533fff640 1 --2- 172.21.15.82:0/1922133520 >> v2:172.21.15.82:3300/0 conn(0x7fc534151230 0x7fc5340cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc533fff640 1 -- 172.21.15.82:0/1922133520 >> v1:172.21.15.164:6789/0 conn(0x7fc5340b4b70 legacy=0x7fc534070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc533fff640 1 -- 172.21.15.82:0/1922133520 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340b4080 con 0x7fc53415dde0 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7fc5337fe640 1 --2- 172.21.15.82:0/1922133520 >> v2:172.21.15.82:3300/0 conn(0x7fc534151230 0x7fc5340cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cded9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0cd80e7a60 0x7f0cd80e7ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:54082/0 (socket says 172.21.15.82:54082) 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cded9d640 1 -- 172.21.15.82:0/1325528365 learned_addr learned my addr 172.21.15.82:0/1325528365 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cddd9b640 1 --2- 172.21.15.82:0/1325528365 >> v2:172.21.15.82:3300/0 conn(0x7f0cd80eb430 0x7f0cd80eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33fc81c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620983776 0 0) 0x7f7e98053a00 con 0x7f7e980ead00 2024-03-14T12:14:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e88003520 con 0x7f7e980ead00 2024-03-14T12:14:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33fc81c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f806ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:52918/0 (socket says 172.21.15.82:52918) 2024-03-14T12:14:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f0cde59c640 1 --2- 172.21.15.82:0/1325528365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80eb050 0x7f0cd80552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6c68f94640 1 -- 172.21.15.82:0/1887616845 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686649547 0 0) 0x7f6c640d2cb0 con 0x7f6c640d1b90 2024-03-14T12:14:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3694084774 0 0) 0x7f7e88003520 con 0x7f7e980ead00 2024-03-14T12:14:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 >> v2:172.21.15.82:3300/0 conn(0x7f7e98005e90 msgr2=0x7f7e980062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 --2- 172.21.15.82:0/4039229324 >> v2:172.21.15.82:3300/0 conn(0x7f7e98005e90 0x7f7e980062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 >> v1:172.21.15.82:6789/0 conn(0x7f7e98006810 legacy=0x7f7e98006c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98053ce0 con 0x7f7e980ead00 2024-03-14T12:14:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f7e90001650 con 0x7f7e980ead00 2024-03-14T12:14:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 >> v1:172.21.15.164:6789/0 conn(0x7f7e980ead00 legacy=0x7f7e9804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6c6a797640 1 --2- 172.21.15.82:0/1887616845 >> v2:172.21.15.164:3300/0 conn(0x7f6c640c9520 0x7f6c640cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6c68f94640 1 -- 172.21.15.82:0/1887616845 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c50003520 con 0x7f6c640d1b90 2024-03-14T12:14:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6c69f96640 1 -- 172.21.15.82:0/1887616845 >> v2:172.21.15.164:3300/0 conn(0x7f6c640c9520 msgr2=0x7f6c640cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6aa0ea5640 1 -- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 msgr2=0x7f6a9c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6aa0ea5640 1 --2- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6aa0ea5640 1 -- 172.21.15.82:0/3154737939 >> v1:172.21.15.82:6789/0 conn(0x7f6a9c071320 legacy=0x7f6a9c0717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6aa0ea5640 1 -- 172.21.15.82:0/3154737939 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0ccdc0 con 0x7f6a9c06d3e0 2024-03-14T12:14:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc59e9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6bc015f3b0 0x7f6bc006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc59e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00cc1f0 0x7f6bc00b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.942+0000 7f33fc81c640 1 -- 172.21.15.82:0/226735745 learned_addr learned my addr 172.21.15.82:0/226735745 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f33fc81c640 1 -- 172.21.15.82:0/226735745 >> v2:172.21.15.164:3300/0 conn(0x7f33f806b250 msgr2=0x7f33f80b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f33fc81c640 1 --2- 172.21.15.82:0/226735745 >> v2:172.21.15.164:3300/0 conn(0x7f33f806b250 0x7f33f80b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f33fc81c640 1 -- 172.21.15.82:0/226735745 >> v1:172.21.15.82:6789/0 conn(0x7f33f81620c0 legacy=0x7f33f81632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f33fc81c640 1 -- 172.21.15.82:0/226735745 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f8166290 con 0x7f33f81624a0 2024-03-14T12:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f33fd81e640 1 --2- 172.21.15.82:0/226735745 >> v2:172.21.15.164:3300/0 conn(0x7f33f806b250 0x7f33f80b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6c69f96640 1 --2- 172.21.15.82:0/1887616845 >> v2:172.21.15.164:3300/0 conn(0x7f6c640c9520 0x7f6c640cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc59e9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6bc00b0a80 0x7f6bc00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6c69f96640 1 -- 172.21.15.82:0/1887616845 >> v1:172.21.15.164:6789/0 conn(0x7f6c640d1b90 legacy=0x7f6c640ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6c69f96640 1 -- 172.21.15.82:0/1887616845 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c640d4a50 con 0x7f6c640d2390 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc59e9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00b0060 con 0x7f6bc00b0a80 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bbffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00cc1f0 0x7f6bc00b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc51e8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6bc00b0a80 0x7f6bc00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc51e8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6bc00b0a80 0x7f6bc00ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48676/0 (socket says 172.21.15.82:48676) 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bbffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00cc1f0 0x7f6bc00b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:52930/0 (socket says 172.21.15.82:52930) 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f0cde59c640 1 -- 172.21.15.82:0/1325528365 >> v2:172.21.15.82:3300/0 conn(0x7f0cd80eb430 msgr2=0x7f0cd80eb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f0cde59c640 1 --2- 172.21.15.82:0/1325528365 >> v2:172.21.15.82:3300/0 conn(0x7f0cd80eb430 0x7f0cd80eb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f0cde59c640 1 -- 172.21.15.82:0/1325528365 >> v1:172.21.15.164:6789/0 conn(0x7f0cd80e7a60 legacy=0x7f0cd80e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f0cde59c640 1 -- 172.21.15.82:0/1325528365 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8056d80 con 0x7f0cd80eb050 2024-03-14T12:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e880039f0 0x7f7e88003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e88004800 0x7f7e88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e880051c0 0x7f7e8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98054330 con 0x7f7e880051c0 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8800c0a0 con 0x7f7e880039f0 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8800c2a0 con 0x7f7e88004800 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f7e9e59c640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e880039f0 0x7f7e88003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f7e9e59c640 1 -- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e88004800 msgr2=0x7f7e88004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f7e9e59c640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e88004800 0x7f7e88004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f7e9e59c640 1 -- 172.21.15.82:0/4039229324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e880051c0 msgr2=0x7f7e8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f7e9e59c640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e880051c0 0x7f7e8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f7e9e59c640 1 -- 172.21.15.82:0/4039229324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98053ce0 con 0x7f7e880039f0 2024-03-14T12:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc59e9640 1 -- 172.21.15.82:0/1211190803 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc0070800 con 0x7f6bc015f3b0 2024-03-14T12:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f7e9ed9d640 1 -- 172.21.15.82:0/4039229324 reap_dead start 2024-03-14T12:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc59e9640 1 -- 172.21.15.82:0/1211190803 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00cff00 con 0x7f6bc00cc1f0 2024-03-14T12:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.943+0000 7f6bc51e8640 1 -- 172.21.15.82:0/1211190803 learned_addr learned my addr 172.21.15.82:0/1211190803 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7fc533fff640 1 --2- 172.21.15.82:0/1922133520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53415dde0 0x7fc53415e210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc53415edd0 tx=0x7fc52c02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab190c2d467b48b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7fc515981640 1 --2- 172.21.15.82:0/2445114942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510162740 0x7fc51006c3b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5000097a0 tx=0x7fc50002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=348b6dda898549db server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6bc49e7640 1 --2- 172.21.15.82:0/1211190803 >> v2:172.21.15.164:3300/0 conn(0x7f6bc015f3b0 0x7f6bc006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7fc5327fc640 1 -- 172.21.15.82:0/1922133520 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c039070 con 0x7fc53415dde0 2024-03-14T12:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7fc5327fc640 1 -- 172.21.15.82:0/1922133520 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c02cbf0 con 0x7fc53415dde0 2024-03-14T12:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7fc5327fc640 1 -- 172.21.15.82:0/1922133520 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c034440 con 0x7fc53415dde0 2024-03-14T12:14:00.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6bbffff640 1 -- 172.21.15.82:0/1211190803 >> v2:172.21.15.164:3300/0 conn(0x7f6bc015f3b0 msgr2=0x7f6bc006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6bbffff640 1 --2- 172.21.15.82:0/1211190803 >> v2:172.21.15.164:3300/0 conn(0x7f6bc015f3b0 0x7f6bc006fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6bbffff640 1 -- 172.21.15.82:0/1211190803 >> v2:172.21.15.82:3300/0 conn(0x7f6bc00b0a80 msgr2=0x7f6bc00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:00.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6bbffff640 1 --2- 172.21.15.82:0/1211190803 >> v2:172.21.15.82:3300/0 conn(0x7f6bc00b0a80 0x7f6bc00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6bbffff640 1 -- 172.21.15.82:0/1211190803 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00d01a0 con 0x7f6bc00cc1f0 2024-03-14T12:14:00.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7fc515180640 1 -- 172.21.15.82:0/2445114942 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000315a0 con 0x7fc510162740 2024-03-14T12:14:00.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6c69f96640 1 --2- 172.21.15.82:0/1887616845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640d2390 0x7f6c640d2770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c6408d5a0 tx=0x7f6c5c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8585b7ae05d1c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f6aa0ea5640 1 --2- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c06d3e0 0x7f6a9c06d860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a980097a0 tx=0x7f6a9802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f294ebd855320e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc515180640 1 -- 172.21.15.82:0/2445114942 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc500037070 con 0x7fc510162740 2024-03-14T12:14:00.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.944+0000 7f33fc81c640 1 --2- 172.21.15.82:0/226735745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f806ad10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009870 tx=0x7f33e802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2faccf57383a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f33effff640 1 -- 172.21.15.82:0/226735745 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8035020 con 0x7f33f81624a0 2024-03-14T12:14:00.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc515180640 1 -- 172.21.15.82:0/2445114942 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500031960 con 0x7fc510162740 2024-03-14T12:14:00.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f0cde59c640 1 --2- 172.21.15.82:0/1325528365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80eb050 0x7f0cd80552f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00097a0 tx=0x7f0cd002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2137d3632b83b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f33effff640 1 -- 172.21.15.82:0/226735745 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e8031640 con 0x7f33f81624a0 2024-03-14T12:14:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f33effff640 1 -- 172.21.15.82:0/226735745 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8035020 con 0x7f33f81624a0 2024-03-14T12:14:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f6a93fff640 1 -- 172.21.15.82:0/3154737939 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a980315a0 con 0x7f6a9c06d3e0 2024-03-14T12:14:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f6c68f94640 1 -- 172.21.15.82:0/1887616845 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c02ca80 con 0x7f6c640d2390 2024-03-14T12:14:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f6c68f94640 1 -- 172.21.15.82:0/1887616845 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c02cc20 con 0x7f6c640d2390 2024-03-14T12:14:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f6c68f94640 1 -- 172.21.15.82:0/1887616845 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0343f0 con 0x7f6c640d2390 2024-03-14T12:14:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f6a93fff640 1 -- 172.21.15.82:0/3154737939 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98037070 con 0x7f6a9c06d3e0 2024-03-14T12:14:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f6a93fff640 1 -- 172.21.15.82:0/3154737939 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98031960 con 0x7f6a9c06d3e0 2024-03-14T12:14:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f0cdd59a640 1 -- 172.21.15.82:0/1325528365 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0031780 con 0x7f0cd80eb050 2024-03-14T12:14:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f0cdd59a640 1 -- 172.21.15.82:0/1325528365 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0037070 con 0x7f0cd80eb050 2024-03-14T12:14:00.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f0cdd59a640 1 -- 172.21.15.82:0/1325528365 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0031b40 con 0x7f0cd80eb050 2024-03-14T12:14:00.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f7e9e59c640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e880039f0 0x7f7e88003e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000c290 tx=0x7f7e88005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b7e86acf1a52ff server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 -- 172.21.15.82:0/1922133520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53415dde0 msgr2=0x7fc53415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 --2- 172.21.15.82:0/1922133520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53415dde0 0x7fc53415e210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc53415edd0 tx=0x7fc52c02be40 comp rx=0 tx=0).stop 2024-03-14T12:14:00.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 -- 172.21.15.82:0/1922133520 shutdown_connections 2024-03-14T12:14:00.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 --2- 172.21.15.82:0/1922133520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc53415dde0 0x7fc53415e210 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 --2- 172.21.15.82:0/1922133520 >> v2:172.21.15.82:3300/0 conn(0x7fc534151230 0x7fc5340cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 -- 172.21.15.82:0/1922133520 >> 172.21.15.82:0/1922133520 conn(0x7fc5340b1c70 msgr2=0x7fc5340cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90039070 con 0x7f7e880039f0 2024-03-14T12:14:00.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9002cae0 con 0x7f7e880039f0 2024-03-14T12:14:00.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f7e9d59a640 1 -- 172.21.15.82:0/4039229324 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90007ce0 con 0x7f7e880039f0 2024-03-14T12:14:00.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 -- 172.21.15.82:0/1922133520 shutdown_connections 2024-03-14T12:14:00.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7fc53a23d640 1 -- 172.21.15.82:0/1922133520 wait complete. 2024-03-14T12:14:00.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f33ff281640 1 -- 172.21.15.82:0/226735745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 msgr2=0x7f33f806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6bbffff640 1 --2- 172.21.15.82:0/1211190803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00cc1f0 0x7f6bc00b0540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0072350 tx=0x7f6bac02fbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c983f786c7eca61 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:00.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.945+0000 7f33ff281640 1 --2- 172.21.15.82:0/226735745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f806ad10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009870 tx=0x7f33e802b8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 -- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c06d3e0 msgr2=0x7f6a9c06d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 --2- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c06d3e0 0x7f6a9c06d860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a980097a0 tx=0x7f6a9802b840 comp rx=0 tx=0).stop 2024-03-14T12:14:00.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f33ff281640 1 -- 172.21.15.82:0/226735745 shutdown_connections 2024-03-14T12:14:00.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f33ff281640 1 --2- 172.21.15.82:0/226735745 >> v2:172.21.15.164:3300/0 conn(0x7f33f806b250 0x7f33f80b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f33ff281640 1 --2- 172.21.15.82:0/226735745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f806ad10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f33ff281640 1 -- 172.21.15.82:0/226735745 >> 172.21.15.82:0/226735745 conn(0x7f33f80b1b10 msgr2=0x7f33f80b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 -- 172.21.15.82:0/3154737939 shutdown_connections 2024-03-14T12:14:00.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 -- 172.21.15.82:0/1325528365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80eb050 msgr2=0x7f0cd80552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 --2- 172.21.15.82:0/1325528365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80eb050 0x7f0cd80552f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00097a0 tx=0x7f0cd002b820 comp rx=0 tx=0).stop 2024-03-14T12:14:00.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/1887616845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640d2390 msgr2=0x7f6c640d2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6c6c1fa640 1 --2- 172.21.15.82:0/1887616845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640d2390 0x7f6c640d2770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c6408d5a0 tx=0x7f6c5c02bee0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7fc5183e6640 1 -- 172.21.15.82:0/2445114942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510162740 msgr2=0x7fc51006c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7fc5183e6640 1 --2- 172.21.15.82:0/2445114942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510162740 0x7fc51006c3b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5000097a0 tx=0x7fc50002b840 comp rx=0 tx=0).stop 2024-03-14T12:14:00.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 --2- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c06d3e0 0x7f6a9c06d860 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a980097a0 tx=0x7f6a9802b840 comp rx=0 tx=0).stop 2024-03-14T12:14:00.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 --2- 172.21.15.82:0/3154737939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 -- 172.21.15.82:0/3154737939 >> 172.21.15.82:0/3154737939 conn(0x7f6a9c0b1560 msgr2=0x7f6a9c0b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 -- 172.21.15.82:0/3154737939 shutdown_connections 2024-03-14T12:14:00.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6bbf7fe640 1 -- 172.21.15.82:0/1211190803 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac03d070 con 0x7f6bc00cc1f0 2024-03-14T12:14:00.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6bbf7fe640 1 -- 172.21.15.82:0/1211190803 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac030970 con 0x7f6bc00cc1f0 2024-03-14T12:14:00.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6aa26a8640 1 -- 172.21.15.82:0/3154737939 wait complete. 2024-03-14T12:14:00.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6bbf7fe640 1 -- 172.21.15.82:0/1211190803 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac03d070 con 0x7f6bc00cc1f0 2024-03-14T12:14:00.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 -- 172.21.15.82:0/1325528365 shutdown_connections 2024-03-14T12:14:00.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 --2- 172.21.15.82:0/1325528365 >> v2:172.21.15.82:3300/0 conn(0x7f0cd80eb430 0x7f0cd80eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 --2- 172.21.15.82:0/1325528365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80eb050 0x7f0cd80552f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 -- 172.21.15.82:0/1325528365 >> 172.21.15.82:0/1325528365 conn(0x7f0cd80559d0 msgr2=0x7f0cd8055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 -- 172.21.15.82:0/1325528365 shutdown_connections 2024-03-14T12:14:00.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f0cdf59e640 1 -- 172.21.15.82:0/1325528365 wait complete. 2024-03-14T12:14:00.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f33ff281640 1 -- 172.21.15.82:0/226735745 shutdown_connections 2024-03-14T12:14:00.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7fc5183e6640 1 -- 172.21.15.82:0/2445114942 shutdown_connections 2024-03-14T12:14:00.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7fc5183e6640 1 --2- 172.21.15.82:0/2445114942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc510162740 0x7fc51006c3b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7fc5183e6640 1 -- 172.21.15.82:0/2445114942 >> 172.21.15.82:0/2445114942 conn(0x7fc5100b1c70 msgr2=0x7fc5100cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f7e9f59e640 1 -- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e880039f0 msgr2=0x7f7e88003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f7e9f59e640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e880039f0 0x7f7e88003e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000c290 tx=0x7f7e88005830 comp rx=0 tx=0).stop 2024-03-14T12:14:00.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/1887616845 shutdown_connections 2024-03-14T12:14:00.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7fc5183e6640 1 -- 172.21.15.82:0/2445114942 shutdown_connections 2024-03-14T12:14:00.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6c6c1fa640 1 --2- 172.21.15.82:0/1887616845 >> v2:172.21.15.164:3300/0 conn(0x7f6c640c9520 0x7f6c640cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f33ff281640 1 -- 172.21.15.82:0/226735745 wait complete. 2024-03-14T12:14:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6c6c1fa640 1 --2- 172.21.15.82:0/1887616845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640d2390 0x7f6c640d2770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.946+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/1887616845 >> 172.21.15.82:0/1887616845 conn(0x7f6c640d3db0 msgr2=0x7f6c640d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f7e9f59e640 1 -- 172.21.15.82:0/4039229324 shutdown_connections 2024-03-14T12:14:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f7e9f59e640 1 --2- 172.21.15.82:0/4039229324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e880039f0 0x7f7e88003e10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f7e9f59e640 1 -- 172.21.15.82:0/4039229324 >> 172.21.15.82:0/4039229324 conn(0x7f7e98009fb0 msgr2=0x7f7e9800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7fc5183e6640 1 -- 172.21.15.82:0/2445114942 wait complete. 2024-03-14T12:14:00.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f7e9f59e640 1 -- 172.21.15.82:0/4039229324 shutdown_connections 2024-03-14T12:14:00.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7fc53a23d640 1 Processor -- start 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f6bc59e9640 1 -- 172.21.15.82:0/1211190803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00cc1f0 msgr2=0x7f6bc00b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f6bc59e9640 1 --2- 172.21.15.82:0/1211190803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00cc1f0 0x7f6bc00b0540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0072350 tx=0x7f6bac02fbc0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f6bc59e9640 1 -- 172.21.15.82:0/1211190803 shutdown_connections 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f6bc59e9640 1 --2- 172.21.15.82:0/1211190803 >> v2:172.21.15.82:3300/0 conn(0x7f6bc00b0a80 0x7f6bc00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f6bc59e9640 1 --2- 172.21.15.82:0/1211190803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00cc1f0 0x7f6bc00b0540 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0072350 tx=0x7f6bac02fbc0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f6bc59e9640 1 --2- 172.21.15.82:0/1211190803 >> v2:172.21.15.164:3300/0 conn(0x7f6bc015f3b0 0x7f6bc006fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f6bc59e9640 1 -- 172.21.15.82:0/1211190803 >> 172.21.15.82:0/1211190803 conn(0x7f6bc00b33b0 msgr2=0x7f6bc00b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:00.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7f7e9f59e640 1 -- 172.21.15.82:0/4039229324 wait complete. 2024-03-14T12:14:00.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.947+0000 7fc53a23d640 1 -- start start 2024-03-14T12:14:00.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f6aa26a8640 1 Processor -- start 2024-03-14T12:14:00.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/1887616845 shutdown_connections 2024-03-14T12:14:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f0cdf59e640 1 Processor -- start 2024-03-14T12:14:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f0cdf59e640 1 -- start start 2024-03-14T12:14:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f33ff281640 1 Processor -- start 2024-03-14T12:14:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f33ff281640 1 -- start start 2024-03-14T12:14:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f6aa26a8640 1 -- start start 2024-03-14T12:14:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.948+0000 7f6bc59e9640 1 -- 172.21.15.82:0/1211190803 shutdown_connections 2024-03-14T12:14:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f6bc59e9640 1 -- 172.21.15.82:0/1211190803 wait complete. 2024-03-14T12:14:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc5183e6640 1 Processor -- start 2024-03-14T12:14:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc5183e6640 1 -- start start 2024-03-14T12:14:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f6aa26a8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc53a23d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5340b4b70 0x7fc5341e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f7e9f59e640 1 Processor -- start 2024-03-14T12:14:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/1887616845 wait complete. 2024-03-14T12:14:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f6aa26a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c071320 0x7f6a9c140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f0cdf59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80e7a60 0x7f0cd8052cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f0cdf59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cd80eb050 0x7f0cd800b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f0cdf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd80eb430 0x7f0cd804ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f0cdf59e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd8057620 con 0x7f0cd80eb050 2024-03-14T12:14:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f0cdf59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd8057340 con 0x7f0cd80eb430 2024-03-14T12:14:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f0cdf59e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80e1e00 con 0x7f0cd80e7a60 2024-03-14T12:14:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5340b4b70 0x7fc5341e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5340b4b70 0x7fc5341e3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48694/0 (socket says 172.21.15.82:48694) 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc5337fe640 1 -- 172.21.15.82:0/2513120012 learned_addr learned my addr 172.21.15.82:0/2513120012 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc53a23d640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc534151230 0x7fc5341403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7fc53a23d640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc53415dde0 0x7fc5341e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53415e750 con 0x7fc5340b4b70 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f33ff281640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33f806b250 0x7f33f809daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f33ff281640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f81620c0 0x7f33f80d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f33ff281640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f809c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f33ff281640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8166b30 con 0x7f33f81620c0 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f33ff281640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8166850 con 0x7f33f806b250 2024-03-14T12:14:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f33ff281640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80ce330 con 0x7f33f81624a0 2024-03-14T12:14:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.949+0000 7f7e9f59e640 1 -- start start 2024-03-14T12:14:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cddd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cd80eb050 0x7f0cd800b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cddd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cd80eb050 0x7f0cd800b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48714/0 (socket says 172.21.15.82:48714) 2024-03-14T12:14:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cddd9b640 1 -- 172.21.15.82:0/3776194059 learned_addr learned my addr 172.21.15.82:0/3776194059 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33f806b250 0x7f33f809daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc533fff640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc53415dde0 0x7fc5341e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53406c690 con 0x7fc534151230 2024-03-14T12:14:00.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc534151610 con 0x7fc53415dde0 2024-03-14T12:14:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc5183e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc51006c8f0 0x7fc5100a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc5183e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc510162740 0x7fc5101f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc5183e6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5101661d0 0x7fc5101fac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc5183e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc510071290 con 0x7fc510162740 2024-03-14T12:14:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc5183e6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100706f0 con 0x7fc51006c8f0 2024-03-14T12:14:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc5183e6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc510071410 con 0x7fc5101661d0 2024-03-14T12:14:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa26a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c1427c0 0x7f6a9c142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33f806b250 0x7f33f809daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:58398/0 (socket says 172.21.15.82:58398) 2024-03-14T12:14:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 -- 172.21.15.82:0/258178992 learned_addr learned my addr 172.21.15.82:0/258178992 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd81e640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f809c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fc81c640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f81620c0 0x7f33f80d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa26a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0cd620 con 0x7f6a9c071320 2024-03-14T12:14:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7fc532ffd640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc534151230 0x7fc5341403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa16a6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa16a6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c144890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:52934/0 (socket says 172.21.15.82:52934) 2024-03-14T12:14:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa16a6640 1 -- 172.21.15.82:0/2803235283 learned_addr learned my addr 172.21.15.82:0/2803235283 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:00.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 -- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 msgr2=0x7f33f809c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cded9d640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd80eb430 0x7f0cd804ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33f81624a0 0x7f33f809c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 -- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f81620c0 msgr2=0x7f33f80d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:00.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f81620c0 0x7f33f80d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:00.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f7e9f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e880039f0 0x7f7e980defe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f7e9f59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e9804de20 0x7f7e9804a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f7e9f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e9804ac90 0x7f7e98047060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:00.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f7e9f59e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98054940 con 0x7f7e880039f0 2024-03-14T12:14:00.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f33fd01d640 1 -- 172.21.15.82:0/258178992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f809c7d0 con 0x7f33f806b250 2024-03-14T12:14:00.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f7e9f59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980547c0 con 0x7f7e9804ac90 2024-03-14T12:14:00.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f7e9f59e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9804f550 con 0x7f7e9804de20 2024-03-14T12:14:00.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa0ea5640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c071320 0x7f6a9c140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:00.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6bc59e9640 1 Processor -- start 2024-03-14T12:14:00.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cded9d640 1 -- 172.21.15.82:0/3776194059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80e7a60 msgr2=0x7f0cd8052cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cded9d640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cd80e7a60 0x7f0cd8052cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cded9d640 1 -- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cd80eb050 msgr2=0x7f0cd800b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cded9d640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cd80eb050 0x7f0cd800b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f0cded9d640 1 -- 172.21.15.82:0/3776194059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd800a0b0 con 0x7f0cd80eb430 2024-03-14T12:14:01.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0cd340 con 0x7f6a9c1427c0 2024-03-14T12:14:01.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516983640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5101661d0 0x7fc5101fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6bc59e9640 1 -- start start 2024-03-14T12:14:01.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.950+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0cd900 con 0x7f6a9c06e620 2024-03-14T12:14:01.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516983640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5101661d0 0x7fc5101fac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:52964/0 (socket says 172.21.15.82:52964) 2024-03-14T12:14:01.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516983640 1 -- 172.21.15.82:0/1587037449 learned_addr learned my addr 172.21.15.82:0/1587037449 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516182640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc51006c8f0 0x7fc5100a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f33fd01d640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33f806b250 0x7f33f809daf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33f000c7a0 tx=0x7f33f000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc532ffd640 1 -- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc53415dde0 msgr2=0x7fc5341e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc532ffd640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc53415dde0 0x7fc5341e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6aa1ea7640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c1427c0 0x7f6a9c142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f0cded9d640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd80eb430 0x7f0cd804ff90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0000b60 tx=0x7f0ce0069e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6c6c1fa640 1 Processor -- start 2024-03-14T12:14:01.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc532ffd640 1 -- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5340b4b70 msgr2=0x7fc5341e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc532ffd640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5340b4b70 0x7fc5341e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6c6c1fa640 1 -- start start 2024-03-14T12:14:01.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc532ffd640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5341e4aa0 con 0x7fc534151230 2024-03-14T12:14:01.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e9804ac90 0x7f7e98047060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e9804de20 0x7f7e9804a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc5337fe640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5340b4b70 0x7fc5341e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:01.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e9804de20 0x7f7e9804a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:52982/0 (socket says 172.21.15.82:52982) 2024-03-14T12:14:01.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 -- 172.21.15.82:0/1531977024 learned_addr learned my addr 172.21.15.82:0/1531977024 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0017070 con 0x7f33f806b250 2024-03-14T12:14:01.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516182640 1 -- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5101661d0 msgr2=0x7fc5101fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516182640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5101661d0 0x7fc5101fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516182640 1 -- 172.21.15.82:0/1587037449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc510162740 msgr2=0x7fc5101f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516182640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc510162740 0x7fc5101f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516182640 1 -- 172.21.15.82:0/1587037449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100a9980 con 0x7fc51006c8f0 2024-03-14T12:14:01.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc516983640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5101661d0 0x7fc5101fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:01.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f809ca50 con 0x7f33f806b250 2024-03-14T12:14:01.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f0cceffd640 1 -- 172.21.15.82:0/3776194059 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0090070 con 0x7f0cd80eb430 2024-03-14T12:14:01.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f0cceffd640 1 -- 172.21.15.82:0/3776194059 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0073260 con 0x7f0cd80eb430 2024-03-14T12:14:01.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f0cceffd640 1 -- 172.21.15.82:0/3776194059 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce004f570 con 0x7f0cd80eb430 2024-03-14T12:14:01.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f0cdf59e640 1 -- 172.21.15.82:0/3776194059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd800a390 con 0x7f0cd80eb430 2024-03-14T12:14:01.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6aa16a6640 1 -- 172.21.15.82:0/2803235283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c1427c0 msgr2=0x7f6a9c142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6aa16a6640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a9c1427c0 0x7f6a9c142c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6aa16a6640 1 -- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c071320 msgr2=0x7f6a9c140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6aa16a6640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c071320 0x7f6a9c140b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f6aa16a6640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c156240 con 0x7f6a9c06e620 2024-03-14T12:14:01.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f809d010 con 0x7f33f806b250 2024-03-14T12:14:01.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7fc532ffd640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc534151230 0x7fc5341403d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c450 tx=0x7fc52800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f0003da0 con 0x7f33f806b250 2024-03-14T12:14:01.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00052b0 con 0x7f33f806b250 2024-03-14T12:14:01.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f0cdf59e640 1 -- 172.21.15.82:0/3776194059 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd800a950 con 0x7f0cd80eb430 2024-03-14T12:14:01.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9e59c640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e880039f0 0x7f7e980defe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 -- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e9804ac90 msgr2=0x7f7e98047060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e9804ac90 0x7f7e98047060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6aa16a6640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c144890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00e9b0 tx=0x7f6a8c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 -- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e880039f0 msgr2=0x7f7e980defe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e880039f0 0x7f7e980defe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.951+0000 7f7e9dd9b640 1 -- 172.21.15.82:0/1531977024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980df520 con 0x7f7e9804de20 2024-03-14T12:14:01.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f7e9dd9b640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e9804de20 0x7f7e9804a750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7e94004650 tx=0x7f7e9400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7fc516182640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc51006c8f0 0x7fc5100a9440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00b3e0 tx=0x7fc50c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6c6c1fa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640c9520 0x7f6c6415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c017070 con 0x7f6a9c06e620 2024-03-14T12:14:01.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c00c9a0 con 0x7f6a9c06e620 2024-03-14T12:14:01.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6bc59e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00b0a80 0x7f6bc00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00cd90 con 0x7f6a9c06e620 2024-03-14T12:14:01.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6bc59e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bc015f3b0 0x7f6bc00c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c156520 con 0x7f6a9c06e620 2024-03-14T12:14:01.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f6bc59e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bc00c8950 0x7f6bc00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94017070 con 0x7f7e9804de20 2024-03-14T12:14:01.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528018070 con 0x7fc534151230 2024-03-14T12:14:01.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc49e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00b0a80 0x7f6bc00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400c940 con 0x7f7e9804de20 2024-03-14T12:14:01.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.952+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cd30 con 0x7f7e9804de20 2024-03-14T12:14:01.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5280074e0 con 0x7fc534151230 2024-03-14T12:14:01.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c149220 con 0x7f6a9c06e620 2024-03-14T12:14:01.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc49e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00b0a80 0x7f6bc00b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:52994/0 (socket says 172.21.15.82:52994) 2024-03-14T12:14:01.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc51e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bc00c8950 0x7f6bc00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc51e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bc00c8950 0x7f6bc00c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48744/0 (socket says 172.21.15.82:48744) 2024-03-14T12:14:01.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc51e8640 1 -- 172.21.15.82:0/3449709542 learned_addr learned my addr 172.21.15.82:0/3449709542 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c015030 con 0x7fc51006c8f0 2024-03-14T12:14:01.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7fc5183e6640 1 -- 172.21.15.82:0/1587037449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51015df50 con 0x7fc51006c8f0 2024-03-14T12:14:01.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7fc5183e6640 1 -- 172.21.15.82:0/1587037449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51015e510 con 0x7fc51006c8f0 2024-03-14T12:14:01.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c004020 con 0x7fc51006c8f0 2024-03-14T12:14:01.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c004450 con 0x7fc51006c8f0 2024-03-14T12:14:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00cc650 con 0x7f6bc00c8950 2024-03-14T12:14:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0070140 con 0x7f6bc015f3b0 2024-03-14T12:14:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f0cceffd640 1 -- 172.21.15.82:0/3776194059 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0ce0093070 con 0x7f0cd80eb430 2024-03-14T12:14:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00cc7d0 con 0x7f6bc00b0a80 2024-03-14T12:14:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.953+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f33d8001030 con 0x7f33f806b250 2024-03-14T12:14:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6bbffff640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bc015f3b0 0x7f6bc00c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5341e4d80 con 0x7fc534151230 2024-03-14T12:14:01.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5341e5340 con 0x7fc534151230 2024-03-14T12:14:01.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9804cd80 con 0x7f7e9804de20 2024-03-14T12:14:01.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9804d340 con 0x7f7e9804de20 2024-03-14T12:14:01.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528004a00 con 0x7fc534151230 2024-03-14T12:14:01.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f0cceffd640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0cc006bc20 0x7f0cc006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f0cceffd640 1 -- 172.21.15.82:0/3776194059 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ce0105170 con 0x7f0cd80eb430 2024-03-14T12:14:01.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6bc49e7640 1 -- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bc015f3b0 msgr2=0x7f6bc00c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6bc49e7640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bc015f3b0 0x7f6bc00c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6bc49e7640 1 -- 172.21.15.82:0/3449709542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bc00c8950 msgr2=0x7f6bc00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f0cde59c640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0cc006bc20 0x7f0cc006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6bc49e7640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bc00c8950 0x7f6bc00c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6bc49e7640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00ca460 con 0x7f6bc00b0a80 2024-03-14T12:14:01.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6a8c01a070 con 0x7f6a9c06e620 2024-03-14T12:14:01.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6bc49e7640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00b0a80 0x7f6bc00b48a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400d620 tx=0x7f6bb400dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a9c068610 con 0x7f6a9c06e620 2024-03-14T12:14:01.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e9804da00 con 0x7f7e9804de20 2024-03-14T12:14:01.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400b620 con 0x7f6bc00b0a80 2024-03-14T12:14:01.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb4019070 con 0x7f6bc00b0a80 2024-03-14T12:14:01.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400ba00 con 0x7f6bc00b0a80 2024-03-14T12:14:01.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc00c5720 con 0x7f6bc00b0a80 2024-03-14T12:14:01.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc00c5fc0 con 0x7f6bc00b0a80 2024-03-14T12:14:01.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc53404f050 con 0x7fc534151230 2024-03-14T12:14:01.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7fc5183e6640 1 -- 172.21.15.82:0/1587037449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4f8001030 con 0x7fc51006c8f0 2024-03-14T12:14:01.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f0cdf59e640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7f0cd800d8b0 0x7f0cd80504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f0cddd9b640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7f0cd800d8b0 0x7f0cd80504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6a91ffb640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a8006bca0 0x7f6a8006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a8c08aa50 con 0x7f6a9c06e620 2024-03-14T12:14:01.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f6a91ffb640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f6a80073ea0 0x7f6a800762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a80076970 con 0x7f6a80073ea0 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6c6c1fa640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c640d1b90 0x7f6c640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6c6c1fa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c640d2390 0x7f6c640b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6c6c1fa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c7ff0 con 0x7f6c640c9520 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6c6c1fa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c7d10 con 0x7f6c640d2390 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.954+0000 7f6c6c1fa640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640ca590 con 0x7f6c640d1b90 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69795640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c640d1b90 0x7f6c640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640c9520 0x7f6c6415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640c9520 0x7f6c6415e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48754/0 (socket says 172.21.15.82:48754) 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 -- 172.21.15.82:0/124038325 learned_addr learned my addr 172.21.15.82:0/124038325 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 -- 172.21.15.82:0/124038325 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c640d1b90 msgr2=0x7f6c640b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6c640d1b90 0x7f6c640b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 -- 172.21.15.82:0/124038325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c640d2390 msgr2=0x7f6c640b90d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c640d2390 0x7f6c640b90d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f6c69f96640 1 -- 172.21.15.82:0/124038325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c641605c0 con 0x7f6c640c9520 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f6c69f96640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640c9520 0x7f6c6415e710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c036040 tx=0x7f6c5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8c091050 con 0x7f6a9c06e620 2024-03-14T12:14:01.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f0cddd9b640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7f0cd800d8b0 0x7f0cd80504d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f0cdf59e640 1 -- 172.21.15.82:0/3776194059 --> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cd80057d0 con 0x7f0cd800d8b0 2024-03-14T12:14:01.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f0cde59c640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0cc006bc20 0x7f0cc006e0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0036000 tx=0x7f0cd0030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f6aa0ea5640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a8006bca0 0x7f6a8006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f6aa1ea7640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f6a80073ea0 0x7f6a800762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f6aa0ea5640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a8006bca0 0x7f6a8006e160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a98036000 tx=0x7f6a9803d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ba0001030 con 0x7f6bc00b0a80 2024-03-14T12:14:01.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.955+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f33f001a070 con 0x7f33f806b250 2024-03-14T12:14:01.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f33edffb640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33dc06bbd0 0x7f33dc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33f008a8d0 con 0x7f33f806b250 2024-03-14T12:14:01.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f33edffb640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f33dc073df0 0x7f33dc076210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 --> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33dc0768c0 con 0x7f33dc073df0 2024-03-14T12:14:01.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f008acc0 con 0x7f33f806b250 2024-03-14T12:14:01.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.956+0000 7f33fd81e640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f33dc073df0 0x7f33dc076210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c043020 con 0x7f6c640c9520 2024-03-14T12:14:01.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c03e450 con 0x7f6c640c9520 2024-03-14T12:14:01.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6bb400bba0 con 0x7f6bc00b0a80 2024-03-14T12:14:01.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc50c018070 con 0x7fc51006c8f0 2024-03-14T12:14:01.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7fc506ffd640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc4f406bca0 0x7fc4f406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7fc515981640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc4f406bca0 0x7fc4f406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc50c08a440 con 0x7fc51006c8f0 2024-03-14T12:14:01.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c03e820 con 0x7f6c640c9520 2024-03-14T12:14:01.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.957+0000 7f33fc81c640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33dc06bbd0 0x7f33dc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f33fd81e640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f33dc073df0 0x7f33dc076210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f0cceffd640 1 -- 172.21.15.82:0/3776194059 <== osd.0 v2:172.21.15.82:6800/3139126855 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0cd80057d0 con 0x7f0cd800d8b0 2024-03-14T12:14:01.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f33fc81c640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33dc06bbd0 0x7f33dc06e090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33e8036000 tx=0x7f33e803c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc52801b070 con 0x7fc534151230 2024-03-14T12:14:01.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f6bbd7fa640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6ba406bca0 0x7f6ba406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6415f910 con 0x7f6c640c9520 2024-03-14T12:14:01.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7e9401a070 con 0x7f7e9804de20 2024-03-14T12:14:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6415fed0 con 0x7f6c640c9520 2024-03-14T12:14:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc515981640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc4f406bca0 0x7fc4f406e160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc500036000 tx=0x7fc500030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc506ffd640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7fc4f4073f20 0x7fc4f4076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 --> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc50c00fe50 con 0x7fc4f4073f20 2024-03-14T12:14:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc50c091050 con 0x7fc51006c8f0 2024-03-14T12:14:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f6bbffff640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6ba406bca0 0x7f6ba406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc51bfff640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc52006bc20 0x7fc52006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f7e8effd640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7e8006bca0 0x7f7e8006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc52808a380 con 0x7fc534151230 2024-03-14T12:14:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e9408a800 con 0x7f7e9804de20 2024-03-14T12:14:01.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bb408adc0 con 0x7f6bc00b0a80 2024-03-14T12:14:01.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc51bfff640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7fc520073ea0 0x7fc5200762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5280074e0 con 0x7fc520073ea0 2024-03-14T12:14:01.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f7e8effd640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f7e80073ec0 0x7f7e800762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f6bbd7fa640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f6ba4073ec0 0x7f6ba40762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 --> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e94001d30 con 0x7f7e80073ec0 2024-03-14T12:14:01.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e94091050 con 0x7f7e9804de20 2024-03-14T12:14:01.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc52808a730 con 0x7fc534151230 2024-03-14T12:14:01.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7fc5337fe640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc52006bc20 0x7fc52006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bb400fe50 con 0x7f6ba4073ec0 2024-03-14T12:14:01.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== osd.6 v2:172.21.15.164:6808/3403808739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f33dc0768c0 con 0x7f33dc073df0 2024-03-14T12:14:01.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb408d420 con 0x7f6bc00b0a80 2024-03-14T12:14:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f6bbffff640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6ba406bca0 0x7f6ba406e160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bac03a000 tx=0x7f6bac034000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.958+0000 7fc516983640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7fc4f4073f20 0x7fc4f4076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c6404f050 con 0x7f6c640c9520 2024-03-14T12:14:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f6bc51e8640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f6ba4073ec0 0x7f6ba40762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f7e9ed9d640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f7e80073ec0 0x7f7e800762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f6aa1ea7640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f6a80073ea0 0x7f6a800762c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7fc5337fe640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc52006bc20 0x7fc52006e0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc524006d40 tx=0x7fc524008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7f7e9e59c640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7e8006bca0 0x7f7e8006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7fc533fff640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7fc520073ea0 0x7fc5200762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.959+0000 7fc516983640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7fc4f4073f20 0x7fc4f4076340 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.960+0000 7f7e9ed9d640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f7e80073ec0 0x7f7e800762e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.960+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== osd.3 v2:172.21.15.82:6824/994456991 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6a80076970 con 0x7f6a80073ea0 2024-03-14T12:14:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.960+0000 7fc533fff640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7fc520073ea0 0x7fc5200762c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.960+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6c5c046020 con 0x7f6c640c9520 2024-03-14T12:14:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.960+0000 7f7e9e59c640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7e8006bca0 0x7f7e8006e160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036000 tx=0x7f7e90030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.960+0000 7f6bc51e8640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f6ba4073ec0 0x7f6ba40762e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.960+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== osd.5 v2:172.21.15.164:6817/2718563770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc50c00fe50 con 0x7fc4f4073f20 2024-03-14T12:14:01.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7f6c5a7fc640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c4806bc20 0x7f6c4806e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== osd.7 v2:172.21.15.164:6816/369064365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7e94001d30 con 0x7f7e80073ec0 2024-03-14T12:14:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== osd.2 v2:172.21.15.82:6808/2455719014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc5280074e0 con 0x7fc520073ea0 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c5c0ad070 con 0x7f6c640c9520 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7f6c5a7fc640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7f6c48073dc0 0x7f6c480761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 --> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c48076890 con 0x7f6c48073dc0 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7f6c6a797640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7f6c48073dc0 0x7f6c480761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.961+0000 7f6c69795640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c4806bc20 0x7f6c4806e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.962+0000 7f6c69795640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c4806bc20 0x7f6c4806e0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ce170 tx=0x7f6c60009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.962+0000 7f6c6a797640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7f6c48073dc0 0x7f6c480761e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.962+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c5c077870 con 0x7f6c640c9520 2024-03-14T12:14:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.962+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== osd.4 v2:172.21.15.164:6800/2779428821 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6bb400fe50 con 0x7f6ba4073ec0 2024-03-14T12:14:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.963+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== osd.1 v2:172.21.15.82:6816/2003973578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6c48076890 con 0x7f6c48073dc0 2024-03-14T12:14:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.989+0000 7f0cdf59e640 1 -- 172.21.15.82:0/3776194059 --> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0cd8038280 con 0x7f0cd800d8b0 2024-03-14T12:14:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.990+0000 7f0cceffd640 1 -- 172.21.15.82:0/3776194059 <== osd.0 v2:172.21.15.82:6800/3139126855 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0cd8038280 con 0x7f0cd800d8b0 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.990+0000 7f0cccff9640 1 -- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7f0cd800d8b0 msgr2=0x7f0cd80504d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.990+0000 7f0cccff9640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7f0cd800d8b0 0x7f0cd80504d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.991+0000 7f0cccff9640 1 -- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0cc006bc20 msgr2=0x7f0cc006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.991+0000 7f0cccff9640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0cc006bc20 0x7f0cc006e0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0036000 tx=0x7f0cd0030000 comp rx=0 tx=0).stop 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.991+0000 7f0cccff9640 1 -- 172.21.15.82:0/3776194059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd80eb430 msgr2=0x7f0cd804ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.991+0000 7f0cccff9640 1 --2- 172.21.15.82:0/3776194059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd80eb430 0x7f0cd804ff90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0000b60 tx=0x7f0ce0069e60 comp rx=0 tx=0).stop 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.991+0000 7f0cded9d640 1 -- 172.21.15.82:0/3776194059 reap_dead start 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.991+0000 7f0cccff9640 1 -- 172.21.15.82:0/3776194059 shutdown_connections 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.991+0000 7f0cccff9640 1 -- 172.21.15.82:0/3776194059 >> 172.21.15.82:0/3776194059 conn(0x7f0cd80559d0 msgr2=0x7f0cd80e8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.992+0000 7f0cccff9640 1 -- 172.21.15.82:0/3776194059 shutdown_connections 2024-03-14T12:14:01.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.992+0000 7f0cccff9640 1 -- 172.21.15.82:0/3776194059 wait complete. 2024-03-14T12:14:01.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.992+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 --> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e98005390 con 0x7f7e80073ec0 2024-03-14T12:14:01.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.992+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 --> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc5341e5790 con 0x7fc520073ea0 2024-03-14T12:14:01.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.992+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 --> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a9c075210 con 0x7f6a80073ea0 2024-03-14T12:14:01.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.993+0000 7fc51bfff640 1 -- 172.21.15.82:0/2513120012 <== osd.2 v2:172.21.15.82:6808/2455719014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc5341e5790 con 0x7fc520073ea0 2024-03-14T12:14:01.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.993+0000 7f7e8effd640 1 -- 172.21.15.82:0/1531977024 <== osd.7 v2:172.21.15.164:6816/369064365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7e98005390 con 0x7f7e80073ec0 2024-03-14T12:14:01.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.993+0000 7f6a91ffb640 1 -- 172.21.15.82:0/2803235283 <== osd.3 v2:172.21.15.82:6824/994456991 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a9c075210 con 0x7f6a80073ea0 2024-03-14T12:14:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.993+0000 7fc5183e6640 1 -- 172.21.15.82:0/1587037449 --> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc4f8002dd0 con 0x7fc4f4073f20 2024-03-14T12:14:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.993+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7fc520073ea0 msgr2=0x7fc5200762c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.993+0000 7fc53a23d640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7fc520073ea0 0x7fc5200762c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f6a80073ea0 msgr2=0x7f6a800762c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6aa26a8640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f6a80073ea0 0x7f6a800762c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f7e80073ec0 msgr2=0x7f7e800762e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f7e80073ec0 0x7f7e800762e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 --> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f33d8002dd0 con 0x7f33dc073df0 2024-03-14T12:14:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc52006bc20 msgr2=0x7fc52006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc53a23d640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc52006bc20 0x7fc52006e0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc524006d40 tx=0x7fc524008040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc534151230 msgr2=0x7fc5341403d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc53a23d640 1 --2- 172.21.15.82:0/2513120012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc534151230 0x7fc5341403d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c450 tx=0x7fc52800c910 comp rx=0 tx=0).stop 2024-03-14T12:14:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc506ffd640 1 -- 172.21.15.82:0/1587037449 <== osd.5 v2:172.21.15.164:6817/2718563770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc4f8002dd0 con 0x7fc4f4073f20 2024-03-14T12:14:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc533fff640 1 -- 172.21.15.82:0/2513120012 reap_dead start 2024-03-14T12:14:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 --> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c6409ea70 con 0x7f6c48073dc0 2024-03-14T12:14:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a8006bca0 msgr2=0x7f6a8006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6aa26a8640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a8006bca0 0x7f6a8006e160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a98036000 tx=0x7f6a9803d040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 msgr2=0x7f6a9c144890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6aa26a8640 1 --2- 172.21.15.82:0/2803235283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a9c06e620 0x7f6a9c144890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00e9b0 tx=0x7f6a8c00ee70 comp rx=0 tx=0).stop 2024-03-14T12:14:01.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6aa1ea7640 1 -- 172.21.15.82:0/2803235283 reap_dead start 2024-03-14T12:14:01.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 --> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6ba0002dd0 con 0x7f6ba4073ec0 2024-03-14T12:14:01.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f33edffb640 1 -- 172.21.15.82:0/258178992 <== osd.6 v2:172.21.15.164:6808/3403808739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f33d8002dd0 con 0x7f33dc073df0 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7e8006bca0 msgr2=0x7f7e8006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7e8006bca0 0x7f7e8006e160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036000 tx=0x7f7e90030000 comp rx=0 tx=0).stop 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e9804de20 msgr2=0x7f7e9804a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 --2- 172.21.15.82:0/1531977024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e9804de20 0x7f7e9804a750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7e94004650 tx=0x7f7e9400ec90 comp rx=0 tx=0).stop 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9ed9d640 1 -- 172.21.15.82:0/1531977024 reap_dead start 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 shutdown_connections 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 >> 172.21.15.82:0/1531977024 conn(0x7f7e98009fb0 msgr2=0x7f7e98036430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 shutdown_connections 2024-03-14T12:14:01.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6c5a7fc640 1 -- 172.21.15.82:0/124038325 <== osd.1 v2:172.21.15.82:6816/2003973578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6c6409ea70 con 0x7f6c48073dc0 2024-03-14T12:14:01.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f7e9f59e640 1 -- 172.21.15.82:0/1531977024 wait complete. 2024-03-14T12:14:01.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 shutdown_connections 2024-03-14T12:14:01.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.994+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 >> 172.21.15.82:0/2513120012 conn(0x7fc5340b1c70 msgr2=0x7fc53415a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6bbd7fa640 1 -- 172.21.15.82:0/3449709542 <== osd.4 v2:172.21.15.164:6800/2779428821 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6ba0002dd0 con 0x7f6ba4073ec0 2024-03-14T12:14:01.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 shutdown_connections 2024-03-14T12:14:01.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 shutdown_connections 2024-03-14T12:14:01.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 >> 172.21.15.82:0/2803235283 conn(0x7f6a9c0b1560 msgr2=0x7f6a9c14b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 -- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7fc4f4073f20 msgr2=0x7fc4f4076340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7fc4f4073f20 0x7fc4f4076340 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc53a23d640 1 -- 172.21.15.82:0/2513120012 wait complete. 2024-03-14T12:14:01.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f33dc073df0 msgr2=0x7f33dc076210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f33ff281640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f33dc073df0 0x7f33dc076210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 -- 172.21.15.82:0/1587037449 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc4f406bca0 msgr2=0x7fc4f406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc4f406bca0 0x7fc4f406e160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc500036000 tx=0x7fc500030000 comp rx=0 tx=0).stop 2024-03-14T12:14:01.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 shutdown_connections 2024-03-14T12:14:01.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 -- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc51006c8f0 msgr2=0x7fc5100a9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 --2- 172.21.15.82:0/1587037449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc51006c8f0 0x7fc5100a9440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00b3e0 tx=0x7fc50c00b8a0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc516983640 1 -- 172.21.15.82:0/1587037449 reap_dead start 2024-03-14T12:14:01.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6aa26a8640 1 -- 172.21.15.82:0/2803235283 wait complete. 2024-03-14T12:14:01.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7f6c48073dc0 msgr2=0x7f6c480761e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6c6c1fa640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7f6c48073dc0 0x7f6c480761e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33dc06bbd0 msgr2=0x7f33dc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f33ff281640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33dc06bbd0 0x7f33dc06e090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33e8036000 tx=0x7f33e803c040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 -- 172.21.15.82:0/1587037449 shutdown_connections 2024-03-14T12:14:01.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7fc504ff9640 1 -- 172.21.15.82:0/1587037449 >> 172.21.15.82:0/1587037449 conn(0x7fc5100b1c70 msgr2=0x7fc5100717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33f806b250 msgr2=0x7f33f809daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f33ff281640 1 --2- 172.21.15.82:0/258178992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33f806b250 0x7f33f809daf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33f000c7a0 tx=0x7f33f000cc60 comp rx=0 tx=0).stop 2024-03-14T12:14:01.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f33fd81e640 1 -- 172.21.15.82:0/258178992 reap_dead start 2024-03-14T12:14:01.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c4806bc20 msgr2=0x7f6c4806e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.995+0000 7f6c6c1fa640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6c4806bc20 0x7f6c4806e0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ce170 tx=0x7f6c60009040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640c9520 msgr2=0x7f6c6415e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6c6c1fa640 1 --2- 172.21.15.82:0/124038325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c640c9520 0x7f6c6415e710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c036040 tx=0x7f6c5c004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:01.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 shutdown_connections 2024-03-14T12:14:01.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6c6a797640 1 -- 172.21.15.82:0/124038325 reap_dead start 2024-03-14T12:14:01.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 >> 172.21.15.82:0/258178992 conn(0x7f33f80b1b10 msgr2=0x7f33f80b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7fc504ff9640 1 -- 172.21.15.82:0/1587037449 shutdown_connections 2024-03-14T12:14:01.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 shutdown_connections 2024-03-14T12:14:01.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 >> 172.21.15.82:0/124038325 conn(0x7f6c640d3db0 msgr2=0x7f6c64161340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 shutdown_connections 2024-03-14T12:14:01.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 shutdown_connections 2024-03-14T12:14:01.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7fc504ff9640 1 -- 172.21.15.82:0/1587037449 wait complete. 2024-03-14T12:14:01.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f33ff281640 1 -- 172.21.15.82:0/258178992 wait complete. 2024-03-14T12:14:01.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f6ba4073ec0 msgr2=0x7f6ba40762e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f6ba4073ec0 0x7f6ba40762e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6ba406bca0 msgr2=0x7f6ba406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6c6c1fa640 1 -- 172.21.15.82:0/124038325 wait complete. 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6ba406bca0 0x7f6ba406e160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bac03a000 tx=0x7f6bac034000 comp rx=0 tx=0).stop 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00b0a80 msgr2=0x7f6bc00b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 --2- 172.21.15.82:0/3449709542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bc00b0a80 0x7f6bc00b48a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400d620 tx=0x7f6bb400dae0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc51e8640 1 -- 172.21.15.82:0/3449709542 reap_dead start 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 shutdown_connections 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 >> 172.21.15.82:0/3449709542 conn(0x7f6bc00b33b0 msgr2=0x7f6bc00af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 shutdown_connections 2024-03-14T12:14:01.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:00.996+0000 7f6bc59e9640 1 -- 172.21.15.82:0/3449709542 wait complete. 2024-03-14T12:14:01.072 INFO:teuthology.orchestra.run.smithi082.stdout:55834574855 2024-03-14T12:14:01.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-14T12:14:01.073 INFO:teuthology.orchestra.run.smithi082.stdout:55834574856 2024-03-14T12:14:01.074 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-14T12:14:01.074 INFO:teuthology.orchestra.run.smithi082.stdout:55834574855 2024-03-14T12:14:01.075 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-14T12:14:01.076 INFO:teuthology.orchestra.run.smithi082.stdout:55834574857 2024-03-14T12:14:01.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-14T12:14:01.077 INFO:teuthology.orchestra.run.smithi082.stdout:55834574854 2024-03-14T12:14:01.077 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-14T12:14:01.078 INFO:teuthology.orchestra.run.smithi082.stdout:55834574856 2024-03-14T12:14:01.078 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-14T12:14:01.079 INFO:teuthology.orchestra.run.smithi082.stdout:55834574855 2024-03-14T12:14:01.079 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-14T12:14:01.080 INFO:teuthology.orchestra.run.smithi082.stdout:55834574856 2024-03-14T12:14:01.080 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-14T12:14:01.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.282+0000 7f7ef2cb8640 1 Processor -- start 2024-03-14T12:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.282+0000 7fbd80f37640 1 Processor -- start 2024-03-14T12:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.282+0000 7fbd80f37640 1 -- start start 2024-03-14T12:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.282+0000 7f7ef2cb8640 1 -- start start 2024-03-14T12:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.283+0000 7f362bad1640 1 Processor -- start 2024-03-14T12:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.283+0000 7f362bad1640 1 -- start start 2024-03-14T12:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.283+0000 7f7ef2cb8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7eec071a70 0x7f7eec0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.283+0000 7f7ef2cb8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec06df90 0x7f7eec06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7ef2cb8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec0b3a60 con 0x7f7eec0704a0 2024-03-14T12:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fa0748e2640 1 Processor -- start 2024-03-14T12:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7eebfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7eec071a70 0x7f7eec0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7ef0a54640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7eec0704a0 0x7f7eec0cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:54102/0 (socket says 172.21.15.82:54102) 2024-03-14T12:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7ef0a54640 1 -- 172.21.15.82:0/1484971060 learned_addr learned my addr 172.21.15.82:0/1484971060 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fa0748e2640 1 -- start start 2024-03-14T12:14:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1484971060 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec06c4a0 con 0x7f7eec071a70 2024-03-14T12:14:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1484971060 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec06c780 con 0x7f7eec06df90 2024-03-14T12:14:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fbd80f37640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbd74048100 0x7fbd74006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fbd80f37640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd74054560 0x7fbd7404f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7eeb7fe640 1 -- 172.21.15.82:0/1484971060 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666923368 0 0) 0x7f7eec0b3a60 con 0x7f7eec0704a0 2024-03-14T12:14:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7eeb7fe640 1 -- 172.21.15.82:0/1484971060 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7edc003520 con 0x7f7eec0704a0 2024-03-14T12:14:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7ef1255640 1 --2- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec06df90 0x7f7eec06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f9120ad5640 1 Processor -- start 2024-03-14T12:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fbd80f37640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd74005ec0 con 0x7fbd74007460 2024-03-14T12:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fbd80f37640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd740e7b90 con 0x7fbd74054560 2024-03-14T12:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7eebfff640 1 -- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec06df90 msgr2=0x7f7eec06e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7eebfff640 1 --2- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec06df90 0x7f7eec06e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7eebfff640 1 -- 172.21.15.82:0/1484971060 >> v1:172.21.15.164:6789/0 conn(0x7f7eec0704a0 legacy=0x7f7eec0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7eebfff640 1 -- 172.21.15.82:0/1484971060 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0720f0 con 0x7f7eec071a70 2024-03-14T12:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f7ef1255640 1 --2- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec06df90 0x7f7eec06e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f362bad1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f36240ce960 0x7f362406e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f362bad1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f3624165c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f362bad1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362406ecd0 con 0x7f36240cef80 2024-03-14T12:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f362bad1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36241628f0 con 0x7f36240ce960 2024-03-14T12:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f362bad1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3624162bd0 con 0x7f36240722e0 2024-03-14T12:14:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fbd80f37640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd74006a90 con 0x7fbd74048100 2024-03-14T12:14:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fbd79d9b640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbd74007460 0x7fbd74053e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:54116/0 (socket says 172.21.15.82:54116) 2024-03-14T12:14:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7fbd79d9b640 1 -- 172.21.15.82:0/502601013 learned_addr learned my addr 172.21.15.82:0/502601013 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f1dcd7f1640 1 Processor -- start 2024-03-14T12:14:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f1dcd7f1640 1 -- start start 2024-03-14T12:14:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7f362906c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f36240cef80 0x7f3624071bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:54124/0 (socket says 172.21.15.82:54124) 2024-03-14T12:14:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7f362906c640 1 -- 172.21.15.82:0/532169663 learned_addr learned my addr 172.21.15.82:0/532169663 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7f362a06e640 1 --2- 172.21.15.82:0/532169663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f3624165c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa0748e2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0680dfee0 0x7fa0680085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa0748e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680e02c0 0x7fa0680e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7f362a06e640 1 -- 172.21.15.82:0/532169663 >> v2:172.21.15.82:3300/0 conn(0x7f36240ce960 msgr2=0x7f362406e610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T12:14:01.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa06f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0680dfee0 0x7fa0680085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa06f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0680dfee0 0x7fa0680085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48778/0 (socket says 172.21.15.82:48778) 2024-03-14T12:14:01.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7f362a06e640 1 --2- 172.21.15.82:0/532169663 >> v2:172.21.15.82:3300/0 conn(0x7f36240ce960 0x7f362406e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa06f7fe640 1 -- 172.21.15.82:0/718509196 learned_addr learned my addr 172.21.15.82:0/718509196 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa06effd640 1 --2- 172.21.15.82:0/718509196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680e02c0 0x7fa0680e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7f362a06e640 1 -- 172.21.15.82:0/532169663 >> v1:172.21.15.164:6789/0 conn(0x7f36240cef80 legacy=0x7f3624071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa068056580 con 0x7fa0680e40f0 2024-03-14T12:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0680e4540 con 0x7fa0680dfee0 2024-03-14T12:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7f362a06e640 1 -- 172.21.15.82:0/532169663 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624162eb0 con 0x7f36240722e0 2024-03-14T12:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fbd7ad9d640 1 --2- 172.21.15.82:0/502601013 >> v2:172.21.15.82:3300/0 conn(0x7fbd74054560 0x7fbd7404f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.284+0000 7f9120ad5640 1 -- start start 2024-03-14T12:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.285+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0680094a0 con 0x7fa0680e02c0 2024-03-14T12:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/502601013 >> v2:172.21.15.164:3300/0 conn(0x7fbd74048100 0x7fbd74006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dcd7f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 0x7f1dc80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588830565 0 0) 0x7fbd74005ec0 con 0x7fbd74007460 2024-03-14T12:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd64003520 con 0x7fbd74007460 2024-03-14T12:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f9120ad5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f911c15c630 0x7f911c070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f8ff5214640 1 Processor -- start 2024-03-14T12:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dcd7f1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc806c450 con 0x7f1dc80b39c0 2024-03-14T12:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dcd7f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc806c730 con 0x7f1dc8165150 2024-03-14T12:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dcd7f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b4350 con 0x7f1dc806e920 2024-03-14T12:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1dc80b39c0 0x7f1dc808b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34600/0 (socket says 172.21.15.82:34600) 2024-03-14T12:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc77fe640 1 -- 172.21.15.82:0/3142370864 learned_addr learned my addr 172.21.15.82:0/3142370864 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dbffff640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 0x7f1dc80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f9120ad5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f911c06daf0 0x7f911c0b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f911affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f911c163280 0x7f911c06d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34612/0 (socket says 172.21.15.82:34612) 2024-03-14T12:14:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132771492 0 0) 0x7f1dc806c730 con 0x7f1dc8165150 2024-03-14T12:14:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1db4003520 con 0x7f1dc8165150 2024-03-14T12:14:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f911affd640 1 -- 172.21.15.82:0/2811361291 learned_addr learned my addr 172.21.15.82:0/2811361291 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fefa9efc640 1 Processor -- start 2024-03-14T12:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f911bfff640 1 --2- 172.21.15.82:0/2811361291 >> v2:172.21.15.82:3300/0 conn(0x7f911c06daf0 0x7f911c0b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090828616 0 0) 0x7f1db4003520 con 0x7f1dc8165150 2024-03-14T12:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 msgr2=0x7f1dc80b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc67fc640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 0x7f1dc80b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f911b7fe640 1 --2- 172.21.15.82:0/2811361291 >> v2:172.21.15.164:3300/0 conn(0x7f911c15c630 0x7f911c070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4092093984 0 0) 0x7fbd64003520 con 0x7fbd74007460 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 >> v2:172.21.15.164:3300/0 conn(0x7fbd74048100 msgr2=0x7fbd74006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 --2- 172.21.15.82:0/502601013 >> v2:172.21.15.164:3300/0 conn(0x7fbd74048100 0x7fbd74006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 >> v2:172.21.15.82:3300/0 conn(0x7fbd74054560 msgr2=0x7fbd7404f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 --2- 172.21.15.82:0/502601013 >> v2:172.21.15.82:3300/0 conn(0x7fbd74054560 0x7fbd7404f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd74053280 con 0x7fbd74007460 2024-03-14T12:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fbd7c04f570 con 0x7fbd74007460 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 >> v1:172.21.15.164:6789/0 conn(0x7fbd74007460 legacy=0x7fbd74053e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd64003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd64004370 0x7fbd64004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd64004fa0 0x7fbd6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740e7b90 con 0x7fbd640039b0 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74006a90 con 0x7fbd64004370 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6400bf50 con 0x7fbd64004fa0 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd64003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7fbd7ad9d640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd64004fa0 0x7fbd6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f7eebfff640 1 --2- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec071a70 0x7f7eec0b3100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80097a0 tx=0x7f7ed802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3299575e1809125f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c15ca10 con 0x7f911c163280 2024-03-14T12:14:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c0717e0 con 0x7f911c06daf0 2024-03-14T12:14:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c071ac0 con 0x7f911c15c630 2024-03-14T12:14:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f8ff5214640 1 -- start start 2024-03-14T12:14:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 >> v1:172.21.15.82:6789/0 conn(0x7f1dc80b39c0 legacy=0x7f1dc808b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc806cef0 con 0x7f1dc8165150 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f1db80015a0 con 0x7f1dc8165150 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 >> v1:172.21.15.164:6789/0 conn(0x7f1dc8165150 legacy=0x7f1dc816ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db40039b0 0x7f1db4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 0x7f1db4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1db4004d40 0x7f1db40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc6ffd640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db40039b0 0x7f1db4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dbffff640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 0x7f1db4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b4350 con 0x7f1db40039b0 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db400baf0 con 0x7f1db4004d40 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db400bcf0 con 0x7f1dc806e920 2024-03-14T12:14:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fa06effd640 1 -- 172.21.15.82:0/718509196 >> v2:172.21.15.82:3300/0 conn(0x7fa0680dfee0 msgr2=0x7fa0680085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fa06effd640 1 --2- 172.21.15.82:0/718509196 >> v2:172.21.15.82:3300/0 conn(0x7fa0680dfee0 0x7fa0680085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fa06effd640 1 -- 172.21.15.82:0/718509196 >> v1:172.21.15.164:6789/0 conn(0x7fa0680e40f0 legacy=0x7fa06803a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fa06effd640 1 -- 172.21.15.82:0/718509196 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068056860 con 0x7fa0680e02c0 2024-03-14T12:14:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.286+0000 7fefa9efc640 1 -- start start 2024-03-14T12:14:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f911bfff640 1 -- 172.21.15.82:0/2811361291 >> v2:172.21.15.164:3300/0 conn(0x7f911c15c630 msgr2=0x7f911c070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f911bfff640 1 --2- 172.21.15.82:0/2811361291 >> v2:172.21.15.164:3300/0 conn(0x7f911c15c630 0x7f911c070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f911a7fc640 1 -- 172.21.15.82:0/2811361291 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796746567 0 0) 0x7f911c15ca10 con 0x7f911c163280 2024-03-14T12:14:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f911bfff640 1 -- 172.21.15.82:0/2811361291 >> v1:172.21.15.82:6789/0 conn(0x7f911c163280 legacy=0x7f911c06d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f8ff5214640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff006d620 0x7f8ff006da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f7eeb7fe640 1 -- 172.21.15.82:0/1484971060 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80315a0 con 0x7f7eec071a70 2024-03-14T12:14:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f7eeb7fe640 1 -- 172.21.15.82:0/1484971060 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed8037070 con 0x7f7eec071a70 2024-03-14T12:14:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f7eeb7fe640 1 -- 172.21.15.82:0/1484971060 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8031960 con 0x7f7eec071a70 2024-03-14T12:14:01.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f8feed9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8ff0162240 0x7f8ff00ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:54140/0 (socket says 172.21.15.82:54140) 2024-03-14T12:14:01.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f1dc77fe640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1db4004d40 0x7f1db40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f8fee59c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff006d620 0x7f8ff006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f8ff5214640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00cf9b0 con 0x7f8ff006ad10 2024-03-14T12:14:01.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8ff5214640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff0162620 con 0x7f8ff0162240 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8ff5214640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00cf700 con 0x7f8ff006d620 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8feed9d640 1 -- 172.21.15.82:0/2291189763 learned_addr learned my addr 172.21.15.82:0/2291189763 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f1dc6ffd640 1 -- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 msgr2=0x7f1db4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f1dc6ffd640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc806e920 0x7f1db4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f1dc6ffd640 1 -- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1db4004d40 msgr2=0x7f1db40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f1dc6ffd640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1db4004d40 0x7f1db40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f1dc6ffd640 1 -- 172.21.15.82:0/3142370864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4003520 con 0x7f1db40039b0 2024-03-14T12:14:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fa06effd640 1 --2- 172.21.15.82:0/718509196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680e02c0 0x7fa0680e3bb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0097a0 tx=0x7fa05c02d980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f04b08cddb59e6a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7f911bfff640 1 -- 172.21.15.82:0/2811361291 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0b2900 con 0x7f911c06daf0 2024-03-14T12:14:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7fbd7a59c640 1 -- 172.21.15.82:0/502601013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd64004fa0 msgr2=0x7fbd6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd64004fa0 0x7fbd6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7fbd7a59c640 1 -- 172.21.15.82:0/502601013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd64004370 msgr2=0x7fbd64004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd64004370 0x7fbd64004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.287+0000 7fbd7a59c640 1 -- 172.21.15.82:0/502601013 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd74053280 con 0x7fbd640039b0 2024-03-14T12:14:01.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fbd7ad9d640 1 -- 172.21.15.82:0/502601013 reap_dead start 2024-03-14T12:14:01.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f362a06e640 1 --2- 172.21.15.82:0/532169663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f3624165c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f36200097a0 tx=0x7f362002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed9ec2d8eda4102 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564870056 0 0) 0x7f8ff0162620 con 0x7f8ff0162240 2024-03-14T12:14:01.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd8003520 con 0x7f8ff0162240 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fefa9efc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fefa40d1440 0x7fefa40d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fefa9efc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fefa40b41a0 0x7fefa40ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fefa9efc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa40d4ab0 con 0x7fefa40b36b0 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fefa9efc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa40b5000 con 0x7fefa40d1440 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fefa9efc640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa40b52e0 con 0x7fefa40b41a0 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fef9bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fefa40d1440 0x7fefa40d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fef9bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fefa40d1440 0x7fefa40d4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48810/0 (socket says 172.21.15.82:48810) 2024-03-14T12:14:01.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fef9bfff640 1 -- 172.21.15.82:0/682719876 learned_addr learned my addr 172.21.15.82:0/682719876 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fa06e7fc640 1 -- 172.21.15.82:0/718509196 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c037020 con 0x7fa0680e02c0 2024-03-14T12:14:01.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fa06e7fc640 1 -- 172.21.15.82:0/718509196 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c02f040 con 0x7fa0680e02c0 2024-03-14T12:14:01.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fa06e7fc640 1 -- 172.21.15.82:0/718509196 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c033a80 con 0x7fa0680e02c0 2024-03-14T12:14:01.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec071a70 msgr2=0x7f7eec0b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec071a70 0x7f7eec0b3100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80097a0 tx=0x7f7ed802b840 comp rx=0 tx=0).stop 2024-03-14T12:14:01.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1171068355 0 0) 0x7f8fd8003520 con 0x7f8ff0162240 2024-03-14T12:14:01.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff006d620 msgr2=0x7f8ff006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd64003df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000ba10 tx=0x7fbd64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66813173bddf7b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff006d620 0x7f8ff006da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 >> v1:172.21.15.82:6789/0 conn(0x7f8ff006ad10 legacy=0x7f8ff00b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff006c860 con 0x7f8ff0162240 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f8fe0001670 con 0x7f8ff0162240 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 >> v1:172.21.15.164:6789/0 conn(0x7f8ff0162240 legacy=0x7f8ff00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fd8003a20 0x7f8fd8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd8004800 0x7f8fd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fd80051c0 0x7f8fd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00cf700 con 0x7f8fd8004800 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd800c090 con 0x7f8fd80051c0 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd800c290 con 0x7f8fd8003a20 2024-03-14T12:14:01.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f8fee59c640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff006d620 0x7f8ff006da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fefa3fff640 1 --2- 172.21.15.82:0/682719876 >> v2:172.21.15.164:3300/0 conn(0x7fefa40b41a0 0x7fefa40ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fef9bfff640 1 -- 172.21.15.82:0/682719876 >> v2:172.21.15.164:3300/0 conn(0x7fefa40b41a0 msgr2=0x7fefa40ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fef9bfff640 1 --2- 172.21.15.82:0/682719876 >> v2:172.21.15.164:3300/0 conn(0x7fefa40b41a0 0x7fefa40ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fef9bfff640 1 -- 172.21.15.82:0/682719876 >> v1:172.21.15.164:6789/0 conn(0x7fefa40b36b0 legacy=0x7fefa40b3a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7fef9bfff640 1 -- 172.21.15.82:0/682719876 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40b55c0 con 0x7fefa40d1440 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.288+0000 7f362886b640 1 -- 172.21.15.82:0/532169663 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36200315a0 con 0x7f36240722e0 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f362886b640 1 -- 172.21.15.82:0/532169663 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620037070 con 0x7f36240722e0 2024-03-14T12:14:01.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f362886b640 1 -- 172.21.15.82:0/532169663 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620031960 con 0x7f36240722e0 2024-03-14T12:14:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f911bfff640 1 --2- 172.21.15.82:0/2811361291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c0b33e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b2100 tx=0x7f911002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719f11ed3e2973f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1484971060 shutdown_connections 2024-03-14T12:14:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec06df90 0x7f7eec06e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1484971060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec071a70 0x7f7eec0b3100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1484971060 >> 172.21.15.82:0/1484971060 conn(0x7f7eec0b13f0 msgr2=0x7f7eec0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f8feed9d640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fd8003a20 0x7f8fd8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70036020 con 0x7fbd640039b0 2024-03-14T12:14:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1484971060 shutdown_connections 2024-03-14T12:14:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd70032620 con 0x7fbd640039b0 2024-03-14T12:14:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fbd80f37640 1 -- 172.21.15.82:0/502601013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 msgr2=0x7fbd64003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fbd7959a640 1 -- 172.21.15.82:0/502601013 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70036020 con 0x7fbd640039b0 2024-03-14T12:14:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fbd80f37640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd64003df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000ba10 tx=0x7fbd64005830 comp rx=0 tx=0).stop 2024-03-14T12:14:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1484971060 wait complete. 2024-03-14T12:14:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f362bad1640 1 -- 172.21.15.82:0/532169663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 msgr2=0x7f3624165c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f362bad1640 1 --2- 172.21.15.82:0/532169663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f3624165c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f36200097a0 tx=0x7f362002b840 comp rx=0 tx=0).stop 2024-03-14T12:14:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f911a7fc640 1 -- 172.21.15.82:0/2811361291 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110035020 con 0x7f911c06daf0 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f911a7fc640 1 -- 172.21.15.82:0/2811361291 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9110031830 con 0x7f911c06daf0 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f8fef59e640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fd80051c0 0x7f8fd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f8feed9d640 1 -- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fd80051c0 msgr2=0x7f8fd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f8feed9d640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fd80051c0 0x7f8fd800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f8feed9d640 1 -- 172.21.15.82:0/2291189763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd8004800 msgr2=0x7f8fd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f8feed9d640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd8004800 0x7f8fd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f8feed9d640 1 -- 172.21.15.82:0/2291189763 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff006c860 con 0x7f8fd8003a20 2024-03-14T12:14:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7f1dc6ffd640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db40039b0 0x7f1db4003d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1db8007e90 tx=0x7f1db4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ab1c8df51653a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680e02c0 msgr2=0x7fa0680e3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 --2- 172.21.15.82:0/718509196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680e02c0 0x7fa0680e3bb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0097a0 tx=0x7fa05c02d980 comp rx=0 tx=0).stop 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 shutdown_connections 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 --2- 172.21.15.82:0/718509196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680e02c0 0x7fa0680e3bb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 --2- 172.21.15.82:0/718509196 >> v2:172.21.15.82:3300/0 conn(0x7fa0680dfee0 0x7fa0680085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 >> 172.21.15.82:0/718509196 conn(0x7fa06801ef30 msgr2=0x7fa068055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 shutdown_connections 2024-03-14T12:14:01.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.289+0000 7fa0748e2640 1 -- 172.21.15.82:0/718509196 wait complete. 2024-03-14T12:14:01.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fef9bfff640 1 --2- 172.21.15.82:0/682719876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40d1440 0x7fefa40d4460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fefa40cadd0 tx=0x7fef8c02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f69cf3a99af843 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 msgr2=0x7f911c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f911a7fc640 1 -- 172.21.15.82:0/2811361291 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110035020 con 0x7f911c06daf0 2024-03-14T12:14:01.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f9120ad5640 1 --2- 172.21.15.82:0/2811361291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c0b33e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b2100 tx=0x7f911002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fbd80f37640 1 -- 172.21.15.82:0/502601013 shutdown_connections 2024-03-14T12:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fbd80f37640 1 --2- 172.21.15.82:0/502601013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd64003df0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fbd80f37640 1 -- 172.21.15.82:0/502601013 >> 172.21.15.82:0/502601013 conn(0x7fbd74009fe0 msgr2=0x7fbd7400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f362bad1640 1 -- 172.21.15.82:0/532169663 shutdown_connections 2024-03-14T12:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f362bad1640 1 --2- 172.21.15.82:0/532169663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f3624165c40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f362bad1640 1 --2- 172.21.15.82:0/532169663 >> v2:172.21.15.82:3300/0 conn(0x7f36240ce960 0x7f362406e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f362bad1640 1 -- 172.21.15.82:0/532169663 >> 172.21.15.82:0/532169663 conn(0x7f36240b13f0 msgr2=0x7f36240b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7fefa2ffd640 1 -- 172.21.15.82:0/682719876 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c039070 con 0x7fefa40d1440 2024-03-14T12:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 shutdown_connections 2024-03-14T12:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f9120ad5640 1 --2- 172.21.15.82:0/2811361291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c0b33e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f9120ad5640 1 --2- 172.21.15.82:0/2811361291 >> v2:172.21.15.164:3300/0 conn(0x7f911c15c630 0x7f911c070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 >> 172.21.15.82:0/2811361291 conn(0x7f911c0b1560 msgr2=0x7f911c0b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8039070 con 0x7f1db40039b0 2024-03-14T12:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db802c890 con 0x7f1db40039b0 2024-03-14T12:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7fbd80f37640 1 -- 172.21.15.82:0/502601013 shutdown_connections 2024-03-14T12:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f1dc67fc640 1 -- 172.21.15.82:0/3142370864 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007ab0 con 0x7f1db40039b0 2024-03-14T12:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.290+0000 7f362bad1640 1 -- 172.21.15.82:0/532169663 shutdown_connections 2024-03-14T12:14:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f362bad1640 1 -- 172.21.15.82:0/532169663 wait complete. 2024-03-14T12:14:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7fefa2ffd640 1 -- 172.21.15.82:0/682719876 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c02c710 con 0x7fefa40d1440 2024-03-14T12:14:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7fefa2ffd640 1 -- 172.21.15.82:0/682719876 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c02cae0 con 0x7fefa40d1440 2024-03-14T12:14:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7fbd80f37640 1 -- 172.21.15.82:0/502601013 wait complete. 2024-03-14T12:14:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f7ef2cb8640 1 Processor -- start 2024-03-14T12:14:01.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 shutdown_connections 2024-03-14T12:14:01.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7fa0748e2640 1 Processor -- start 2024-03-14T12:14:01.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f9120ad5640 1 -- 172.21.15.82:0/2811361291 wait complete. 2024-03-14T12:14:01.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f8feed9d640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fd8003a20 0x7f8fd8003e40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c340 tx=0x7f8fd8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26057763c9ad178f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3142370864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db40039b0 msgr2=0x7f1db4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f1dcd7f1640 1 --2- 172.21.15.82:0/3142370864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db40039b0 0x7f1db4003d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1db8007e90 tx=0x7f1db4005830 comp rx=0 tx=0).stop 2024-03-14T12:14:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7fefa9efc640 1 -- 172.21.15.82:0/682719876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40d1440 msgr2=0x7fefa40d4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0007dd0 con 0x7f8fd8003a20 2024-03-14T12:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe0030070 con 0x7f8fd8003a20 2024-03-14T12:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f8fedd9b640 1 -- 172.21.15.82:0/2291189763 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe002ce60 con 0x7f8fd8003a20 2024-03-14T12:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7fefa9efc640 1 --2- 172.21.15.82:0/682719876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40d1440 0x7fefa40d4460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fefa40cadd0 tx=0x7fef8c02bb70 comp rx=0 tx=0).stop 2024-03-14T12:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7fbd80f37640 1 Processor -- start 2024-03-14T12:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7f7ef2cb8640 1 -- start start 2024-03-14T12:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f7ef2cb8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec06df90 0x7f7eec15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f362bad1640 1 Processor -- start 2024-03-14T12:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7fbd80f37640 1 -- start start 2024-03-14T12:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f362bad1640 1 -- start start 2024-03-14T12:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.291+0000 7fa0748e2640 1 -- start start 2024-03-14T12:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f7ef2cb8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec0704a0 0x7f7eec15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7fa0748e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680dfee0 0x7fa0680e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.292+0000 7f9120ad5640 1 Processor -- start 2024-03-14T12:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f1dc77fe640 1 -- 172.21.15.82:0/3142370864 reap_dead start 2024-03-14T12:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3142370864 shutdown_connections 2024-03-14T12:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3142370864 >> 172.21.15.82:0/3142370864 conn(0x7f1dc80cf930 msgr2=0x7f1dc80cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fefa9efc640 1 -- 172.21.15.82:0/682719876 shutdown_connections 2024-03-14T12:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fefa9efc640 1 --2- 172.21.15.82:0/682719876 >> v2:172.21.15.164:3300/0 conn(0x7fefa40b41a0 0x7fefa40ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fefa9efc640 1 --2- 172.21.15.82:0/682719876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40d1440 0x7fefa40d4460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fefa9efc640 1 -- 172.21.15.82:0/682719876 >> 172.21.15.82:0/682719876 conn(0x7fefa409d030 msgr2=0x7fefa40d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa0748e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0680e40f0 0x7fa06804efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa0748e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0680e06b0 0x7fa0680e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3142370864 shutdown_connections 2024-03-14T12:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3142370864 wait complete. 2024-03-14T12:14:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f8ff5214640 1 -- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fd8003a20 msgr2=0x7f8fd8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f8ff5214640 1 --2- 172.21.15.82:0/2291189763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fd8003a20 0x7f8fd8003e40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c340 tx=0x7f8fd8005830 comp rx=0 tx=0).stop 2024-03-14T12:14:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f7ef0a54640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec06df90 0x7f7eec15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f8fef59e640 1 -- 172.21.15.82:0/2291189763 reap_dead start 2024-03-14T12:14:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f8ff5214640 1 -- 172.21.15.82:0/2291189763 shutdown_connections 2024-03-14T12:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f8ff5214640 1 -- 172.21.15.82:0/2291189763 >> 172.21.15.82:0/2291189763 conn(0x7f8ff00b1930 msgr2=0x7f8ff00b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa06f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680dfee0 0x7fa0680e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f7ef0a54640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec06df90 0x7f7eec15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:58522/0 (socket says 172.21.15.82:58522) 2024-03-14T12:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f7eebfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec0704a0 0x7f7eec15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fefa9efc640 1 -- 172.21.15.82:0/682719876 shutdown_connections 2024-03-14T12:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fefa9efc640 1 -- 172.21.15.82:0/682719876 wait complete. 2024-03-14T12:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f7ef0a54640 1 -- 172.21.15.82:0/1520349894 learned_addr learned my addr 172.21.15.82:0/1520349894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eec071a70 0x7f7eec15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa06f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680dfee0 0x7fa0680e21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:53066/0 (socket says 172.21.15.82:53066) 2024-03-14T12:14:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0680e06b0 0x7fa0680e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0680e06b0 0x7fa0680e0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48824/0 (socket says 172.21.15.82:48824) 2024-03-14T12:14:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa06effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0680e40f0 0x7fa06804efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa06effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0680e40f0 0x7fa06804efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:58530/0 (socket says 172.21.15.82:58530) 2024-03-14T12:14:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa06f7fe640 1 -- 172.21.15.82:0/1879889138 learned_addr learned my addr 172.21.15.82:0/1879889138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fbd80f37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd740d3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fbd80f37640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd7404e450 0x7fbd7404e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fbd80f37640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd740e5430 0x7fbd740e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fbd80f37640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74006720 con 0x7fbd640039b0 2024-03-14T12:14:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fbd80f37640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74006440 con 0x7fbd7404e450 2024-03-14T12:14:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fbd80f37640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74006f20 con 0x7fbd740e5430 2024-03-14T12:14:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f7ef1255640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eec071a70 0x7f7eec15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f362bad1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f362415f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0b37c0 con 0x7f7eec071a70 2024-03-14T12:14:01.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0b3640 con 0x7f7eec06df90 2024-03-14T12:14:01.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec070880 con 0x7f7eec0704a0 2024-03-14T12:14:01.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f362bad1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240ce960 0x7f36241f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7f9120ad5640 1 -- start start 2024-03-14T12:14:01.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f362986d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f362415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f8ff5214640 1 -- 172.21.15.82:0/2291189763 shutdown_connections 2024-03-14T12:14:01.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f362986d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f362415f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:53084/0 (socket says 172.21.15.82:53084) 2024-03-14T12:14:01.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f8ff5214640 1 -- 172.21.15.82:0/2291189763 wait complete. 2024-03-14T12:14:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fbd7ad9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd740e5430 0x7fbd740e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f9120ad5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f9120ad5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f911c15c630 0x7f911c1fb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c1517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48840/0 (socket says 172.21.15.82:48840) 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f911b7fe640 1 -- 172.21.15.82:0/1629123480 learned_addr learned my addr 172.21.15.82:0/1629123480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068056b40 con 0x7fa0680e06b0 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068009130 con 0x7fa0680e40f0 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.293+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa068008b00 con 0x7fa0680dfee0 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f9120ad5640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f911c163280 0x7f911c160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0713f0 con 0x7f911c06daf0 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0b2ec0 con 0x7f911c15c630 2024-03-14T12:14:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c071570 con 0x7f911c163280 2024-03-14T12:14:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fa06f7fe640 1 -- 172.21.15.82:0/1879889138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0680e40f0 msgr2=0x7fa06804efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fa06f7fe640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0680e40f0 0x7fa06804efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fa06f7fe640 1 -- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0680e06b0 msgr2=0x7fa0680e0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fa06f7fe640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0680e06b0 0x7fa0680e0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fa06f7fe640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0680e1130 con 0x7fa0680dfee0 2024-03-14T12:14:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fa06ffff640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0680e06b0 0x7fa0680e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fa06f7fe640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680dfee0 0x7fa0680e21d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e8b0 tx=0x7fa06400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f911affd640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f911c15c630 0x7f911c1fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fbd7ad9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd740e5430 0x7fbd740e58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:53070/0 (socket says 172.21.15.82:53070) 2024-03-14T12:14:01.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7fbd7ad9d640 1 -- 172.21.15.82:0/3929429408 learned_addr learned my addr 172.21.15.82:0/3929429408 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f7ef1255640 1 -- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec0704a0 msgr2=0x7f7eec15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f7ef1255640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec0704a0 0x7f7eec15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ef1255640 1 -- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec06df90 msgr2=0x7f7eec15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f911bfff640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f911c163280 0x7f911c160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fbd7ad9d640 1 -- 172.21.15.82:0/3929429408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd7404e450 msgr2=0x7fbd7404e8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T12:14:01.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fbd7ad9d640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd7404e450 0x7fbd7404e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fbd7ad9d640 1 -- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 msgr2=0x7fbd740d3790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T12:14:01.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f1dcd7f1640 1 Processor -- start 2024-03-14T12:14:01.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f1dcd7f1640 1 -- start start 2024-03-14T12:14:01.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fefa9efc640 1 Processor -- start 2024-03-14T12:14:01.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.294+0000 7f362986d640 1 -- 172.21.15.82:0/4001626864 learned_addr learned my addr 172.21.15.82:0/4001626864 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f362bad1640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36240cef80 0x7f36241f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362406d1c0 con 0x7f36240cef80 2024-03-14T12:14:01.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362406d040 con 0x7f36240ce960 2024-03-14T12:14:01.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36241633d0 con 0x7f36240722e0 2024-03-14T12:14:01.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f362906c640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240ce960 0x7f36241f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fefa9efc640 1 -- start start 2024-03-14T12:14:01.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ef1255640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec06df90 0x7f7eec15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f911bfff640 1 -- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f911c15c630 msgr2=0x7f911c1fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f911bfff640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f911c15c630 0x7f911c1fb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f911bfff640 1 -- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 msgr2=0x7f911c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f911bfff640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f911bfff640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0ca700 con 0x7f911c163280 2024-03-14T12:14:01.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f911b7fe640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:01.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fa05bfff640 1 -- 172.21.15.82:0/1879889138 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064017070 con 0x7fa0680dfee0 2024-03-14T12:14:01.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fa05bfff640 1 -- 172.21.15.82:0/1879889138 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400c940 con 0x7fa0680dfee0 2024-03-14T12:14:01.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd740d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ef1255640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec15a530 con 0x7f7eec071a70 2024-03-14T12:14:01.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ef0a54640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec06df90 0x7f7eec15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7eebfff640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec0704a0 0x7f7eec15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ef1255640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eec071a70 0x7f7eec15c4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400ccc0 tx=0x7f7ee400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ee97fa640 1 -- 172.21.15.82:0/1520349894 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400f680 con 0x7f7eec071a70 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fbd7ad9d640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd740d3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fbd7ad9d640 1 -- 172.21.15.82:0/3929429408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd740e40f0 con 0x7fbd740e5430 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd740d3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ee97fa640 1 -- 172.21.15.82:0/1520349894 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee400f840 con 0x7f7eec071a70 2024-03-14T12:14:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ee97fa640 1 -- 172.21.15.82:0/1520349894 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400fc30 con 0x7f7eec071a70 2024-03-14T12:14:01.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.295+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec15a810 con 0x7f7eec071a70 2024-03-14T12:14:01.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f362906c640 1 -- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 msgr2=0x7f362415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f362906c640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f362415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f362906c640 1 -- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36240cef80 msgr2=0x7f36241f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fa05bfff640 1 -- 172.21.15.82:0/1879889138 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400cd10 con 0x7fa0680dfee0 2024-03-14T12:14:01.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fbd7ad9d640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd740e5430 0x7fbd740e58e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd7404f060 tx=0x7fbd6c011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0680e1410 con 0x7fa0680dfee0 2024-03-14T12:14:01.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dcd7f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc8156c40 0x7f1dc8167760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dcd7f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc816d530 0x7f1dc808a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dcd7f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc8167ca0 0x7f1dc808aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f362906c640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36240cef80 0x7f36241f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f362906c640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624201770 con 0x7f36240ce960 2024-03-14T12:14:01.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0680e19d0 con 0x7fa0680dfee0 2024-03-14T12:14:01.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dcd7f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b3e30 con 0x7f1dc8167ca0 2024-03-14T12:14:01.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc816d530 0x7f1dc808a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc816d530 0x7f1dc808a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:53096/0 (socket says 172.21.15.82:53096) 2024-03-14T12:14:01.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dbffff640 1 -- 172.21.15.82:0/3945671340 learned_addr learned my addr 172.21.15.82:0/3945671340 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc806ca10 con 0x7f1dc8156c40 2024-03-14T12:14:01.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fefa9efc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40b36b0 0x7fefa41eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f362906c640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240ce960 0x7f36241f6150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f361c00bd00 tx=0x7f361c00bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f362986d640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f362415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:01.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f8ff5214640 1 Processor -- start 2024-03-14T12:14:01.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f8ff5214640 1 -- start start 2024-03-14T12:14:01.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec15add0 con 0x7f7eec071a70 2024-03-14T12:14:01.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40b36b0 0x7fefa41eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40b36b0 0x7fefa41eadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48876/0 (socket says 172.21.15.82:48876) 2024-03-14T12:14:01.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7fefa37fe640 1 -- 172.21.15.82:0/778940614 learned_addr learned my addr 172.21.15.82:0/778940614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa40b41a0 0x7fefa40b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f911bfff640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f911c163280 0x7f911c160360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f911c074970 tx=0x7f91100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f9103fff640 1 -- 172.21.15.82:0/1629123480 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110048070 con 0x7f911c163280 2024-03-14T12:14:01.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f9103fff640 1 -- 172.21.15.82:0/1629123480 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9110004450 con 0x7f911c163280 2024-03-14T12:14:01.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f9103fff640 1 -- 172.21.15.82:0/1629123480 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110035070 con 0x7f911c163280 2024-03-14T12:14:01.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c0c5c80 con 0x7f911c163280 2024-03-14T12:14:01.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c1fc020 con 0x7f911c163280 2024-03-14T12:14:01.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7f361a7fc640 1 -- 172.21.15.82:0/4001626864 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c011710 con 0x7f36240ce960 2024-03-14T12:14:01.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3624201a50 con 0x7f36240ce960 2024-03-14T12:14:01.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7f1dc77fe640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc8167ca0 0x7f1dc808aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.296+0000 7fbd6affd640 1 -- 172.21.15.82:0/3929429408 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c003610 con 0x7fbd740e5430 2024-03-14T12:14:01.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7fbd6affd640 1 -- 172.21.15.82:0/3929429408 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c01b070 con 0x7fbd740e5430 2024-03-14T12:14:01.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7fbd6affd640 1 -- 172.21.15.82:0/3929429408 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c003a10 con 0x7fbd740e5430 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3624202010 con 0x7f36240ce960 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f361a7fc640 1 -- 172.21.15.82:0/4001626864 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c0118b0 con 0x7f36240ce960 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7f8ff5214640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff0200040 0x7f8ff0164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.297+0000 7f8ff5214640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff01611e0 0x7f8ff0071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f8ff5214640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff0164e70 0x7f8ff0071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f8ff5214640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff006d100 con 0x7f8ff01611e0 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f8ff5214640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff006ce20 con 0x7f8ff0164e70 2024-03-14T12:14:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f8ff5214640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00af120 con 0x7f8ff0200040 2024-03-14T12:14:01.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc81681d0 con 0x7f1dc816d530 2024-03-14T12:14:01.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7404edf0 con 0x7fbd740e5430 2024-03-14T12:14:01.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd740e4670 con 0x7fbd740e5430 2024-03-14T12:14:01.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefa40d1440 0x7fefa41eb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40cef20 con 0x7fefa40b36b0 2024-03-14T12:14:01.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40ceda0 con 0x7fefa40b41a0 2024-03-14T12:14:01.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40cf0a0 con 0x7fefa40d1440 2024-03-14T12:14:01.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f361a7fc640 1 -- 172.21.15.82:0/4001626864 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c011c80 con 0x7f36240ce960 2024-03-14T12:14:01.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fef9bfff640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa40b41a0 0x7fefa40b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fefa3fff640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefa40d1440 0x7fefa41eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f8fef59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff0164e70 0x7f8ff0071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f8fef59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff0164e70 0x7f8ff0071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:58550/0 (socket says 172.21.15.82:58550) 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f8fef59e640 1 -- 172.21.15.82:0/3372335689 learned_addr learned my addr 172.21.15.82:0/3372335689 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f8fee59c640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff01611e0 0x7f8ff0071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f8feed9d640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff0200040 0x7f8ff0164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fef9bfff640 1 -- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefa40d1440 msgr2=0x7fefa41eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fef9bfff640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefa40d1440 0x7fefa41eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fef9bfff640 1 -- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40b36b0 msgr2=0x7fefa41eadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fef9bfff640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40b36b0 0x7fefa41eadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fef9bfff640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40b8880 con 0x7fefa40b41a0 2024-03-14T12:14:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f7ee97fa640 1 -- 172.21.15.82:0/1520349894 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7ee40213f0 con 0x7f7eec071a70 2024-03-14T12:14:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f1dc6ffd640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc8156c40 0x7f1dc8167760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f1dbffff640 1 -- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc8156c40 msgr2=0x7f1dc8167760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f1dbffff640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc8156c40 0x7f1dc8167760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f1dbffff640 1 -- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc8167ca0 msgr2=0x7f1dc808aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f1dbffff640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc8167ca0 0x7f1dc808aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7fef9bfff640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa40b41a0 0x7fefa40b7ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef8c033000 tx=0x7fef8c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7f1dbffff640 1 -- 172.21.15.82:0/3945671340 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc81684b0 con 0x7f1dc816d530 2024-03-14T12:14:01.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f1dbffff640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc816d530 0x7f1dc808a4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1db0011950 tx=0x7f1db0011e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec075210 con 0x7f7eec071a70 2024-03-14T12:14:01.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f7ee97fa640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7ed006bca0 0x7f7ed006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f7ee97fa640 1 -- 172.21.15.82:0/1520349894 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ee408b640 con 0x7f7eec071a70 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c075210 con 0x7f911c163280 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f9103fff640 1 -- 172.21.15.82:0/1629123480 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f911004b070 con 0x7f911c163280 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f8fee59c640 1 -- 172.21.15.82:0/3372335689 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff0200040 msgr2=0x7f8ff0164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f8fee59c640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff0200040 0x7f8ff0164930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f8fee59c640 1 -- 172.21.15.82:0/3372335689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff0164e70 msgr2=0x7f8ff0071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f8fee59c640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff0164e70 0x7f8ff0071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f8fee59c640 1 -- 172.21.15.82:0/3372335689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0165380 con 0x7f8ff01611e0 2024-03-14T12:14:01.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7f7ef0a54640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7ed006bca0 0x7f7ed006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.298+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0680e4900 con 0x7fa0680dfee0 2024-03-14T12:14:01.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740053c0 con 0x7fbd740e5430 2024-03-14T12:14:01.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f361a7fc640 1 -- 172.21.15.82:0/4001626864 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f361c01c400 con 0x7f36240ce960 2024-03-14T12:14:01.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f9103fff640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f910806bca0 0x7f910806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f911b7fe640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f910806bca0 0x7f910806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f9103fff640 1 -- 172.21.15.82:0/1629123480 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91100ac1d0 con 0x7f911c163280 2024-03-14T12:14:01.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f8fee59c640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff01611e0 0x7f8ff0071920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ff006be10 tx=0x7f8fe400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f8fd77fe640 1 -- 172.21.15.82:0/3372335689 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4017070 con 0x7f8ff01611e0 2024-03-14T12:14:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff0165660 con 0x7f8ff01611e0 2024-03-14T12:14:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff00ad240 con 0x7f8ff01611e0 2024-03-14T12:14:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f8fd77fe640 1 -- 172.21.15.82:0/3372335689 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4013610 con 0x7f8ff01611e0 2024-03-14T12:14:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7fa05bfff640 1 -- 172.21.15.82:0/1879889138 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa06401a070 con 0x7fa0680dfee0 2024-03-14T12:14:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7fa05bfff640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa05406bcd0 0x7fa05406e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7fa05bfff640 1 -- 172.21.15.82:0/1879889138 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa06408ac20 con 0x7fa0680dfee0 2024-03-14T12:14:01.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7f8fd77fe640 1 -- 172.21.15.82:0/3372335689 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400f040 con 0x7f8ff01611e0 2024-03-14T12:14:01.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7fefa0ff9640 1 -- 172.21.15.82:0/778940614 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c039050 con 0x7fefa40b41a0 2024-03-14T12:14:01.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7fefa0ff9640 1 -- 172.21.15.82:0/778940614 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c045070 con 0x7fefa40b41a0 2024-03-14T12:14:01.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.299+0000 7fefa0ff9640 1 -- 172.21.15.82:0/778940614 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c003a20 con 0x7fefa40b41a0 2024-03-14T12:14:01.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa40b8b60 con 0x7fefa40b41a0 2024-03-14T12:14:01.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.300+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa40b9120 con 0x7fefa40b41a0 2024-03-14T12:14:01.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7f361a7fc640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f360806bc20 0x7f360806e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7fa06effd640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa05406bcd0 0x7fa05406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c005290 con 0x7f36240ce960 2024-03-14T12:14:01.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7f361a7fc640 1 -- 172.21.15.82:0/4001626864 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f361c08b5e0 con 0x7f36240ce960 2024-03-14T12:14:01.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7f362986d640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f360806bc20 0x7f360806e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7fbd6affd640 1 -- 172.21.15.82:0/3929429408 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbd6c003bb0 con 0x7fbd740e5430 2024-03-14T12:14:01.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7fbd6affd640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd5c06bca0 0x7fbd5c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7fbd6affd640 1 -- 172.21.15.82:0/3929429408 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd6c08a790 con 0x7fbd740e5430 2024-03-14T12:14:01.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.301+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa404f050 con 0x7fefa40b41a0 2024-03-14T12:14:01.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd5c06bca0 0x7fbd5c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd0005290 con 0x7f8ff01611e0 2024-03-14T12:14:01.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7fa06effd640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa05406bcd0 0x7fa05406e190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa05c038000 tx=0x7fa05c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7f7ef0a54640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7ed006bca0 0x7f7ed006e160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00011e0 tx=0x7f7ee0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7f911b7fe640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f910806bca0 0x7f910806e160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9114009a40 tx=0x7f911400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7f362986d640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f360806bc20 0x7f360806e0e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f361400a8f0 tx=0x7f3614008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7fefa0ff9640 1 -- 172.21.15.82:0/778940614 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fef8c003bc0 con 0x7fefa40b41a0 2024-03-14T12:14:01.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7fbd7a59c640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd5c06bca0 0x7fbd5c06e160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b9e0 tx=0x7fbd70030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7fefa0ff9640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8406bca0 0x7fef8406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.302+0000 7fefa37fe640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8406bca0 0x7fef8406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.303+0000 7fefa0ff9640 1 -- 172.21.15.82:0/778940614 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef8c0acc60 con 0x7fefa40b41a0 2024-03-14T12:14:01.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.303+0000 7fefa37fe640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8406bca0 0x7fef8406e160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef94006fd0 tx=0x7fef94008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.304+0000 7fa05bfff640 1 -- 172.21.15.82:0/1879889138 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa0680e4900 con 0x7fa0680dfee0 2024-03-14T12:14:01.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.306+0000 7f7ee97fa640 1 -- 172.21.15.82:0/1520349894 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7ee4014030 con 0x7f7eec071a70 2024-03-14T12:14:01.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.307+0000 7f361a7fc640 1 -- 172.21.15.82:0/4001626864 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f361c05a240 con 0x7f36240ce960 2024-03-14T12:14:01.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.308+0000 7f8fd77fe640 1 -- 172.21.15.82:0/3372335689 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8fe401a070 con 0x7f8ff01611e0 2024-03-14T12:14:01.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.308+0000 7f8fd77fe640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8fdc06bcd0 0x7f8fdc06e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.308+0000 7f8fd77fe640 1 -- 172.21.15.82:0/3372335689 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe408a7e0 con 0x7f8ff01611e0 2024-03-14T12:14:01.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.309+0000 7f8feed9d640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8fdc06bcd0 0x7f8fdc06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.309+0000 7f9103fff640 1 -- 172.21.15.82:0/1629123480 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9110040070 con 0x7f911c163280 2024-03-14T12:14:01.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.309+0000 7f1dbf7fe640 1 -- 172.21.15.82:0/3945671340 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000b040 con 0x7f1dc816d530 2024-03-14T12:14:01.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.309+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8161b00 con 0x7f1dc816d530 2024-03-14T12:14:01.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.309+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc81620c0 con 0x7f1dc816d530 2024-03-14T12:14:01.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.309+0000 7f8feed9d640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8fdc06bcd0 0x7f8fdc06e190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0034000 tx=0x7f8fe0009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.310+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc81622c0 con 0x7f1dc816d530 2024-03-14T12:14:01.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.311+0000 7fefa0ff9640 1 -- 172.21.15.82:0/778940614 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fef8c03d360 con 0x7fefa40b41a0 2024-03-14T12:14:01.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.313+0000 7fbd6affd640 1 -- 172.21.15.82:0/3929429408 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbd6c024020 con 0x7fbd740e5430 2024-03-14T12:14:01.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.313+0000 7f1dbf7fe640 1 -- 172.21.15.82:0/3945671340 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db000fd90 con 0x7f1dc816d530 2024-03-14T12:14:01.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.313+0000 7f1dbf7fe640 1 -- 172.21.15.82:0/3945671340 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db001c680 con 0x7f1dc816d530 2024-03-14T12:14:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.314+0000 7f8fd77fe640 1 -- 172.21.15.82:0/3372335689 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8fe4058cf0 con 0x7f8ff01611e0 2024-03-14T12:14:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.315+0000 7f1dbf7fe640 1 -- 172.21.15.82:0/3945671340 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1db0013440 con 0x7f1dc816d530 2024-03-14T12:14:01.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.316+0000 7f1dbf7fe640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1da806bc00 0x7f1da806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.316+0000 7f1dbf7fe640 1 -- 172.21.15.82:0/3945671340 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1db008ba70 con 0x7f1dc816d530 2024-03-14T12:14:01.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.317+0000 7f1dc6ffd640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1da806bc00 0x7f1da806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.317+0000 7f1dc6ffd640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1da806bc00 0x7f1da806e0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1db8036000 tx=0x7f1db8030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.318+0000 7f1dbf7fe640 1 -- 172.21.15.82:0/3945671340 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1dc81622c0 con 0x7f1dc816d530 2024-03-14T12:14:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.717+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa0680057d0 con 0x7fa0680dfee0 2024-03-14T12:14:01.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.718+0000 7fa05bfff640 1 -- 172.21.15.82:0/1879889138 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0640590a0 con 0x7fa0680dfee0 2024-03-14T12:14:01.718 INFO:teuthology.orchestra.run.smithi082.stdout:55834574855 2024-03-14T12:14:01.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.719+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f360c005850 con 0x7f36240ce960 2024-03-14T12:14:01.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.719+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7eec15eb60 con 0x7f7eec071a70 2024-03-14T12:14:01.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.719+0000 7f361a7fc640 1 -- 172.21.15.82:0/4001626864 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f361c059b40 con 0x7f36240ce960 2024-03-14T12:14:01.720 INFO:teuthology.orchestra.run.smithi082.stdout:55834574855 2024-03-14T12:14:01.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.720+0000 7f7ee97fa640 1 -- 172.21.15.82:0/1520349894 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ee4059ac0 con 0x7f7eec071a70 2024-03-14T12:14:01.720 INFO:teuthology.orchestra.run.smithi082.stdout:55834574856 2024-03-14T12:14:01.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.721+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fefa4074690 con 0x7fefa40b41a0 2024-03-14T12:14:01.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.722+0000 7fefa0ff9640 1 -- 172.21.15.82:0/778940614 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef8c07b850 con 0x7fefa40b41a0 2024-03-14T12:14:01.722 INFO:teuthology.orchestra.run.smithi082.stdout:55834574856 2024-03-14T12:14:01.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.722+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa05406bcd0 msgr2=0x7fa05406e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.722+0000 7fa0748e2640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa05406bcd0 0x7fa05406e190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa05c038000 tx=0x7fa05c008030 comp rx=0 tx=0).stop 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.722+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680dfee0 msgr2=0x7fa0680e21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.722+0000 7fa0748e2640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680dfee0 0x7fa0680e21d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e8b0 tx=0x7fa06400ed70 comp rx=0 tx=0).stop 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 shutdown_connections 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0680e06b0 0x7fa0680e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0680e40f0 0x7fa06804efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa05406bcd0 0x7fa05406e190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 --2- 172.21.15.82:0/1879889138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa0680dfee0 0x7fa0680e21d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 >> 172.21.15.82:0/1879889138 conn(0x7fa06801ef30 msgr2=0x7fa068053e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f360806bc20 msgr2=0x7f360806e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f360806bc20 0x7f360806e0e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f361400a8f0 tx=0x7f3614008040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240ce960 msgr2=0x7f36241f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240ce960 0x7f36241f6150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f361c00bd00 tx=0x7f361c00bdf0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f911c0b1c80 con 0x7f911c163280 2024-03-14T12:14:01.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 shutdown_connections 2024-03-14T12:14:01.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7fa0748e2640 1 -- 172.21.15.82:0/1879889138 wait complete. 2024-03-14T12:14:01.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7ed006bca0 msgr2=0x7f7ed006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 shutdown_connections 2024-03-14T12:14:01.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7ed006bca0 0x7f7ed006e160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00011e0 tx=0x7f7ee0009040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36240cef80 0x7f36241f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240ce960 0x7f36241f6150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f360806bc20 0x7f360806e0e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 --2- 172.21.15.82:0/4001626864 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240722e0 0x7f362415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 >> 172.21.15.82:0/4001626864 conn(0x7f36240b13f0 msgr2=0x7f36240cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eec071a70 msgr2=0x7f7eec15c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eec071a70 0x7f7eec15c4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400ccc0 tx=0x7f7ee400a650 comp rx=0 tx=0).stop 2024-03-14T12:14:01.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 shutdown_connections 2024-03-14T12:14:01.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.723+0000 7f362bad1640 1 -- 172.21.15.82:0/4001626864 wait complete. 2024-03-14T12:14:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 shutdown_connections 2024-03-14T12:14:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7eec071a70 0x7f7eec15c4c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7ed006bca0 0x7f7ed006e160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7eec0704a0 0x7f7eec15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 --2- 172.21.15.82:0/1520349894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7eec06df90 0x7f7eec15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 >> 172.21.15.82:0/1520349894 conn(0x7f7eec0b13f0 msgr2=0x7f7eec160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f9103fff640 1 -- 172.21.15.82:0/1629123480 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f911007a6e0 con 0x7f911c163280 2024-03-14T12:14:01.729 INFO:teuthology.orchestra.run.smithi082.stdout:55834574857 2024-03-14T12:14:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 shutdown_connections 2024-03-14T12:14:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.724+0000 7f7ef2cb8640 1 -- 172.21.15.82:0/1520349894 wait complete. 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.725+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8406bca0 msgr2=0x7fef8406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.725+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8406bca0 0x7fef8406e160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef94006fd0 tx=0x7fef94008040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.725+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa40b41a0 msgr2=0x7fefa40b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.725+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa40b41a0 0x7fefa40b7ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef8c033000 tx=0x7fef8c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 shutdown_connections 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8406bca0 0x7fef8406e160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefa40d1440 0x7fefa41eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa40b41a0 0x7fefa40b7ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 --2- 172.21.15.82:0/778940614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa40b36b0 0x7fefa41eadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 >> 172.21.15.82:0/778940614 conn(0x7fefa409d030 msgr2=0x7fefa406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 shutdown_connections 2024-03-14T12:14:01.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.726+0000 7fefa9efc640 1 -- 172.21.15.82:0/778940614 wait complete. 2024-03-14T12:14:01.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.727+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f910806bca0 msgr2=0x7f910806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.727+0000 7f9120ad5640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f910806bca0 0x7f910806e160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9114009a40 tx=0x7f911400b040 comp rx=0 tx=0).stop 2024-03-14T12:14:01.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.727+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f911c163280 msgr2=0x7f911c160360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.727+0000 7f9120ad5640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f911c163280 0x7f911c160360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f911c074970 tx=0x7f91100046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 shutdown_connections 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f910806bca0 0x7f910806e160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f911c163280 0x7f911c160360 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f911c15c630 0x7f911c1fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 --2- 172.21.15.82:0/1629123480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f911c06daf0 0x7f911c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 >> 172.21.15.82:0/1629123480 conn(0x7f911c0b1560 msgr2=0x7f911c0ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 shutdown_connections 2024-03-14T12:14:01.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.728+0000 7f9120ad5640 1 -- 172.21.15.82:0/1629123480 wait complete. 2024-03-14T12:14:01.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.732+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8fd0005850 con 0x7f8ff01611e0 2024-03-14T12:14:01.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.733+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd74052850 con 0x7fbd740e5430 2024-03-14T12:14:01.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.733+0000 7f8fd77fe640 1 -- 172.21.15.82:0/3372335689 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8fe405cd90 con 0x7f8ff01611e0 2024-03-14T12:14:01.733 INFO:teuthology.orchestra.run.smithi082.stdout:55834574854 2024-03-14T12:14:01.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.733+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1dc804f050 con 0x7f1dc816d530 2024-03-14T12:14:01.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.734+0000 7fbd6affd640 1 -- 172.21.15.82:0/3929429408 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbd6c059260 con 0x7fbd740e5430 2024-03-14T12:14:01.734 INFO:teuthology.orchestra.run.smithi082.stdout:55834574855 2024-03-14T12:14:01.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.734+0000 7f1dbf7fe640 1 -- 172.21.15.82:0/3945671340 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1db005a650 con 0x7f1dc816d530 2024-03-14T12:14:01.735 INFO:teuthology.orchestra.run.smithi082.stdout:55834574856 2024-03-14T12:14:01.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8fdc06bcd0 msgr2=0x7f8fdc06e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8fdc06bcd0 0x7f8fdc06e190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0034000 tx=0x7f8fe0009030 comp rx=0 tx=0).stop 2024-03-14T12:14:01.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff01611e0 msgr2=0x7f8ff0071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff01611e0 0x7f8ff0071920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ff006be10 tx=0x7f8fe400e970 comp rx=0 tx=0).stop 2024-03-14T12:14:01.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 shutdown_connections 2024-03-14T12:14:01.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ff0164e70 0x7f8ff0071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff01611e0 0x7f8ff0071920 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ff006be10 tx=0x7f8fe400e970 comp rx=0 tx=0).stop 2024-03-14T12:14:01.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8fdc06bcd0 0x7f8fdc06e190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 --2- 172.21.15.82:0/3372335689 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ff0200040 0x7f8ff0164930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 >> 172.21.15.82:0/3372335689 conn(0x7f8ff00b1930 msgr2=0x7f8ff00b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 shutdown_connections 2024-03-14T12:14:01.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7f8ff5214640 1 -- 172.21.15.82:0/3372335689 wait complete. 2024-03-14T12:14:01.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd5c06bca0 msgr2=0x7fbd5c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7fbd80f37640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd5c06bca0 0x7fbd5c06e160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b9e0 tx=0x7fbd70030000 comp rx=0 tx=0).stop 2024-03-14T12:14:01.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd740e5430 msgr2=0x7fbd740e58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.737+0000 7fbd80f37640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd740e5430 0x7fbd740e58e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd7404f060 tx=0x7fbd6c011920 comp rx=0 tx=0).stop 2024-03-14T12:14:01.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 shutdown_connections 2024-03-14T12:14:01.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd5c06bca0 0x7fbd5c06e160 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b9e0 tx=0x7fbd70030000 comp rx=0 tx=0).stop 2024-03-14T12:14:01.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd740e5430 0x7fbd740e58e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd7404e450 0x7fbd7404e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 --2- 172.21.15.82:0/3929429408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd640039b0 0x7fbd740d3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 >> 172.21.15.82:0/3929429408 conn(0x7fbd74009fe0 msgr2=0x7fbd74035b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 shutdown_connections 2024-03-14T12:14:01.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.738+0000 7fbd80f37640 1 -- 172.21.15.82:0/3929429408 wait complete. 2024-03-14T12:14:01.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1da806bc00 msgr2=0x7f1da806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1da806bc00 0x7f1da806e0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1db8036000 tx=0x7f1db8030000 comp rx=0 tx=0).stop 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc816d530 msgr2=0x7f1dc808a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc816d530 0x7f1dc808a4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1db0011950 tx=0x7f1db0011e10 comp rx=0 tx=0).stop 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 shutdown_connections 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc8167ca0 0x7f1dc808aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1da806bc00 0x7f1da806e0c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dc816d530 0x7f1dc808a4f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 --2- 172.21.15.82:0/3945671340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dc8156c40 0x7f1dc8167760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 >> 172.21.15.82:0/3945671340 conn(0x7f1dc80cf930 msgr2=0x7f1dc80cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 shutdown_connections 2024-03-14T12:14:01.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.739+0000 7f1dcd7f1640 1 -- 172.21.15.82:0/3945671340 wait complete. 2024-03-14T12:14:01.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-03-14T12:14:01.747 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-03-14T12:14:01.748 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-03-14T12:14:01.749 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-03-14T12:14:01.750 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-03-14T12:14:01.753 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-03-14T12:14:01.757 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-03-14T12:14:01.758 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-03-14T12:14:01.758 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:01.758 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-14T12:14:01.758 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:14:01.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.897+0000 7f76feb16640 1 Processor -- start 2024-03-14T12:14:01.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.897+0000 7f76feb16640 1 -- start start 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76feb16640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f76f80cdae0 0x7f76f806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76feb16640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f80d1000 con 0x7f76f8162350 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76feb16640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f80cdec0 con 0x7f76f806cf80 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76feb16640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f80d0b70 con 0x7f76f80cdae0 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76fd0b3640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f76f8162350 0x7f76f80cc0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54198/0 (socket says 172.21.15.82:54198) 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76fd0b3640 1 -- 172.21.15.82:0/3792776830 learned_addr learned my addr 172.21.15.82:0/3792776830 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76fc8b2640 1 --2- 172.21.15.82:0/3792776830 >> v2:172.21.15.164:3300/0 conn(0x7f76f80cdae0 0x7f76f806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041898636 0 0) 0x7f76f0001490 con 0x7f76f8162350 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76e8003520 con 0x7f76f8162350 2024-03-14T12:14:01.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1376248410 0 0) 0x7f76e8003520 con 0x7f76f8162350 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 >> v2:172.21.15.164:3300/0 conn(0x7f76f80cdae0 msgr2=0x7f76f806ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 --2- 172.21.15.82:0/3792776830 >> v2:172.21.15.164:3300/0 conn(0x7f76f80cdae0 0x7f76f806ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 >> v1:172.21.15.164:6789/0 conn(0x7f76f806cf80 legacy=0x7f76f8161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f80cff60 con 0x7f76f8162350 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f76f0001bc0 con 0x7f76f8162350 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 >> v1:172.21.15.82:6789/0 conn(0x7f76f8162350 legacy=0x7f76f80cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e80039b0 0x7f76e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8004800 0x7f76e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e80051c0 0x7f76e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f80d0b70 con 0x7f76e8004800 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e800c070 con 0x7f76e80039b0 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e800c270 con 0x7f76e80051c0 2024-03-14T12:14:01.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76fc8b2640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e80039b0 0x7f76e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.898+0000 7f76f7fff640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8004800 0x7f76e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76fd0b3640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e80051c0 0x7f76e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f7fff640 1 -- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e80051c0 msgr2=0x7f76e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f7fff640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e80051c0 0x7f76e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f7fff640 1 -- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e80039b0 msgr2=0x7f76e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f7fff640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e80039b0 0x7f76e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f7fff640 1 -- 172.21.15.82:0/3792776830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8003520 con 0x7f76e8004800 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76fd0b3640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e80051c0 0x7f76e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76fd0b3640 1 -- 172.21.15.82:0/3792776830 reap_dead start 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f7fff640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8004800 0x7f76e8004c80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76e4001750 tx=0x7f76e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b32222263f6d934 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4001370 con 0x7f76e8004800 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e40052d0 con 0x7f76e8004800 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.899+0000 7f76f77fe640 1 -- 172.21.15.82:0/3792776830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003830 con 0x7f76e8004800 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.900+0000 7f76feb16640 1 -- 172.21.15.82:0/3792776830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8004800 msgr2=0x7f76e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.900+0000 7f76feb16640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8004800 0x7f76e8004c80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76e4001750 tx=0x7f76e8005830 comp rx=0 tx=0).stop 2024-03-14T12:14:01.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.900+0000 7f76feb16640 1 -- 172.21.15.82:0/3792776830 shutdown_connections 2024-03-14T12:14:01.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.900+0000 7f76feb16640 1 --2- 172.21.15.82:0/3792776830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e8004800 0x7f76e8004c80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.900+0000 7f76feb16640 1 -- 172.21.15.82:0/3792776830 >> 172.21.15.82:0/3792776830 conn(0x7f76f80b3610 msgr2=0x7f76f80b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:01.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.900+0000 7f76feb16640 1 -- 172.21.15.82:0/3792776830 shutdown_connections 2024-03-14T12:14:01.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.900+0000 7f76feb16640 1 -- 172.21.15.82:0/3792776830 wait complete. 2024-03-14T12:14:01.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.901+0000 7f76feb16640 1 Processor -- start 2024-03-14T12:14:01.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.901+0000 7f76feb16640 1 -- start start 2024-03-14T12:14:01.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76feb16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8004800 0x7f76f815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76feb16640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f815e7d0 0x7f76f815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76feb16640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76f81f3ec0 0x7f76f81f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76feb16640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f80d0800 con 0x7f76f815e7d0 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76feb16640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f80d0520 con 0x7f76e8004800 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76feb16640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f80b4360 con 0x7f76f81f3ec0 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f815e7d0 0x7f76f815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f815e7d0 0x7f76f815ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51168/0 (socket says 172.21.15.82:51168) 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 -- 172.21.15.82:0/1522384726 learned_addr learned my addr 172.21.15.82:0/1522384726 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76fc8b2640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8004800 0x7f76f815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76fd0b3640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76f81f3ec0 0x7f76f81f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 -- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76f81f3ec0 msgr2=0x7f76f81f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76f81f3ec0 0x7f76f81f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 -- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8004800 msgr2=0x7f76f815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8004800 0x7f76f815cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:01.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76f7fff640 1 -- 172.21.15.82:0/1522384726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f81f4bf0 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.902+0000 7f76fd0b3640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76f81f3ec0 0x7f76f81f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.903+0000 7f76f7fff640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f815e7d0 0x7f76f815ec50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005040 tx=0x7f76e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.903+0000 7f76f57fa640 1 -- 172.21.15.82:0/1522384726 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4041020 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.903+0000 7f76f57fa640 1 -- 172.21.15.82:0/1522384726 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4005480 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.903+0000 7f76f57fa640 1 -- 172.21.15.82:0/1522384726 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4007a30 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.903+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f80af3f0 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.903+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f80af9b0 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.904+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f804f050 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.904+0000 7f76f57fa640 1 -- 172.21.15.82:0/1522384726 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f76e4044020 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.904+0000 7f76f57fa640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76dc06bc00 0x7f76dc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.904+0000 7f76f57fa640 1 -- 172.21.15.82:0/1522384726 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76e40ac3b0 con 0x7f76f815e7d0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.904+0000 7f76fc8b2640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76dc06bc00 0x7f76dc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.905+0000 7f76fc8b2640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76dc06bc00 0x7f76dc06e0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009c40 tx=0x7f76ec009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:01.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:01.910+0000 7f76f57fa640 1 -- 172.21.15.82:0/1522384726 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f76e404b030 con 0x7f76f815e7d0 2024-03-14T12:14:02.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.079+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76f815d270 con 0x7f76dc06bc00 2024-03-14T12:14:02.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.085+0000 7f76f57fa640 1 -- 172.21.15.82:0/1522384726 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f76f815d270 con 0x7f76dc06bc00 2024-03-14T12:14:02.088 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:02.089 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76dc06bc00 msgr2=0x7f76dc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76dc06bc00 0x7f76dc06e0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009c40 tx=0x7f76ec009190 comp rx=0 tx=0).stop 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f815e7d0 msgr2=0x7f76f815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f815e7d0 0x7f76f815ec50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005040 tx=0x7f76e4001120 comp rx=0 tx=0).stop 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 shutdown_connections 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76dc06bc00 0x7f76dc06e0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76f81f3ec0 0x7f76f81f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f815e7d0 0x7f76f815ec50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 --2- 172.21.15.82:0/1522384726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8004800 0x7f76f815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 >> 172.21.15.82:0/1522384726 conn(0x7f76f80b3610 msgr2=0x7f76f815fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:02.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 shutdown_connections 2024-03-14T12:14:02.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.088+0000 7f76feb16640 1 -- 172.21.15.82:0/1522384726 wait complete. 2024-03-14T12:14:02.099 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-03-14T12:14:01.500062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.938828"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478859+0000","last_change":"2024-03-14T12:13:51.582275+0000","last_active":"2024-03-14T12:13:54.478859+0000","last_peered":"2024-03-14T12:13:54.478859+0000","last_clean":"2024-03-14T12:13:54.478859+0000","last_became_active":"2024-03-14T12:13:51.581999+0000","last_became_peered":"2024-03-14T12:13:51.581999+0000","last_unstale":"2024-03-14T12:13:54.478859+0000","last_undegraded":"2024-03-14T12:13:54.478859+0000","last_fullsized":"2024-03-14T12:13:54.478859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574007+0000","last_change":"2024-03-14T12:13:51.575382+0000","last_active":"2024-03-14T12:13:52.574007+0000","last_peered":"2024-03-14T12:13:52.574007+0000","last_clean":"2024-03-14T12:13:52.574007+0000","last_became_active":"2024-03-14T12:13:51.575139+0000","last_became_peered":"2024-03-14T12:13:51.575139+0000","last_unstale":"2024-03-14T12:13:52.574007+0000","last_undegraded":"2024-03-14T12:13:52.574007+0000","last_fullsized":"2024-03-14T12:13:52.574007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777473+0000","last_change":"2024-03-14T12:13:51.574924+0000","last_active":"2024-03-14T12:13:52.777473+0000","last_peered":"2024-03-14T12:13:52.777473+0000","last_clean":"2024-03-14T12:13:52.777473+0000","last_became_active":"2024-03-14T12:13:51.574299+0000","last_became_peered":"2024-03-14T12:13:51.574299+0000","last_unstale":"2024-03-14T12:13:52.777473+0000","last_undegraded":"2024-03-14T12:13:52.777473+0000","last_fullsized":"2024-03-14T12:13:52.777473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579676+0000","last_change":"2024-03-14T12:13:51.581446+0000","last_active":"2024-03-14T12:13:52.579676+0000","last_peered":"2024-03-14T12:13:52.579676+0000","last_clean":"2024-03-14T12:13:52.579676+0000","last_became_active":"2024-03-14T12:13:51.581192+0000","last_became_peered":"2024-03-14T12:13:51.581192+0000","last_unstale":"2024-03-14T12:13:52.579676+0000","last_undegraded":"2024-03-14T12:13:52.579676+0000","last_fullsized":"2024-03-14T12:13:52.579676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478459+0000","last_change":"2024-03-14T12:13:51.587466+0000","last_active":"2024-03-14T12:13:54.478459+0000","last_peered":"2024-03-14T12:13:54.478459+0000","last_clean":"2024-03-14T12:13:54.478459+0000","last_became_active":"2024-03-14T12:13:51.585375+0000","last_became_peered":"2024-03-14T12:13:51.585375+0000","last_unstale":"2024-03-14T12:13:54.478459+0000","last_undegraded":"2024-03-14T12:13:54.478459+0000","last_fullsized":"2024-03-14T12:13:54.478459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580498+0000","last_change":"2024-03-14T12:13:51.580794+0000","last_active":"2024-03-14T12:13:52.580498+0000","last_peered":"2024-03-14T12:13:52.580498+0000","last_clean":"2024-03-14T12:13:52.580498+0000","last_became_active":"2024-03-14T12:13:51.580511+0000","last_became_peered":"2024-03-14T12:13:51.580511+0000","last_unstale":"2024-03-14T12:13:52.580498+0000","last_undegraded":"2024-03-14T12:13:52.580498+0000","last_fullsized":"2024-03-14T12:13:52.580498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478351+0000","last_change":"2024-03-14T12:13:51.588949+0000","last_active":"2024-03-14T12:13:54.478351+0000","last_peered":"2024-03-14T12:13:54.478351+0000","last_clean":"2024-03-14T12:13:54.478351+0000","last_became_active":"2024-03-14T12:13:51.588671+0000","last_became_peered":"2024-03-14T12:13:51.588671+0000","last_unstale":"2024-03-14T12:13:54.478351+0000","last_undegraded":"2024-03-14T12:13:54.478351+0000","last_fullsized":"2024-03-14T12:13:54.478351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477849+0000","last_change":"2024-03-14T12:13:51.579815+0000","last_active":"2024-03-14T12:13:54.477849+0000","last_peered":"2024-03-14T12:13:54.477849+0000","last_clean":"2024-03-14T12:13:54.477849+0000","last_became_active":"2024-03-14T12:13:51.579580+0000","last_became_peered":"2024-03-14T12:13:51.579580+0000","last_unstale":"2024-03-14T12:13:54.477849+0000","last_undegraded":"2024-03-14T12:13:54.477849+0000","last_fullsized":"2024-03-14T12:13:54.477849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478866+0000","last_change":"2024-03-14T12:13:51.593442+0000","last_active":"2024-03-14T12:13:54.478866+0000","last_peered":"2024-03-14T12:13:54.478866+0000","last_clean":"2024-03-14T12:13:54.478866+0000","last_became_active":"2024-03-14T12:13:51.593151+0000","last_became_peered":"2024-03-14T12:13:51.593151+0000","last_unstale":"2024-03-14T12:13:54.478866+0000","last_undegraded":"2024-03-14T12:13:54.478866+0000","last_fullsized":"2024-03-14T12:13:54.478866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582259+0000","last_change":"2024-03-14T12:13:51.584838+0000","last_active":"2024-03-14T12:13:52.582259+0000","last_peered":"2024-03-14T12:13:52.582259+0000","last_clean":"2024-03-14T12:13:52.582259+0000","last_became_active":"2024-03-14T12:13:51.584501+0000","last_became_peered":"2024-03-14T12:13:51.584501+0000","last_unstale":"2024-03-14T12:13:52.582259+0000","last_undegraded":"2024-03-14T12:13:52.582259+0000","last_fullsized":"2024-03-14T12:13:52.582259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581960+0000","last_change":"2024-03-14T12:13:51.584374+0000","last_active":"2024-03-14T12:13:52.581960+0000","last_peered":"2024-03-14T12:13:52.581960+0000","last_clean":"2024-03-14T12:13:52.581960+0000","last_became_active":"2024-03-14T12:13:51.584094+0000","last_became_peered":"2024-03-14T12:13:51.584094+0000","last_unstale":"2024-03-14T12:13:52.581960+0000","last_undegraded":"2024-03-14T12:13:52.581960+0000","last_fullsized":"2024-03-14T12:13:52.581960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580082+0000","last_change":"2024-03-14T12:13:51.583802+0000","last_active":"2024-03-14T12:13:52.580082+0000","last_peered":"2024-03-14T12:13:52.580082+0000","last_clean":"2024-03-14T12:13:52.580082+0000","last_became_active":"2024-03-14T12:13:51.583284+0000","last_became_peered":"2024-03-14T12:13:51.583284+0000","last_unstale":"2024-03-14T12:13:52.580082+0000","last_undegraded":"2024-03-14T12:13:52.580082+0000","last_fullsized":"2024-03-14T12:13:52.580082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478086+0000","last_change":"2024-03-14T12:13:51.578671+0000","last_active":"2024-03-14T12:13:54.478086+0000","last_peered":"2024-03-14T12:13:54.478086+0000","last_clean":"2024-03-14T12:13:54.478086+0000","last_became_active":"2024-03-14T12:13:51.578427+0000","last_became_peered":"2024-03-14T12:13:51.578427+0000","last_unstale":"2024-03-14T12:13:54.478086+0000","last_undegraded":"2024-03-14T12:13:54.478086+0000","last_fullsized":"2024-03-14T12:13:54.478086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777400+0000","last_change":"2024-03-14T12:13:51.581289+0000","last_active":"2024-03-14T12:13:52.777400+0000","last_peered":"2024-03-14T12:13:52.777400+0000","last_clean":"2024-03-14T12:13:52.777400+0000","last_became_active":"2024-03-14T12:13:51.580645+0000","last_became_peered":"2024-03-14T12:13:51.580645+0000","last_unstale":"2024-03-14T12:13:52.777400+0000","last_undegraded":"2024-03-14T12:13:52.777400+0000","last_fullsized":"2024-03-14T12:13:52.777400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:01.290201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573796+0000","last_change":"2024-03-14T12:13:51.571032+0000","last_active":"2024-03-14T12:13:52.573796+0000","last_peered":"2024-03-14T12:13:52.573796+0000","last_clean":"2024-03-14T12:13:52.573796+0000","last_became_active":"2024-03-14T12:13:51.570723+0000","last_became_peered":"2024-03-14T12:13:51.570723+0000","last_unstale":"2024-03-14T12:13:52.573796+0000","last_undegraded":"2024-03-14T12:13:52.573796+0000","last_fullsized":"2024-03-14T12:13:52.573796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580831+0000","last_change":"2024-03-14T12:13:51.582495+0000","last_active":"2024-03-14T12:13:52.580831+0000","last_peered":"2024-03-14T12:13:52.580831+0000","last_clean":"2024-03-14T12:13:52.580831+0000","last_became_active":"2024-03-14T12:13:51.582170+0000","last_became_peered":"2024-03-14T12:13:51.582170+0000","last_unstale":"2024-03-14T12:13:52.580831+0000","last_undegraded":"2024-03-14T12:13:52.580831+0000","last_fullsized":"2024-03-14T12:13:52.580831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580861+0000","last_change":"2024-03-14T12:13:51.582330+0000","last_active":"2024-03-14T12:13:52.580861+0000","last_peered":"2024-03-14T12:13:52.580861+0000","last_clean":"2024-03-14T12:13:52.580861+0000","last_became_active":"2024-03-14T12:13:51.581920+0000","last_became_peered":"2024-03-14T12:13:51.581920+0000","last_unstale":"2024-03-14T12:13:52.580861+0000","last_undegraded":"2024-03-14T12:13:52.580861+0000","last_fullsized":"2024-03-14T12:13:52.580861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477706+0000","last_change":"2024-03-14T12:13:51.582708+0000","last_active":"2024-03-14T12:13:54.477706+0000","last_peered":"2024-03-14T12:13:54.477706+0000","last_clean":"2024-03-14T12:13:54.477706+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:54.477706+0000","last_undegraded":"2024-03-14T12:13:54.477706+0000","last_fullsized":"2024-03-14T12:13:54.477706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.479134+0000","last_change":"2024-03-14T12:13:51.587758+0000","last_active":"2024-03-14T12:13:54.479134+0000","last_peered":"2024-03-14T12:13:54.479134+0000","last_clean":"2024-03-14T12:13:54.479134+0000","last_became_active":"2024-03-14T12:13:51.586310+0000","last_became_peered":"2024-03-14T12:13:51.586310+0000","last_unstale":"2024-03-14T12:13:54.479134+0000","last_undegraded":"2024-03-14T12:13:54.479134+0000","last_fullsized":"2024-03-14T12:13:54.479134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:57:08.624764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582057+0000","last_change":"2024-03-14T12:13:51.589443+0000","last_active":"2024-03-14T12:13:52.582057+0000","last_peered":"2024-03-14T12:13:52.582057+0000","last_clean":"2024-03-14T12:13:52.582057+0000","last_became_active":"2024-03-14T12:13:51.589174+0000","last_became_peered":"2024-03-14T12:13:51.589174+0000","last_unstale":"2024-03-14T12:13:52.582057+0000","last_undegraded":"2024-03-14T12:13:52.582057+0000","last_fullsized":"2024-03-14T12:13:52.582057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777419+0000","last_change":"2024-03-14T12:13:51.580813+0000","last_active":"2024-03-14T12:13:52.777419+0000","last_peered":"2024-03-14T12:13:52.777419+0000","last_clean":"2024-03-14T12:13:52.777419+0000","last_became_active":"2024-03-14T12:13:51.580334+0000","last_became_peered":"2024-03-14T12:13:51.580334+0000","last_unstale":"2024-03-14T12:13:52.777419+0000","last_undegraded":"2024-03-14T12:13:52.777419+0000","last_fullsized":"2024-03-14T12:13:52.777419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580443+0000","last_change":"2024-03-14T12:13:51.583228+0000","last_active":"2024-03-14T12:13:52.580443+0000","last_peered":"2024-03-14T12:13:52.580443+0000","last_clean":"2024-03-14T12:13:52.580443+0000","last_became_active":"2024-03-14T12:13:51.582937+0000","last_became_peered":"2024-03-14T12:13:51.582937+0000","last_unstale":"2024-03-14T12:13:52.580443+0000","last_undegraded":"2024-03-14T12:13:52.580443+0000","last_fullsized":"2024-03-14T12:13:52.580443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477840+0000","last_change":"2024-03-14T12:13:51.582656+0000","last_active":"2024-03-14T12:13:54.477840+0000","last_peered":"2024-03-14T12:13:54.477840+0000","last_clean":"2024-03-14T12:13:54.477840+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:54.477840+0000","last_undegraded":"2024-03-14T12:13:54.477840+0000","last_fullsized":"2024-03-14T12:13:54.477840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580802+0000","last_change":"2024-03-14T12:13:51.579810+0000","last_active":"2024-03-14T12:13:52.580802+0000","last_peered":"2024-03-14T12:13:52.580802+0000","last_clean":"2024-03-14T12:13:52.580802+0000","last_became_active":"2024-03-14T12:13:51.579523+0000","last_became_peered":"2024-03-14T12:13:51.579523+0000","last_unstale":"2024-03-14T12:13:52.580802+0000","last_undegraded":"2024-03-14T12:13:52.580802+0000","last_fullsized":"2024-03-14T12:13:52.580802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478115+0000","last_change":"2024-03-14T12:13:51.582442+0000","last_active":"2024-03-14T12:13:54.478115+0000","last_peered":"2024-03-14T12:13:54.478115+0000","last_clean":"2024-03-14T12:13:54.478115+0000","last_became_active":"2024-03-14T12:13:51.582175+0000","last_became_peered":"2024-03-14T12:13:51.582175+0000","last_unstale":"2024-03-14T12:13:54.478115+0000","last_undegraded":"2024-03-14T12:13:54.478115+0000","last_fullsized":"2024-03-14T12:13:54.478115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582355+0000","last_change":"2024-03-14T12:13:51.590516+0000","last_active":"2024-03-14T12:13:52.582355+0000","last_peered":"2024-03-14T12:13:52.582355+0000","last_clean":"2024-03-14T12:13:52.582355+0000","last_became_active":"2024-03-14T12:13:51.590223+0000","last_became_peered":"2024-03-14T12:13:51.590223+0000","last_unstale":"2024-03-14T12:13:52.582355+0000","last_undegraded":"2024-03-14T12:13:52.582355+0000","last_fullsized":"2024-03-14T12:13:52.582355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573995+0000","last_change":"2024-03-14T12:13:51.574721+0000","last_active":"2024-03-14T12:13:52.573995+0000","last_peered":"2024-03-14T12:13:52.573995+0000","last_clean":"2024-03-14T12:13:52.573995+0000","last_became_active":"2024-03-14T12:13:51.574443+0000","last_became_peered":"2024-03-14T12:13:51.574443+0000","last_unstale":"2024-03-14T12:13:52.573995+0000","last_undegraded":"2024-03-14T12:13:52.573995+0000","last_fullsized":"2024-03-14T12:13:52.573995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478558+0000","last_change":"2024-03-14T12:13:51.582640+0000","last_active":"2024-03-14T12:13:54.478558+0000","last_peered":"2024-03-14T12:13:54.478558+0000","last_clean":"2024-03-14T12:13:54.478558+0000","last_became_active":"2024-03-14T12:13:51.582355+0000","last_became_peered":"2024-03-14T12:13:51.582355+0000","last_unstale":"2024-03-14T12:13:54.478558+0000","last_undegraded":"2024-03-14T12:13:54.478558+0000","last_fullsized":"2024-03-14T12:13:54.478558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777477+0000","last_change":"2024-03-14T12:13:51.577794+0000","last_active":"2024-03-14T12:13:52.777477+0000","last_peered":"2024-03-14T12:13:52.777477+0000","last_clean":"2024-03-14T12:13:52.777477+0000","last_became_active":"2024-03-14T12:13:51.577518+0000","last_became_peered":"2024-03-14T12:13:51.577518+0000","last_unstale":"2024-03-14T12:13:52.777477+0000","last_undegraded":"2024-03-14T12:13:52.777477+0000","last_fullsized":"2024-03-14T12:13:52.777477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580629+0000","last_change":"2024-03-14T12:13:51.580193+0000","last_active":"2024-03-14T12:13:52.580629+0000","last_peered":"2024-03-14T12:13:52.580629+0000","last_clean":"2024-03-14T12:13:52.580629+0000","last_became_active":"2024-03-14T12:13:51.577667+0000","last_became_peered":"2024-03-14T12:13:51.577667+0000","last_unstale":"2024-03-14T12:13:52.580629+0000","last_undegraded":"2024-03-14T12:13:52.580629+0000","last_fullsized":"2024-03-14T12:13:52.580629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477630+0000","last_change":"2024-03-14T12:13:51.579889+0000","last_active":"2024-03-14T12:13:54.477630+0000","last_peered":"2024-03-14T12:13:54.477630+0000","last_clean":"2024-03-14T12:13:54.477630+0000","last_became_active":"2024-03-14T12:13:51.579585+0000","last_became_peered":"2024-03-14T12:13:51.579585+0000","last_unstale":"2024-03-14T12:13:54.477630+0000","last_undegraded":"2024-03-14T12:13:54.477630+0000","last_fullsized":"2024-03-14T12:13:54.477630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581817+0000","last_change":"2024-03-14T12:13:51.577340+0000","last_active":"2024-03-14T12:13:52.581817+0000","last_peered":"2024-03-14T12:13:52.581817+0000","last_clean":"2024-03-14T12:13:52.581817+0000","last_became_active":"2024-03-14T12:13:51.577080+0000","last_became_peered":"2024-03-14T12:13:51.577080+0000","last_unstale":"2024-03-14T12:13:52.581817+0000","last_undegraded":"2024-03-14T12:13:52.581817+0000","last_fullsized":"2024-03-14T12:13:52.581817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579599+0000","last_change":"2024-03-14T12:13:51.573447+0000","last_active":"2024-03-14T12:13:52.579599+0000","last_peered":"2024-03-14T12:13:52.579599+0000","last_clean":"2024-03-14T12:13:52.579599+0000","last_became_active":"2024-03-14T12:13:51.573137+0000","last_became_peered":"2024-03-14T12:13:51.573137+0000","last_unstale":"2024-03-14T12:13:52.579599+0000","last_undegraded":"2024-03-14T12:13:52.579599+0000","last_fullsized":"2024-03-14T12:13:52.579599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478302+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:54.478302+0000","last_peered":"2024-03-14T12:13:54.478302+0000","last_clean":"2024-03-14T12:13:54.478302+0000","last_became_active":"2024-03-14T12:13:50.569748+0000","last_became_peered":"2024-03-14T12:13:50.569748+0000","last_unstale":"2024-03-14T12:13:54.478302+0000","last_undegraded":"2024-03-14T12:13:54.478302+0000","last_fullsized":"2024-03-14T12:13:54.478302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777628+0000","last_change":"2024-03-14T12:13:51.581108+0000","last_active":"2024-03-14T12:13:52.777628+0000","last_peered":"2024-03-14T12:13:52.777628+0000","last_clean":"2024-03-14T12:13:52.777628+0000","last_became_active":"2024-03-14T12:13:51.580829+0000","last_became_peered":"2024-03-14T12:13:51.580829+0000","last_unstale":"2024-03-14T12:13:52.777628+0000","last_undegraded":"2024-03-14T12:13:52.777628+0000","last_fullsized":"2024-03-14T12:13:52.777628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581951+0000","last_change":"2024-03-14T12:13:50.565826+0000","last_active":"2024-03-14T12:13:52.581951+0000","last_peered":"2024-03-14T12:13:52.581951+0000","last_clean":"2024-03-14T12:13:52.581951+0000","last_became_active":"2024-03-14T12:13:50.565553+0000","last_became_peered":"2024-03-14T12:13:50.565553+0000","last_unstale":"2024-03-14T12:13:52.581951+0000","last_undegraded":"2024-03-14T12:13:52.581951+0000","last_fullsized":"2024-03-14T12:13:52.581951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477885+0000","last_change":"2024-03-14T12:13:50.563462+0000","last_active":"2024-03-14T12:13:54.477885+0000","last_peered":"2024-03-14T12:13:54.477885+0000","last_clean":"2024-03-14T12:13:54.477885+0000","last_became_active":"2024-03-14T12:13:50.563198+0000","last_became_peered":"2024-03-14T12:13:50.563198+0000","last_unstale":"2024-03-14T12:13:54.477885+0000","last_undegraded":"2024-03-14T12:13:54.477885+0000","last_fullsized":"2024-03-14T12:13:54.477885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477930+0000","last_change":"2024-03-14T12:13:51.577103+0000","last_active":"2024-03-14T12:13:54.477930+0000","last_peered":"2024-03-14T12:13:54.477930+0000","last_clean":"2024-03-14T12:13:54.477930+0000","last_became_active":"2024-03-14T12:13:51.576602+0000","last_became_peered":"2024-03-14T12:13:51.576602+0000","last_unstale":"2024-03-14T12:13:54.477930+0000","last_undegraded":"2024-03-14T12:13:54.477930+0000","last_fullsized":"2024-03-14T12:13:54.477930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580881+0000","last_change":"2024-03-14T12:13:50.559389+0000","last_active":"2024-03-14T12:13:52.580881+0000","last_peered":"2024-03-14T12:13:52.580881+0000","last_clean":"2024-03-14T12:13:52.580881+0000","last_became_active":"2024-03-14T12:13:50.559100+0000","last_became_peered":"2024-03-14T12:13:50.559100+0000","last_unstale":"2024-03-14T12:13:52.580881+0000","last_undegraded":"2024-03-14T12:13:52.580881+0000","last_fullsized":"2024-03-14T12:13:52.580881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573924+0000","last_change":"2024-03-14T12:13:51.576591+0000","last_active":"2024-03-14T12:13:52.573924+0000","last_peered":"2024-03-14T12:13:52.573924+0000","last_clean":"2024-03-14T12:13:52.573924+0000","last_became_active":"2024-03-14T12:13:51.576232+0000","last_became_peered":"2024-03-14T12:13:51.576232+0000","last_unstale":"2024-03-14T12:13:52.573924+0000","last_undegraded":"2024-03-14T12:13:52.573924+0000","last_fullsized":"2024-03-14T12:13:52.573924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581963+0000","last_change":"2024-03-14T12:13:50.568392+0000","last_active":"2024-03-14T12:13:52.581963+0000","last_peered":"2024-03-14T12:13:52.581963+0000","last_clean":"2024-03-14T12:13:52.581963+0000","last_became_active":"2024-03-14T12:13:50.568127+0000","last_became_peered":"2024-03-14T12:13:50.568127+0000","last_unstale":"2024-03-14T12:13:52.581963+0000","last_undegraded":"2024-03-14T12:13:52.581963+0000","last_fullsized":"2024-03-14T12:13:52.581963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580126+0000","last_change":"2024-03-14T12:13:51.584429+0000","last_active":"2024-03-14T12:13:52.580126+0000","last_peered":"2024-03-14T12:13:52.580126+0000","last_clean":"2024-03-14T12:13:52.580126+0000","last_became_active":"2024-03-14T12:13:51.584098+0000","last_became_peered":"2024-03-14T12:13:51.584098+0000","last_unstale":"2024-03-14T12:13:52.580126+0000","last_undegraded":"2024-03-14T12:13:52.580126+0000","last_fullsized":"2024-03-14T12:13:52.580126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580241+0000","last_change":"2024-03-14T12:13:50.567672+0000","last_active":"2024-03-14T12:13:52.580241+0000","last_peered":"2024-03-14T12:13:52.580241+0000","last_clean":"2024-03-14T12:13:52.580241+0000","last_became_active":"2024-03-14T12:13:50.567366+0000","last_became_peered":"2024-03-14T12:13:50.567366+0000","last_unstale":"2024-03-14T12:13:52.580241+0000","last_undegraded":"2024-03-14T12:13:52.580241+0000","last_fullsized":"2024-03-14T12:13:52.580241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777722+0000","last_change":"2024-03-14T12:13:51.575322+0000","last_active":"2024-03-14T12:13:52.777722+0000","last_peered":"2024-03-14T12:13:52.777722+0000","last_clean":"2024-03-14T12:13:52.777722+0000","last_became_active":"2024-03-14T12:13:51.574694+0000","last_became_peered":"2024-03-14T12:13:51.574694+0000","last_unstale":"2024-03-14T12:13:52.777722+0000","last_undegraded":"2024-03-14T12:13:52.777722+0000","last_fullsized":"2024-03-14T12:13:52.777722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573967+0000","last_change":"2024-03-14T12:13:50.567057+0000","last_active":"2024-03-14T12:13:52.573967+0000","last_peered":"2024-03-14T12:13:52.573967+0000","last_clean":"2024-03-14T12:13:52.573967+0000","last_became_active":"2024-03-14T12:13:50.566800+0000","last_became_peered":"2024-03-14T12:13:50.566800+0000","last_unstale":"2024-03-14T12:13:52.573967+0000","last_undegraded":"2024-03-14T12:13:52.573967+0000","last_fullsized":"2024-03-14T12:13:52.573967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580236+0000","last_change":"2024-03-14T12:13:51.585716+0000","last_active":"2024-03-14T12:13:52.580236+0000","last_peered":"2024-03-14T12:13:52.580236+0000","last_clean":"2024-03-14T12:13:52.580236+0000","last_became_active":"2024-03-14T12:13:51.585226+0000","last_became_peered":"2024-03-14T12:13:51.585226+0000","last_unstale":"2024-03-14T12:13:52.580236+0000","last_undegraded":"2024-03-14T12:13:52.580236+0000","last_fullsized":"2024-03-14T12:13:52.580236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582041+0000","last_change":"2024-03-14T12:13:50.562666+0000","last_active":"2024-03-14T12:13:52.582041+0000","last_peered":"2024-03-14T12:13:52.582041+0000","last_clean":"2024-03-14T12:13:52.582041+0000","last_became_active":"2024-03-14T12:13:50.562367+0000","last_became_peered":"2024-03-14T12:13:50.562367+0000","last_unstale":"2024-03-14T12:13:52.582041+0000","last_undegraded":"2024-03-14T12:13:52.582041+0000","last_fullsized":"2024-03-14T12:13:52.582041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478606+0000","last_change":"2024-03-14T12:13:51.587183+0000","last_active":"2024-03-14T12:13:54.478606+0000","last_peered":"2024-03-14T12:13:54.478606+0000","last_clean":"2024-03-14T12:13:54.478606+0000","last_became_active":"2024-03-14T12:13:51.585685+0000","last_became_peered":"2024-03-14T12:13:51.585685+0000","last_unstale":"2024-03-14T12:13:54.478606+0000","last_undegraded":"2024-03-14T12:13:54.478606+0000","last_fullsized":"2024-03-14T12:13:54.478606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777610+0000","last_change":"2024-03-14T12:13:50.557053+0000","last_active":"2024-03-14T12:13:52.777610+0000","last_peered":"2024-03-14T12:13:52.777610+0000","last_clean":"2024-03-14T12:13:52.777610+0000","last_became_active":"2024-03-14T12:13:50.556737+0000","last_became_peered":"2024-03-14T12:13:50.556737+0000","last_unstale":"2024-03-14T12:13:52.777610+0000","last_undegraded":"2024-03-14T12:13:52.777610+0000","last_fullsized":"2024-03-14T12:13:52.777610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777653+0000","last_change":"2024-03-14T12:13:51.581665+0000","last_active":"2024-03-14T12:13:52.777653+0000","last_peered":"2024-03-14T12:13:52.777653+0000","last_clean":"2024-03-14T12:13:52.777653+0000","last_became_active":"2024-03-14T12:13:51.580980+0000","last_became_peered":"2024-03-14T12:13:51.580980+0000","last_unstale":"2024-03-14T12:13:52.777653+0000","last_undegraded":"2024-03-14T12:13:52.777653+0000","last_fullsized":"2024-03-14T12:13:52.777653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580677+0000","last_change":"2024-03-14T12:13:50.566403+0000","last_active":"2024-03-14T12:13:52.580677+0000","last_peered":"2024-03-14T12:13:52.580677+0000","last_clean":"2024-03-14T12:13:52.580677+0000","last_became_active":"2024-03-14T12:13:50.565782+0000","last_became_peered":"2024-03-14T12:13:50.565782+0000","last_unstale":"2024-03-14T12:13:52.580677+0000","last_undegraded":"2024-03-14T12:13:52.580677+0000","last_fullsized":"2024-03-14T12:13:52.580677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478689+0000","last_change":"2024-03-14T12:13:51.592512+0000","last_active":"2024-03-14T12:13:54.478689+0000","last_peered":"2024-03-14T12:13:54.478689+0000","last_clean":"2024-03-14T12:13:54.478689+0000","last_became_active":"2024-03-14T12:13:51.592273+0000","last_became_peered":"2024-03-14T12:13:51.592273+0000","last_unstale":"2024-03-14T12:13:54.478689+0000","last_undegraded":"2024-03-14T12:13:54.478689+0000","last_fullsized":"2024-03-14T12:13:54.478689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581835+0000","last_change":"2024-03-14T12:13:50.564239+0000","last_active":"2024-03-14T12:13:52.581835+0000","last_peered":"2024-03-14T12:13:52.581835+0000","last_clean":"2024-03-14T12:13:52.581835+0000","last_became_active":"2024-03-14T12:13:50.563920+0000","last_became_peered":"2024-03-14T12:13:50.563920+0000","last_unstale":"2024-03-14T12:13:52.581835+0000","last_undegraded":"2024-03-14T12:13:52.581835+0000","last_fullsized":"2024-03-14T12:13:52.581835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581784+0000","last_change":"2024-03-14T12:13:51.576682+0000","last_active":"2024-03-14T12:13:52.581784+0000","last_peered":"2024-03-14T12:13:52.581784+0000","last_clean":"2024-03-14T12:13:52.581784+0000","last_became_active":"2024-03-14T12:13:51.576420+0000","last_became_peered":"2024-03-14T12:13:51.576420+0000","last_unstale":"2024-03-14T12:13:52.581784+0000","last_undegraded":"2024-03-14T12:13:52.581784+0000","last_fullsized":"2024-03-14T12:13:52.581784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477551+0000","last_change":"2024-03-14T12:13:50.564352+0000","last_active":"2024-03-14T12:13:54.477551+0000","last_peered":"2024-03-14T12:13:54.477551+0000","last_clean":"2024-03-14T12:13:54.477551+0000","last_became_active":"2024-03-14T12:13:50.563979+0000","last_became_peered":"2024-03-14T12:13:50.563979+0000","last_unstale":"2024-03-14T12:13:54.477551+0000","last_undegraded":"2024-03-14T12:13:54.477551+0000","last_fullsized":"2024-03-14T12:13:54.477551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478395+0000","last_change":"2024-03-14T12:13:51.592382+0000","last_active":"2024-03-14T12:13:54.478395+0000","last_peered":"2024-03-14T12:13:54.478395+0000","last_clean":"2024-03-14T12:13:54.478395+0000","last_became_active":"2024-03-14T12:13:51.592109+0000","last_became_peered":"2024-03-14T12:13:51.592109+0000","last_unstale":"2024-03-14T12:13:54.478395+0000","last_undegraded":"2024-03-14T12:13:54.478395+0000","last_fullsized":"2024-03-14T12:13:54.478395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579929+0000","last_change":"2024-03-14T12:13:50.565493+0000","last_active":"2024-03-14T12:13:52.579929+0000","last_peered":"2024-03-14T12:13:52.579929+0000","last_clean":"2024-03-14T12:13:52.579929+0000","last_became_active":"2024-03-14T12:13:50.565247+0000","last_became_peered":"2024-03-14T12:13:50.565247+0000","last_unstale":"2024-03-14T12:13:52.579929+0000","last_undegraded":"2024-03-14T12:13:52.579929+0000","last_fullsized":"2024-03-14T12:13:52.579929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:39:10.529585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478754+0000","last_change":"2024-03-14T12:13:51.591007+0000","last_active":"2024-03-14T12:13:54.478754+0000","last_peered":"2024-03-14T12:13:54.478754+0000","last_clean":"2024-03-14T12:13:54.478754+0000","last_became_active":"2024-03-14T12:13:51.590750+0000","last_became_peered":"2024-03-14T12:13:51.590750+0000","last_unstale":"2024-03-14T12:13:54.478754+0000","last_undegraded":"2024-03-14T12:13:54.478754+0000","last_fullsized":"2024-03-14T12:13:54.478754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477792+0000","last_change":"2024-03-14T12:13:50.568050+0000","last_active":"2024-03-14T12:13:54.477792+0000","last_peered":"2024-03-14T12:13:54.477792+0000","last_clean":"2024-03-14T12:13:54.477792+0000","last_became_active":"2024-03-14T12:13:50.567713+0000","last_became_peered":"2024-03-14T12:13:50.567713+0000","last_unstale":"2024-03-14T12:13:54.477792+0000","last_undegraded":"2024-03-14T12:13:54.477792+0000","last_fullsized":"2024-03-14T12:13:54.477792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478989+0000","last_change":"2024-03-14T12:13:51.592562+0000","last_active":"2024-03-14T12:13:54.478989+0000","last_peered":"2024-03-14T12:13:54.478989+0000","last_clean":"2024-03-14T12:13:54.478989+0000","last_became_active":"2024-03-14T12:13:51.592255+0000","last_became_peered":"2024-03-14T12:13:51.592255+0000","last_unstale":"2024-03-14T12:13:54.478989+0000","last_undegraded":"2024-03-14T12:13:54.478989+0000","last_fullsized":"2024-03-14T12:13:54.478989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574041+0000","last_change":"2024-03-14T12:13:50.566339+0000","last_active":"2024-03-14T12:13:52.574041+0000","last_peered":"2024-03-14T12:13:52.574041+0000","last_clean":"2024-03-14T12:13:52.574041+0000","last_became_active":"2024-03-14T12:13:50.566058+0000","last_became_peered":"2024-03-14T12:13:50.566058+0000","last_unstale":"2024-03-14T12:13:52.574041+0000","last_undegraded":"2024-03-14T12:13:52.574041+0000","last_fullsized":"2024-03-14T12:13:52.574041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579993+0000","last_change":"2024-03-14T12:13:51.581453+0000","last_active":"2024-03-14T12:13:52.579993+0000","last_peered":"2024-03-14T12:13:52.579993+0000","last_clean":"2024-03-14T12:13:52.579993+0000","last_became_active":"2024-03-14T12:13:51.581168+0000","last_became_peered":"2024-03-14T12:13:51.581168+0000","last_unstale":"2024-03-14T12:13:52.579993+0000","last_undegraded":"2024-03-14T12:13:52.579993+0000","last_fullsized":"2024-03-14T12:13:52.579993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477730+0000","last_change":"2024-03-14T12:13:50.568172+0000","last_active":"2024-03-14T12:13:54.477730+0000","last_peered":"2024-03-14T12:13:54.477730+0000","last_clean":"2024-03-14T12:13:54.477730+0000","last_became_active":"2024-03-14T12:13:50.567825+0000","last_became_peered":"2024-03-14T12:13:50.567825+0000","last_unstale":"2024-03-14T12:13:54.477730+0000","last_undegraded":"2024-03-14T12:13:54.477730+0000","last_fullsized":"2024-03-14T12:13:54.477730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478581+0000","last_change":"2024-03-14T12:13:51.593995+0000","last_active":"2024-03-14T12:13:54.478581+0000","last_peered":"2024-03-14T12:13:54.478581+0000","last_clean":"2024-03-14T12:13:54.478581+0000","last_became_active":"2024-03-14T12:13:51.593473+0000","last_became_peered":"2024-03-14T12:13:51.593473+0000","last_unstale":"2024-03-14T12:13:54.478581+0000","last_undegraded":"2024-03-14T12:13:54.478581+0000","last_fullsized":"2024-03-14T12:13:54.478581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478363+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:13:54.478363+0000","last_peered":"2024-03-14T12:13:54.478363+0000","last_clean":"2024-03-14T12:13:54.478363+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:13:54.478363+0000","last_undegraded":"2024-03-14T12:13:54.478363+0000","last_fullsized":"2024-03-14T12:13:54.478363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580336+0000","last_change":"2024-03-14T12:13:50.565229+0000","last_active":"2024-03-14T12:13:52.580336+0000","last_peered":"2024-03-14T12:13:52.580336+0000","last_clean":"2024-03-14T12:13:52.580336+0000","last_became_active":"2024-03-14T12:13:50.564952+0000","last_became_peered":"2024-03-14T12:13:50.564952+0000","last_unstale":"2024-03-14T12:13:52.580336+0000","last_undegraded":"2024-03-14T12:13:52.580336+0000","last_fullsized":"2024-03-14T12:13:52.580336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478072+0000","last_change":"2024-03-14T12:13:51.573189+0000","last_active":"2024-03-14T12:13:54.478072+0000","last_peered":"2024-03-14T12:13:54.478072+0000","last_clean":"2024-03-14T12:13:54.478072+0000","last_became_active":"2024-03-14T12:13:51.572900+0000","last_became_peered":"2024-03-14T12:13:51.572900+0000","last_unstale":"2024-03-14T12:13:54.478072+0000","last_undegraded":"2024-03-14T12:13:54.478072+0000","last_fullsized":"2024-03-14T12:13:54.478072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478221+0000","last_change":"2024-03-14T12:13:50.571689+0000","last_active":"2024-03-14T12:13:54.478221+0000","last_peered":"2024-03-14T12:13:54.478221+0000","last_clean":"2024-03-14T12:13:54.478221+0000","last_became_active":"2024-03-14T12:13:50.571292+0000","last_became_peered":"2024-03-14T12:13:50.571292+0000","last_unstale":"2024-03-14T12:13:54.478221+0000","last_undegraded":"2024-03-14T12:13:54.478221+0000","last_fullsized":"2024-03-14T12:13:54.478221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777395+0000","last_change":"2024-03-14T12:13:51.582845+0000","last_active":"2024-03-14T12:13:52.777395+0000","last_peered":"2024-03-14T12:13:52.777395+0000","last_clean":"2024-03-14T12:13:52.777395+0000","last_became_active":"2024-03-14T12:13:51.582580+0000","last_became_peered":"2024-03-14T12:13:51.582580+0000","last_unstale":"2024-03-14T12:13:52.777395+0000","last_undegraded":"2024-03-14T12:13:52.777395+0000","last_fullsized":"2024-03-14T12:13:52.777395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477686+0000","last_change":"2024-03-14T12:13:50.568041+0000","last_active":"2024-03-14T12:13:54.477686+0000","last_peered":"2024-03-14T12:13:54.477686+0000","last_clean":"2024-03-14T12:13:54.477686+0000","last_became_active":"2024-03-14T12:13:50.567698+0000","last_became_peered":"2024-03-14T12:13:50.567698+0000","last_unstale":"2024-03-14T12:13:54.477686+0000","last_undegraded":"2024-03-14T12:13:54.477686+0000","last_fullsized":"2024-03-14T12:13:54.477686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573917+0000","last_change":"2024-03-14T12:13:51.572247+0000","last_active":"2024-03-14T12:13:52.573917+0000","last_peered":"2024-03-14T12:13:52.573917+0000","last_clean":"2024-03-14T12:13:52.573917+0000","last_became_active":"2024-03-14T12:13:51.572000+0000","last_became_peered":"2024-03-14T12:13:51.572000+0000","last_unstale":"2024-03-14T12:13:52.573917+0000","last_undegraded":"2024-03-14T12:13:52.573917+0000","last_fullsized":"2024-03-14T12:13:52.573917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580774+0000","last_change":"2024-03-14T12:13:50.562695+0000","last_active":"2024-03-14T12:13:52.580774+0000","last_peered":"2024-03-14T12:13:52.580774+0000","last_clean":"2024-03-14T12:13:52.580774+0000","last_became_active":"2024-03-14T12:13:50.562419+0000","last_became_peered":"2024-03-14T12:13:50.562419+0000","last_unstale":"2024-03-14T12:13:52.580774+0000","last_undegraded":"2024-03-14T12:13:52.580774+0000","last_fullsized":"2024-03-14T12:13:52.580774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580029+0000","last_change":"2024-03-14T12:13:51.578692+0000","last_active":"2024-03-14T12:13:52.580029+0000","last_peered":"2024-03-14T12:13:52.580029+0000","last_clean":"2024-03-14T12:13:52.580029+0000","last_became_active":"2024-03-14T12:13:51.578375+0000","last_became_peered":"2024-03-14T12:13:51.578375+0000","last_unstale":"2024-03-14T12:13:52.580029+0000","last_undegraded":"2024-03-14T12:13:52.580029+0000","last_fullsized":"2024-03-14T12:13:52.580029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573848+0000","last_change":"2024-03-14T12:13:50.568583+0000","last_active":"2024-03-14T12:13:52.573848+0000","last_peered":"2024-03-14T12:13:52.573848+0000","last_clean":"2024-03-14T12:13:52.573848+0000","last_became_active":"2024-03-14T12:13:50.568267+0000","last_became_peered":"2024-03-14T12:13:50.568267+0000","last_unstale":"2024-03-14T12:13:52.573848+0000","last_undegraded":"2024-03-14T12:13:52.573848+0000","last_fullsized":"2024-03-14T12:13:52.573848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580655+0000","last_change":"2024-03-14T12:13:51.579205+0000","last_active":"2024-03-14T12:13:52.580655+0000","last_peered":"2024-03-14T12:13:52.580655+0000","last_clean":"2024-03-14T12:13:52.580655+0000","last_became_active":"2024-03-14T12:13:51.578901+0000","last_became_peered":"2024-03-14T12:13:51.578901+0000","last_unstale":"2024-03-14T12:13:52.580655+0000","last_undegraded":"2024-03-14T12:13:52.580655+0000","last_fullsized":"2024-03-14T12:13:52.580655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.479092+0000","last_change":"2024-03-14T12:13:50.571020+0000","last_active":"2024-03-14T12:13:54.479092+0000","last_peered":"2024-03-14T12:13:54.479092+0000","last_clean":"2024-03-14T12:13:54.479092+0000","last_became_active":"2024-03-14T12:13:50.570744+0000","last_became_peered":"2024-03-14T12:13:50.570744+0000","last_unstale":"2024-03-14T12:13:54.479092+0000","last_undegraded":"2024-03-14T12:13:54.479092+0000","last_fullsized":"2024-03-14T12:13:54.479092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573864+0000","last_change":"2024-03-14T12:13:51.574187+0000","last_active":"2024-03-14T12:13:52.573864+0000","last_peered":"2024-03-14T12:13:52.573864+0000","last_clean":"2024-03-14T12:13:52.573864+0000","last_became_active":"2024-03-14T12:13:51.573916+0000","last_became_peered":"2024-03-14T12:13:51.573916+0000","last_unstale":"2024-03-14T12:13:52.573864+0000","last_undegraded":"2024-03-14T12:13:52.573864+0000","last_fullsized":"2024-03-14T12:13:52.573864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580829+0000","last_change":"2024-03-14T12:13:50.569678+0000","last_active":"2024-03-14T12:13:52.580829+0000","last_peered":"2024-03-14T12:13:52.580829+0000","last_clean":"2024-03-14T12:13:52.580829+0000","last_became_active":"2024-03-14T12:13:50.569377+0000","last_became_peered":"2024-03-14T12:13:50.569377+0000","last_unstale":"2024-03-14T12:13:52.580829+0000","last_undegraded":"2024-03-14T12:13:52.580829+0000","last_fullsized":"2024-03-14T12:13:52.580829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478237+0000","last_change":"2024-03-14T12:13:51.579167+0000","last_active":"2024-03-14T12:13:54.478237+0000","last_peered":"2024-03-14T12:13:54.478237+0000","last_clean":"2024-03-14T12:13:54.478237+0000","last_became_active":"2024-03-14T12:13:51.578971+0000","last_became_peered":"2024-03-14T12:13:51.578971+0000","last_unstale":"2024-03-14T12:13:54.478237+0000","last_undegraded":"2024-03-14T12:13:54.478237+0000","last_fullsized":"2024-03-14T12:13:54.478237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582080+0000","last_change":"2024-03-14T12:13:50.570044+0000","last_active":"2024-03-14T12:13:52.582080+0000","last_peered":"2024-03-14T12:13:52.582080+0000","last_clean":"2024-03-14T12:13:52.582080+0000","last_became_active":"2024-03-14T12:13:50.569759+0000","last_became_peered":"2024-03-14T12:13:50.569759+0000","last_unstale":"2024-03-14T12:13:52.582080+0000","last_undegraded":"2024-03-14T12:13:52.582080+0000","last_fullsized":"2024-03-14T12:13:52.582080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777595+0000","last_change":"2024-03-14T12:13:51.581169+0000","last_active":"2024-03-14T12:13:52.777595+0000","last_peered":"2024-03-14T12:13:52.777595+0000","last_clean":"2024-03-14T12:13:52.777595+0000","last_became_active":"2024-03-14T12:13:51.580757+0000","last_became_peered":"2024-03-14T12:13:51.580757+0000","last_unstale":"2024-03-14T12:13:52.777595+0000","last_undegraded":"2024-03-14T12:13:52.777595+0000","last_fullsized":"2024-03-14T12:13:52.777595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478220+0000","last_change":"2024-03-14T12:13:50.571439+0000","last_active":"2024-03-14T12:13:54.478220+0000","last_peered":"2024-03-14T12:13:54.478220+0000","last_clean":"2024-03-14T12:13:54.478220+0000","last_became_active":"2024-03-14T12:13:50.571184+0000","last_became_peered":"2024-03-14T12:13:50.571184+0000","last_unstale":"2024-03-14T12:13:54.478220+0000","last_undegraded":"2024-03-14T12:13:54.478220+0000","last_fullsized":"2024-03-14T12:13:54.478220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579859+0000","last_change":"2024-03-14T12:13:51.573261+0000","last_active":"2024-03-14T12:13:52.579859+0000","last_peered":"2024-03-14T12:13:52.579859+0000","last_clean":"2024-03-14T12:13:52.579859+0000","last_became_active":"2024-03-14T12:13:51.572866+0000","last_became_peered":"2024-03-14T12:13:51.572866+0000","last_unstale":"2024-03-14T12:13:52.579859+0000","last_undegraded":"2024-03-14T12:13:52.579859+0000","last_fullsized":"2024-03-14T12:13:52.579859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:39.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573916+0000","last_change":"2024-03-14T12:13:50.570820+0000","last_active":"2024-03-14T12:13:52.573916+0000","last_peered":"2024-03-14T12:13:52.573916+0000","last_clean":"2024-03-14T12:13:52.573916+0000","last_became_active":"2024-03-14T12:13:50.570423+0000","last_became_peered":"2024-03-14T12:13:50.570423+0000","last_unstale":"2024-03-14T12:13:52.573916+0000","last_undegraded":"2024-03-14T12:13:52.573916+0000","last_fullsized":"2024-03-14T12:13:52.573916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477998+0000","last_change":"2024-03-14T12:13:51.575973+0000","last_active":"2024-03-14T12:13:54.477998+0000","last_peered":"2024-03-14T12:13:54.477998+0000","last_clean":"2024-03-14T12:13:54.477998+0000","last_became_active":"2024-03-14T12:13:51.575695+0000","last_became_peered":"2024-03-14T12:13:51.575695+0000","last_unstale":"2024-03-14T12:13:54.477998+0000","last_undegraded":"2024-03-14T12:13:54.477998+0000","last_fullsized":"2024-03-14T12:13:54.477998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573877+0000","last_change":"2024-03-14T12:13:50.565398+0000","last_active":"2024-03-14T12:13:52.573877+0000","last_peered":"2024-03-14T12:13:52.573877+0000","last_clean":"2024-03-14T12:13:52.573877+0000","last_became_active":"2024-03-14T12:13:50.565109+0000","last_became_peered":"2024-03-14T12:13:50.565109+0000","last_unstale":"2024-03-14T12:13:52.573877+0000","last_undegraded":"2024-03-14T12:13:52.573877+0000","last_fullsized":"2024-03-14T12:13:52.573877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478222+0000","last_change":"2024-03-14T12:13:51.576916+0000","last_active":"2024-03-14T12:13:54.478222+0000","last_peered":"2024-03-14T12:13:54.478222+0000","last_clean":"2024-03-14T12:13:54.478222+0000","last_became_active":"2024-03-14T12:13:51.576639+0000","last_became_peered":"2024-03-14T12:13:51.576639+0000","last_unstale":"2024-03-14T12:13:54.478222+0000","last_undegraded":"2024-03-14T12:13:54.478222+0000","last_fullsized":"2024-03-14T12:13:54.478222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478704+0000","last_change":"2024-03-14T12:13:50.575585+0000","last_active":"2024-03-14T12:13:54.478704+0000","last_peered":"2024-03-14T12:13:54.478704+0000","last_clean":"2024-03-14T12:13:54.478704+0000","last_became_active":"2024-03-14T12:13:50.575288+0000","last_became_peered":"2024-03-14T12:13:50.575288+0000","last_unstale":"2024-03-14T12:13:54.478704+0000","last_undegraded":"2024-03-14T12:13:54.478704+0000","last_fullsized":"2024-03-14T12:13:54.478704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581882+0000","last_change":"2024-03-14T12:13:51.588022+0000","last_active":"2024-03-14T12:13:52.581882+0000","last_peered":"2024-03-14T12:13:52.581882+0000","last_clean":"2024-03-14T12:13:52.581882+0000","last_became_active":"2024-03-14T12:13:51.587766+0000","last_became_peered":"2024-03-14T12:13:51.587766+0000","last_unstale":"2024-03-14T12:13:52.581882+0000","last_undegraded":"2024-03-14T12:13:52.581882+0000","last_fullsized":"2024-03-14T12:13:52.581882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580894+0000","last_change":"2024-03-14T12:13:51.572504+0000","last_active":"2024-03-14T12:13:52.580894+0000","last_peered":"2024-03-14T12:13:52.580894+0000","last_clean":"2024-03-14T12:13:52.580894+0000","last_became_active":"2024-03-14T12:13:51.572219+0000","last_became_peered":"2024-03-14T12:13:51.572219+0000","last_unstale":"2024-03-14T12:13:52.580894+0000","last_undegraded":"2024-03-14T12:13:52.580894+0000","last_fullsized":"2024-03-14T12:13:52.580894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478474+0000","last_change":"2024-03-14T12:13:50.572648+0000","last_active":"2024-03-14T12:13:54.478474+0000","last_peered":"2024-03-14T12:13:54.478474+0000","last_clean":"2024-03-14T12:13:54.478474+0000","last_became_active":"2024-03-14T12:13:50.572409+0000","last_became_peered":"2024-03-14T12:13:50.572409+0000","last_unstale":"2024-03-14T12:13:54.478474+0000","last_undegraded":"2024-03-14T12:13:54.478474+0000","last_fullsized":"2024-03-14T12:13:54.478474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579831+0000","last_change":"2024-03-14T12:13:51.580823+0000","last_active":"2024-03-14T12:13:52.579831+0000","last_peered":"2024-03-14T12:13:52.579831+0000","last_clean":"2024-03-14T12:13:52.579831+0000","last_became_active":"2024-03-14T12:13:51.574625+0000","last_became_peered":"2024-03-14T12:13:51.574625+0000","last_unstale":"2024-03-14T12:13:52.579831+0000","last_undegraded":"2024-03-14T12:13:52.579831+0000","last_fullsized":"2024-03-14T12:13:52.579831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478728+0000","last_change":"2024-03-14T12:13:50.572321+0000","last_active":"2024-03-14T12:13:54.478728+0000","last_peered":"2024-03-14T12:13:54.478728+0000","last_clean":"2024-03-14T12:13:54.478728+0000","last_became_active":"2024-03-14T12:13:50.572001+0000","last_became_peered":"2024-03-14T12:13:50.572001+0000","last_unstale":"2024-03-14T12:13:54.478728+0000","last_undegraded":"2024-03-14T12:13:54.478728+0000","last_fullsized":"2024-03-14T12:13:54.478728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777630+0000","last_change":"2024-03-14T12:13:51.575693+0000","last_active":"2024-03-14T12:13:52.777630+0000","last_peered":"2024-03-14T12:13:52.777630+0000","last_clean":"2024-03-14T12:13:52.777630+0000","last_became_active":"2024-03-14T12:13:51.574493+0000","last_became_peered":"2024-03-14T12:13:51.574493+0000","last_unstale":"2024-03-14T12:13:52.777630+0000","last_undegraded":"2024-03-14T12:13:52.777630+0000","last_fullsized":"2024-03-14T12:13:52.777630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478473+0000","last_change":"2024-03-14T12:13:50.565913+0000","last_active":"2024-03-14T12:13:54.478473+0000","last_peered":"2024-03-14T12:13:54.478473+0000","last_clean":"2024-03-14T12:13:54.478473+0000","last_became_active":"2024-03-14T12:13:50.565652+0000","last_became_peered":"2024-03-14T12:13:50.565652+0000","last_unstale":"2024-03-14T12:13:54.478473+0000","last_undegraded":"2024-03-14T12:13:54.478473+0000","last_fullsized":"2024-03-14T12:13:54.478473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478392+0000","last_change":"2024-03-14T12:13:51.581638+0000","last_active":"2024-03-14T12:13:54.478392+0000","last_peered":"2024-03-14T12:13:54.478392+0000","last_clean":"2024-03-14T12:13:54.478392+0000","last_became_active":"2024-03-14T12:13:51.581373+0000","last_became_peered":"2024-03-14T12:13:51.581373+0000","last_unstale":"2024-03-14T12:13:54.478392+0000","last_undegraded":"2024-03-14T12:13:54.478392+0000","last_fullsized":"2024-03-14T12:13:54.478392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777698+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:52.777698+0000","last_peered":"2024-03-14T12:13:52.777698+0000","last_clean":"2024-03-14T12:13:52.777698+0000","last_became_active":"2024-03-14T12:13:50.569816+0000","last_became_peered":"2024-03-14T12:13:50.569816+0000","last_unstale":"2024-03-14T12:13:52.777698+0000","last_undegraded":"2024-03-14T12:13:52.777698+0000","last_fullsized":"2024-03-14T12:13:52.777698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:14:02.103 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:14:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.240+0000 7f581c9d4640 1 Processor -- start 2024-03-14T12:14:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.240+0000 7f581c9d4640 1 -- start start 2024-03-14T12:14:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581c9d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58180af150 0x7f58180af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581c9d4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58180b01d0 con 0x7f581806b420 2024-03-14T12:14:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581c9d4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581815e710 con 0x7f581815f1f0 2024-03-14T12:14:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581c9d4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581815e9f0 con 0x7f58180af150 2024-03-14T12:14:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58180af150 0x7f58180af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58180af150 0x7f58180af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51172/0 (socket says 172.21.15.82:51172) 2024-03-14T12:14:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581659c640 1 -- 172.21.15.82:0/4015428216 learned_addr learned my addr 172.21.15.82:0/4015428216 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581659c640 1 -- 172.21.15.82:0/4015428216 >> v1:172.21.15.164:6789/0 conn(0x7f581815f1f0 legacy=0x7f581806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581659c640 1 -- 172.21.15.82:0/4015428216 >> v1:172.21.15.82:6789/0 conn(0x7f581806b420 legacy=0x7f5818157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581659c640 1 -- 172.21.15.82:0/4015428216 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581815ecd0 con 0x7f58180af150 2024-03-14T12:14:02.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581659c640 1 --2- 172.21.15.82:0/4015428216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58180af880 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5808009e50 tx=0x7f580802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77aa5602c1edb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581559a640 1 -- 172.21.15.82:0/4015428216 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580802ca80 con 0x7f58180af150 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581559a640 1 -- 172.21.15.82:0/4015428216 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580802cc20 con 0x7f58180af150 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.242+0000 7f581559a640 1 -- 172.21.15.82:0/4015428216 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080343f0 con 0x7f58180af150 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.243+0000 7f581c9d4640 1 -- 172.21.15.82:0/4015428216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 msgr2=0x7f58180af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.243+0000 7f581c9d4640 1 --2- 172.21.15.82:0/4015428216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58180af880 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5808009e50 tx=0x7f580802bee0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.243+0000 7f581c9d4640 1 -- 172.21.15.82:0/4015428216 shutdown_connections 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.243+0000 7f581c9d4640 1 --2- 172.21.15.82:0/4015428216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58180af880 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.243+0000 7f581c9d4640 1 -- 172.21.15.82:0/4015428216 >> 172.21.15.82:0/4015428216 conn(0x7f58180d0a20 msgr2=0x7f58180d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.243+0000 7f581c9d4640 1 -- 172.21.15.82:0/4015428216 shutdown_connections 2024-03-14T12:14:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.243+0000 7f581c9d4640 1 -- 172.21.15.82:0/4015428216 wait complete. 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.244+0000 7f581c9d4640 1 Processor -- start 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.244+0000 7f581c9d4640 1 -- start start 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581c9d4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f581806b420 0x7f581815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581c9d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58181f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581c9d4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581815f1f0 0x7f58181f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581c9d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180aff40 con 0x7f58180af150 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581c9d4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180afdc0 con 0x7f581815f1f0 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581c9d4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58180b58f0 con 0x7f581806b420 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58181f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58181f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51180/0 (socket says 172.21.15.82:51180) 2024-03-14T12:14:02.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 -- 172.21.15.82:0/3510902784 learned_addr learned my addr 172.21.15.82:0/3510902784 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581659c640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f581806b420 0x7f581815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5816d9d640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581815f1f0 0x7f58181f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 -- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f581806b420 msgr2=0x7f581815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f581806b420 0x7f581815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 -- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581815f1f0 msgr2=0x7f58181f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581815f1f0 0x7f58181f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 -- 172.21.15.82:0/3510902784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818158d70 con 0x7f58180af150 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f581659c640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f581806b420 0x7f581815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5816d9d640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581815f1f0 0x7f58181f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.245+0000 7f5815d9b640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58181f5770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b3f0 tx=0x7f580c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.246+0000 7f5806ffd640 1 -- 172.21.15.82:0/3510902784 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c015020 con 0x7f58180af150 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.246+0000 7f5806ffd640 1 -- 172.21.15.82:0/3510902784 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c011830 con 0x7f58180af150 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.246+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58181584d0 con 0x7f58180af150 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.246+0000 7f5806ffd640 1 -- 172.21.15.82:0/3510902784 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c011c20 con 0x7f58180af150 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.246+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5818158a90 con 0x7f58180af150 2024-03-14T12:14:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.247+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58180b2550 con 0x7f58180af150 2024-03-14T12:14:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.247+0000 7f5806ffd640 1 -- 172.21.15.82:0/3510902784 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f580c018020 con 0x7f58180af150 2024-03-14T12:14:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.247+0000 7f5806ffd640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57f806bbd0 0x7f57f806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.248+0000 7f581659c640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57f806bbd0 0x7f57f806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.248+0000 7f5806ffd640 1 -- 172.21.15.82:0/3510902784 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f580c08a690 con 0x7f58180af150 2024-03-14T12:14:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.248+0000 7f581659c640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57f806bbd0 0x7f57f806e090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5808035000 tx=0x7f5808030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.253+0000 7f5806ffd640 1 -- 172.21.15.82:0/3510902784 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f58180b2550 con 0x7f58180af150 2024-03-14T12:14:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.418+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58180b44e0 con 0x7f57f806bbd0 2024-03-14T12:14:02.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.424+0000 7f5806ffd640 1 -- 172.21.15.82:0/3510902784 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f58180b44e0 con 0x7f57f806bbd0 2024-03-14T12:14:02.426 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:14:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57f806bbd0 msgr2=0x7f57f806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57f806bbd0 0x7f57f806e090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5808035000 tx=0x7f5808030000 comp rx=0 tx=0).stop 2024-03-14T12:14:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 msgr2=0x7f58181f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58181f5770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b3f0 tx=0x7f580c00b8b0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 shutdown_connections 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581815f1f0 0x7f58181f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58180af150 0x7f58181f5770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57f806bbd0 0x7f57f806e090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 --2- 172.21.15.82:0/3510902784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f581806b420 0x7f581815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 >> 172.21.15.82:0/3510902784 conn(0x7f58180d0a20 msgr2=0x7f58180716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 shutdown_connections 2024-03-14T12:14:02.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.427+0000 7f581c9d4640 1 -- 172.21.15.82:0/3510902784 wait complete. 2024-03-14T12:14:02.438 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-03-14T12:14:01.500062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.938828"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478859+0000","last_change":"2024-03-14T12:13:51.582275+0000","last_active":"2024-03-14T12:13:54.478859+0000","last_peered":"2024-03-14T12:13:54.478859+0000","last_clean":"2024-03-14T12:13:54.478859+0000","last_became_active":"2024-03-14T12:13:51.581999+0000","last_became_peered":"2024-03-14T12:13:51.581999+0000","last_unstale":"2024-03-14T12:13:54.478859+0000","last_undegraded":"2024-03-14T12:13:54.478859+0000","last_fullsized":"2024-03-14T12:13:54.478859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574007+0000","last_change":"2024-03-14T12:13:51.575382+0000","last_active":"2024-03-14T12:13:52.574007+0000","last_peered":"2024-03-14T12:13:52.574007+0000","last_clean":"2024-03-14T12:13:52.574007+0000","last_became_active":"2024-03-14T12:13:51.575139+0000","last_became_peered":"2024-03-14T12:13:51.575139+0000","last_unstale":"2024-03-14T12:13:52.574007+0000","last_undegraded":"2024-03-14T12:13:52.574007+0000","last_fullsized":"2024-03-14T12:13:52.574007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777473+0000","last_change":"2024-03-14T12:13:51.574924+0000","last_active":"2024-03-14T12:13:52.777473+0000","last_peered":"2024-03-14T12:13:52.777473+0000","last_clean":"2024-03-14T12:13:52.777473+0000","last_became_active":"2024-03-14T12:13:51.574299+0000","last_became_peered":"2024-03-14T12:13:51.574299+0000","last_unstale":"2024-03-14T12:13:52.777473+0000","last_undegraded":"2024-03-14T12:13:52.777473+0000","last_fullsized":"2024-03-14T12:13:52.777473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579676+0000","last_change":"2024-03-14T12:13:51.581446+0000","last_active":"2024-03-14T12:13:52.579676+0000","last_peered":"2024-03-14T12:13:52.579676+0000","last_clean":"2024-03-14T12:13:52.579676+0000","last_became_active":"2024-03-14T12:13:51.581192+0000","last_became_peered":"2024-03-14T12:13:51.581192+0000","last_unstale":"2024-03-14T12:13:52.579676+0000","last_undegraded":"2024-03-14T12:13:52.579676+0000","last_fullsized":"2024-03-14T12:13:52.579676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478459+0000","last_change":"2024-03-14T12:13:51.587466+0000","last_active":"2024-03-14T12:13:54.478459+0000","last_peered":"2024-03-14T12:13:54.478459+0000","last_clean":"2024-03-14T12:13:54.478459+0000","last_became_active":"2024-03-14T12:13:51.585375+0000","last_became_peered":"2024-03-14T12:13:51.585375+0000","last_unstale":"2024-03-14T12:13:54.478459+0000","last_undegraded":"2024-03-14T12:13:54.478459+0000","last_fullsized":"2024-03-14T12:13:54.478459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580498+0000","last_change":"2024-03-14T12:13:51.580794+0000","last_active":"2024-03-14T12:13:52.580498+0000","last_peered":"2024-03-14T12:13:52.580498+0000","last_clean":"2024-03-14T12:13:52.580498+0000","last_became_active":"2024-03-14T12:13:51.580511+0000","last_became_peered":"2024-03-14T12:13:51.580511+0000","last_unstale":"2024-03-14T12:13:52.580498+0000","last_undegraded":"2024-03-14T12:13:52.580498+0000","last_fullsized":"2024-03-14T12:13:52.580498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478351+0000","last_change":"2024-03-14T12:13:51.588949+0000","last_active":"2024-03-14T12:13:54.478351+0000","last_peered":"2024-03-14T12:13:54.478351+0000","last_clean":"2024-03-14T12:13:54.478351+0000","last_became_active":"2024-03-14T12:13:51.588671+0000","last_became_peered":"2024-03-14T12:13:51.588671+0000","last_unstale":"2024-03-14T12:13:54.478351+0000","last_undegraded":"2024-03-14T12:13:54.478351+0000","last_fullsized":"2024-03-14T12:13:54.478351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477849+0000","last_change":"2024-03-14T12:13:51.579815+0000","last_active":"2024-03-14T12:13:54.477849+0000","last_peered":"2024-03-14T12:13:54.477849+0000","last_clean":"2024-03-14T12:13:54.477849+0000","last_became_active":"2024-03-14T12:13:51.579580+0000","last_became_peered":"2024-03-14T12:13:51.579580+0000","last_unstale":"2024-03-14T12:13:54.477849+0000","last_undegraded":"2024-03-14T12:13:54.477849+0000","last_fullsized":"2024-03-14T12:13:54.477849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478866+0000","last_change":"2024-03-14T12:13:51.593442+0000","last_active":"2024-03-14T12:13:54.478866+0000","last_peered":"2024-03-14T12:13:54.478866+0000","last_clean":"2024-03-14T12:13:54.478866+0000","last_became_active":"2024-03-14T12:13:51.593151+0000","last_became_peered":"2024-03-14T12:13:51.593151+0000","last_unstale":"2024-03-14T12:13:54.478866+0000","last_undegraded":"2024-03-14T12:13:54.478866+0000","last_fullsized":"2024-03-14T12:13:54.478866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582259+0000","last_change":"2024-03-14T12:13:51.584838+0000","last_active":"2024-03-14T12:13:52.582259+0000","last_peered":"2024-03-14T12:13:52.582259+0000","last_clean":"2024-03-14T12:13:52.582259+0000","last_became_active":"2024-03-14T12:13:51.584501+0000","last_became_peered":"2024-03-14T12:13:51.584501+0000","last_unstale":"2024-03-14T12:13:52.582259+0000","last_undegraded":"2024-03-14T12:13:52.582259+0000","last_fullsized":"2024-03-14T12:13:52.582259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581960+0000","last_change":"2024-03-14T12:13:51.584374+0000","last_active":"2024-03-14T12:13:52.581960+0000","last_peered":"2024-03-14T12:13:52.581960+0000","last_clean":"2024-03-14T12:13:52.581960+0000","last_became_active":"2024-03-14T12:13:51.584094+0000","last_became_peered":"2024-03-14T12:13:51.584094+0000","last_unstale":"2024-03-14T12:13:52.581960+0000","last_undegraded":"2024-03-14T12:13:52.581960+0000","last_fullsized":"2024-03-14T12:13:52.581960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580082+0000","last_change":"2024-03-14T12:13:51.583802+0000","last_active":"2024-03-14T12:13:52.580082+0000","last_peered":"2024-03-14T12:13:52.580082+0000","last_clean":"2024-03-14T12:13:52.580082+0000","last_became_active":"2024-03-14T12:13:51.583284+0000","last_became_peered":"2024-03-14T12:13:51.583284+0000","last_unstale":"2024-03-14T12:13:52.580082+0000","last_undegraded":"2024-03-14T12:13:52.580082+0000","last_fullsized":"2024-03-14T12:13:52.580082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478086+0000","last_change":"2024-03-14T12:13:51.578671+0000","last_active":"2024-03-14T12:13:54.478086+0000","last_peered":"2024-03-14T12:13:54.478086+0000","last_clean":"2024-03-14T12:13:54.478086+0000","last_became_active":"2024-03-14T12:13:51.578427+0000","last_became_peered":"2024-03-14T12:13:51.578427+0000","last_unstale":"2024-03-14T12:13:54.478086+0000","last_undegraded":"2024-03-14T12:13:54.478086+0000","last_fullsized":"2024-03-14T12:13:54.478086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777400+0000","last_change":"2024-03-14T12:13:51.581289+0000","last_active":"2024-03-14T12:13:52.777400+0000","last_peered":"2024-03-14T12:13:52.777400+0000","last_clean":"2024-03-14T12:13:52.777400+0000","last_became_active":"2024-03-14T12:13:51.580645+0000","last_became_peered":"2024-03-14T12:13:51.580645+0000","last_unstale":"2024-03-14T12:13:52.777400+0000","last_undegraded":"2024-03-14T12:13:52.777400+0000","last_fullsized":"2024-03-14T12:13:52.777400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:01.290201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573796+0000","last_change":"2024-03-14T12:13:51.571032+0000","last_active":"2024-03-14T12:13:52.573796+0000","last_peered":"2024-03-14T12:13:52.573796+0000","last_clean":"2024-03-14T12:13:52.573796+0000","last_became_active":"2024-03-14T12:13:51.570723+0000","last_became_peered":"2024-03-14T12:13:51.570723+0000","last_unstale":"2024-03-14T12:13:52.573796+0000","last_undegraded":"2024-03-14T12:13:52.573796+0000","last_fullsized":"2024-03-14T12:13:52.573796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580831+0000","last_change":"2024-03-14T12:13:51.582495+0000","last_active":"2024-03-14T12:13:52.580831+0000","last_peered":"2024-03-14T12:13:52.580831+0000","last_clean":"2024-03-14T12:13:52.580831+0000","last_became_active":"2024-03-14T12:13:51.582170+0000","last_became_peered":"2024-03-14T12:13:51.582170+0000","last_unstale":"2024-03-14T12:13:52.580831+0000","last_undegraded":"2024-03-14T12:13:52.580831+0000","last_fullsized":"2024-03-14T12:13:52.580831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:59.497532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580861+0000","last_change":"2024-03-14T12:13:51.582330+0000","last_active":"2024-03-14T12:13:52.580861+0000","last_peered":"2024-03-14T12:13:52.580861+0000","last_clean":"2024-03-14T12:13:52.580861+0000","last_became_active":"2024-03-14T12:13:51.581920+0000","last_became_peered":"2024-03-14T12:13:51.581920+0000","last_unstale":"2024-03-14T12:13:52.580861+0000","last_undegraded":"2024-03-14T12:13:52.580861+0000","last_fullsized":"2024-03-14T12:13:52.580861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:18.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477706+0000","last_change":"2024-03-14T12:13:51.582708+0000","last_active":"2024-03-14T12:13:54.477706+0000","last_peered":"2024-03-14T12:13:54.477706+0000","last_clean":"2024-03-14T12:13:54.477706+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:54.477706+0000","last_undegraded":"2024-03-14T12:13:54.477706+0000","last_fullsized":"2024-03-14T12:13:54.477706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:36.940630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.479134+0000","last_change":"2024-03-14T12:13:51.587758+0000","last_active":"2024-03-14T12:13:54.479134+0000","last_peered":"2024-03-14T12:13:54.479134+0000","last_clean":"2024-03-14T12:13:54.479134+0000","last_became_active":"2024-03-14T12:13:51.586310+0000","last_became_peered":"2024-03-14T12:13:51.586310+0000","last_unstale":"2024-03-14T12:13:54.479134+0000","last_undegraded":"2024-03-14T12:13:54.479134+0000","last_fullsized":"2024-03-14T12:13:54.479134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:57:08.624764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582057+0000","last_change":"2024-03-14T12:13:51.589443+0000","last_active":"2024-03-14T12:13:52.582057+0000","last_peered":"2024-03-14T12:13:52.582057+0000","last_clean":"2024-03-14T12:13:52.582057+0000","last_became_active":"2024-03-14T12:13:51.589174+0000","last_became_peered":"2024-03-14T12:13:51.589174+0000","last_unstale":"2024-03-14T12:13:52.582057+0000","last_undegraded":"2024-03-14T12:13:52.582057+0000","last_fullsized":"2024-03-14T12:13:52.582057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777419+0000","last_change":"2024-03-14T12:13:51.580813+0000","last_active":"2024-03-14T12:13:52.777419+0000","last_peered":"2024-03-14T12:13:52.777419+0000","last_clean":"2024-03-14T12:13:52.777419+0000","last_became_active":"2024-03-14T12:13:51.580334+0000","last_became_peered":"2024-03-14T12:13:51.580334+0000","last_unstale":"2024-03-14T12:13:52.777419+0000","last_undegraded":"2024-03-14T12:13:52.777419+0000","last_fullsized":"2024-03-14T12:13:52.777419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580443+0000","last_change":"2024-03-14T12:13:51.583228+0000","last_active":"2024-03-14T12:13:52.580443+0000","last_peered":"2024-03-14T12:13:52.580443+0000","last_clean":"2024-03-14T12:13:52.580443+0000","last_became_active":"2024-03-14T12:13:51.582937+0000","last_became_peered":"2024-03-14T12:13:51.582937+0000","last_unstale":"2024-03-14T12:13:52.580443+0000","last_undegraded":"2024-03-14T12:13:52.580443+0000","last_fullsized":"2024-03-14T12:13:52.580443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477840+0000","last_change":"2024-03-14T12:13:51.582656+0000","last_active":"2024-03-14T12:13:54.477840+0000","last_peered":"2024-03-14T12:13:54.477840+0000","last_clean":"2024-03-14T12:13:54.477840+0000","last_became_active":"2024-03-14T12:13:51.582401+0000","last_became_peered":"2024-03-14T12:13:51.582401+0000","last_unstale":"2024-03-14T12:13:54.477840+0000","last_undegraded":"2024-03-14T12:13:54.477840+0000","last_fullsized":"2024-03-14T12:13:54.477840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580802+0000","last_change":"2024-03-14T12:13:51.579810+0000","last_active":"2024-03-14T12:13:52.580802+0000","last_peered":"2024-03-14T12:13:52.580802+0000","last_clean":"2024-03-14T12:13:52.580802+0000","last_became_active":"2024-03-14T12:13:51.579523+0000","last_became_peered":"2024-03-14T12:13:51.579523+0000","last_unstale":"2024-03-14T12:13:52.580802+0000","last_undegraded":"2024-03-14T12:13:52.580802+0000","last_fullsized":"2024-03-14T12:13:52.580802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478115+0000","last_change":"2024-03-14T12:13:51.582442+0000","last_active":"2024-03-14T12:13:54.478115+0000","last_peered":"2024-03-14T12:13:54.478115+0000","last_clean":"2024-03-14T12:13:54.478115+0000","last_became_active":"2024-03-14T12:13:51.582175+0000","last_became_peered":"2024-03-14T12:13:51.582175+0000","last_unstale":"2024-03-14T12:13:54.478115+0000","last_undegraded":"2024-03-14T12:13:54.478115+0000","last_fullsized":"2024-03-14T12:13:54.478115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582355+0000","last_change":"2024-03-14T12:13:51.590516+0000","last_active":"2024-03-14T12:13:52.582355+0000","last_peered":"2024-03-14T12:13:52.582355+0000","last_clean":"2024-03-14T12:13:52.582355+0000","last_became_active":"2024-03-14T12:13:51.590223+0000","last_became_peered":"2024-03-14T12:13:51.590223+0000","last_unstale":"2024-03-14T12:13:52.582355+0000","last_undegraded":"2024-03-14T12:13:52.582355+0000","last_fullsized":"2024-03-14T12:13:52.582355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573995+0000","last_change":"2024-03-14T12:13:51.574721+0000","last_active":"2024-03-14T12:13:52.573995+0000","last_peered":"2024-03-14T12:13:52.573995+0000","last_clean":"2024-03-14T12:13:52.573995+0000","last_became_active":"2024-03-14T12:13:51.574443+0000","last_became_peered":"2024-03-14T12:13:51.574443+0000","last_unstale":"2024-03-14T12:13:52.573995+0000","last_undegraded":"2024-03-14T12:13:52.573995+0000","last_fullsized":"2024-03-14T12:13:52.573995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478558+0000","last_change":"2024-03-14T12:13:51.582640+0000","last_active":"2024-03-14T12:13:54.478558+0000","last_peered":"2024-03-14T12:13:54.478558+0000","last_clean":"2024-03-14T12:13:54.478558+0000","last_became_active":"2024-03-14T12:13:51.582355+0000","last_became_peered":"2024-03-14T12:13:51.582355+0000","last_unstale":"2024-03-14T12:13:54.478558+0000","last_undegraded":"2024-03-14T12:13:54.478558+0000","last_fullsized":"2024-03-14T12:13:54.478558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777477+0000","last_change":"2024-03-14T12:13:51.577794+0000","last_active":"2024-03-14T12:13:52.777477+0000","last_peered":"2024-03-14T12:13:52.777477+0000","last_clean":"2024-03-14T12:13:52.777477+0000","last_became_active":"2024-03-14T12:13:51.577518+0000","last_became_peered":"2024-03-14T12:13:51.577518+0000","last_unstale":"2024-03-14T12:13:52.777477+0000","last_undegraded":"2024-03-14T12:13:52.777477+0000","last_fullsized":"2024-03-14T12:13:52.777477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580629+0000","last_change":"2024-03-14T12:13:51.580193+0000","last_active":"2024-03-14T12:13:52.580629+0000","last_peered":"2024-03-14T12:13:52.580629+0000","last_clean":"2024-03-14T12:13:52.580629+0000","last_became_active":"2024-03-14T12:13:51.577667+0000","last_became_peered":"2024-03-14T12:13:51.577667+0000","last_unstale":"2024-03-14T12:13:52.580629+0000","last_undegraded":"2024-03-14T12:13:52.580629+0000","last_fullsized":"2024-03-14T12:13:52.580629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477630+0000","last_change":"2024-03-14T12:13:51.579889+0000","last_active":"2024-03-14T12:13:54.477630+0000","last_peered":"2024-03-14T12:13:54.477630+0000","last_clean":"2024-03-14T12:13:54.477630+0000","last_became_active":"2024-03-14T12:13:51.579585+0000","last_became_peered":"2024-03-14T12:13:51.579585+0000","last_unstale":"2024-03-14T12:13:54.477630+0000","last_undegraded":"2024-03-14T12:13:54.477630+0000","last_fullsized":"2024-03-14T12:13:54.477630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581817+0000","last_change":"2024-03-14T12:13:51.577340+0000","last_active":"2024-03-14T12:13:52.581817+0000","last_peered":"2024-03-14T12:13:52.581817+0000","last_clean":"2024-03-14T12:13:52.581817+0000","last_became_active":"2024-03-14T12:13:51.577080+0000","last_became_peered":"2024-03-14T12:13:51.577080+0000","last_unstale":"2024-03-14T12:13:52.581817+0000","last_undegraded":"2024-03-14T12:13:52.581817+0000","last_fullsized":"2024-03-14T12:13:52.581817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:26:52.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579599+0000","last_change":"2024-03-14T12:13:51.573447+0000","last_active":"2024-03-14T12:13:52.579599+0000","last_peered":"2024-03-14T12:13:52.579599+0000","last_clean":"2024-03-14T12:13:52.579599+0000","last_became_active":"2024-03-14T12:13:51.573137+0000","last_became_peered":"2024-03-14T12:13:51.573137+0000","last_unstale":"2024-03-14T12:13:52.579599+0000","last_undegraded":"2024-03-14T12:13:52.579599+0000","last_fullsized":"2024-03-14T12:13:52.579599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:02.273975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478302+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:54.478302+0000","last_peered":"2024-03-14T12:13:54.478302+0000","last_clean":"2024-03-14T12:13:54.478302+0000","last_became_active":"2024-03-14T12:13:50.569748+0000","last_became_peered":"2024-03-14T12:13:50.569748+0000","last_unstale":"2024-03-14T12:13:54.478302+0000","last_undegraded":"2024-03-14T12:13:54.478302+0000","last_fullsized":"2024-03-14T12:13:54.478302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777628+0000","last_change":"2024-03-14T12:13:51.581108+0000","last_active":"2024-03-14T12:13:52.777628+0000","last_peered":"2024-03-14T12:13:52.777628+0000","last_clean":"2024-03-14T12:13:52.777628+0000","last_became_active":"2024-03-14T12:13:51.580829+0000","last_became_peered":"2024-03-14T12:13:51.580829+0000","last_unstale":"2024-03-14T12:13:52.777628+0000","last_undegraded":"2024-03-14T12:13:52.777628+0000","last_fullsized":"2024-03-14T12:13:52.777628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581951+0000","last_change":"2024-03-14T12:13:50.565826+0000","last_active":"2024-03-14T12:13:52.581951+0000","last_peered":"2024-03-14T12:13:52.581951+0000","last_clean":"2024-03-14T12:13:52.581951+0000","last_became_active":"2024-03-14T12:13:50.565553+0000","last_became_peered":"2024-03-14T12:13:50.565553+0000","last_unstale":"2024-03-14T12:13:52.581951+0000","last_undegraded":"2024-03-14T12:13:52.581951+0000","last_fullsized":"2024-03-14T12:13:52.581951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477885+0000","last_change":"2024-03-14T12:13:50.563462+0000","last_active":"2024-03-14T12:13:54.477885+0000","last_peered":"2024-03-14T12:13:54.477885+0000","last_clean":"2024-03-14T12:13:54.477885+0000","last_became_active":"2024-03-14T12:13:50.563198+0000","last_became_peered":"2024-03-14T12:13:50.563198+0000","last_unstale":"2024-03-14T12:13:54.477885+0000","last_undegraded":"2024-03-14T12:13:54.477885+0000","last_fullsized":"2024-03-14T12:13:54.477885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477930+0000","last_change":"2024-03-14T12:13:51.577103+0000","last_active":"2024-03-14T12:13:54.477930+0000","last_peered":"2024-03-14T12:13:54.477930+0000","last_clean":"2024-03-14T12:13:54.477930+0000","last_became_active":"2024-03-14T12:13:51.576602+0000","last_became_peered":"2024-03-14T12:13:51.576602+0000","last_unstale":"2024-03-14T12:13:54.477930+0000","last_undegraded":"2024-03-14T12:13:54.477930+0000","last_fullsized":"2024-03-14T12:13:54.477930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580881+0000","last_change":"2024-03-14T12:13:50.559389+0000","last_active":"2024-03-14T12:13:52.580881+0000","last_peered":"2024-03-14T12:13:52.580881+0000","last_clean":"2024-03-14T12:13:52.580881+0000","last_became_active":"2024-03-14T12:13:50.559100+0000","last_became_peered":"2024-03-14T12:13:50.559100+0000","last_unstale":"2024-03-14T12:13:52.580881+0000","last_undegraded":"2024-03-14T12:13:52.580881+0000","last_fullsized":"2024-03-14T12:13:52.580881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573924+0000","last_change":"2024-03-14T12:13:51.576591+0000","last_active":"2024-03-14T12:13:52.573924+0000","last_peered":"2024-03-14T12:13:52.573924+0000","last_clean":"2024-03-14T12:13:52.573924+0000","last_became_active":"2024-03-14T12:13:51.576232+0000","last_became_peered":"2024-03-14T12:13:51.576232+0000","last_unstale":"2024-03-14T12:13:52.573924+0000","last_undegraded":"2024-03-14T12:13:52.573924+0000","last_fullsized":"2024-03-14T12:13:52.573924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581963+0000","last_change":"2024-03-14T12:13:50.568392+0000","last_active":"2024-03-14T12:13:52.581963+0000","last_peered":"2024-03-14T12:13:52.581963+0000","last_clean":"2024-03-14T12:13:52.581963+0000","last_became_active":"2024-03-14T12:13:50.568127+0000","last_became_peered":"2024-03-14T12:13:50.568127+0000","last_unstale":"2024-03-14T12:13:52.581963+0000","last_undegraded":"2024-03-14T12:13:52.581963+0000","last_fullsized":"2024-03-14T12:13:52.581963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580126+0000","last_change":"2024-03-14T12:13:51.584429+0000","last_active":"2024-03-14T12:13:52.580126+0000","last_peered":"2024-03-14T12:13:52.580126+0000","last_clean":"2024-03-14T12:13:52.580126+0000","last_became_active":"2024-03-14T12:13:51.584098+0000","last_became_peered":"2024-03-14T12:13:51.584098+0000","last_unstale":"2024-03-14T12:13:52.580126+0000","last_undegraded":"2024-03-14T12:13:52.580126+0000","last_fullsized":"2024-03-14T12:13:52.580126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580241+0000","last_change":"2024-03-14T12:13:50.567672+0000","last_active":"2024-03-14T12:13:52.580241+0000","last_peered":"2024-03-14T12:13:52.580241+0000","last_clean":"2024-03-14T12:13:52.580241+0000","last_became_active":"2024-03-14T12:13:50.567366+0000","last_became_peered":"2024-03-14T12:13:50.567366+0000","last_unstale":"2024-03-14T12:13:52.580241+0000","last_undegraded":"2024-03-14T12:13:52.580241+0000","last_fullsized":"2024-03-14T12:13:52.580241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777722+0000","last_change":"2024-03-14T12:13:51.575322+0000","last_active":"2024-03-14T12:13:52.777722+0000","last_peered":"2024-03-14T12:13:52.777722+0000","last_clean":"2024-03-14T12:13:52.777722+0000","last_became_active":"2024-03-14T12:13:51.574694+0000","last_became_peered":"2024-03-14T12:13:51.574694+0000","last_unstale":"2024-03-14T12:13:52.777722+0000","last_undegraded":"2024-03-14T12:13:52.777722+0000","last_fullsized":"2024-03-14T12:13:52.777722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573967+0000","last_change":"2024-03-14T12:13:50.567057+0000","last_active":"2024-03-14T12:13:52.573967+0000","last_peered":"2024-03-14T12:13:52.573967+0000","last_clean":"2024-03-14T12:13:52.573967+0000","last_became_active":"2024-03-14T12:13:50.566800+0000","last_became_peered":"2024-03-14T12:13:50.566800+0000","last_unstale":"2024-03-14T12:13:52.573967+0000","last_undegraded":"2024-03-14T12:13:52.573967+0000","last_fullsized":"2024-03-14T12:13:52.573967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580236+0000","last_change":"2024-03-14T12:13:51.585716+0000","last_active":"2024-03-14T12:13:52.580236+0000","last_peered":"2024-03-14T12:13:52.580236+0000","last_clean":"2024-03-14T12:13:52.580236+0000","last_became_active":"2024-03-14T12:13:51.585226+0000","last_became_peered":"2024-03-14T12:13:51.585226+0000","last_unstale":"2024-03-14T12:13:52.580236+0000","last_undegraded":"2024-03-14T12:13:52.580236+0000","last_fullsized":"2024-03-14T12:13:52.580236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:26.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582041+0000","last_change":"2024-03-14T12:13:50.562666+0000","last_active":"2024-03-14T12:13:52.582041+0000","last_peered":"2024-03-14T12:13:52.582041+0000","last_clean":"2024-03-14T12:13:52.582041+0000","last_became_active":"2024-03-14T12:13:50.562367+0000","last_became_peered":"2024-03-14T12:13:50.562367+0000","last_unstale":"2024-03-14T12:13:52.582041+0000","last_undegraded":"2024-03-14T12:13:52.582041+0000","last_fullsized":"2024-03-14T12:13:52.582041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478606+0000","last_change":"2024-03-14T12:13:51.587183+0000","last_active":"2024-03-14T12:13:54.478606+0000","last_peered":"2024-03-14T12:13:54.478606+0000","last_clean":"2024-03-14T12:13:54.478606+0000","last_became_active":"2024-03-14T12:13:51.585685+0000","last_became_peered":"2024-03-14T12:13:51.585685+0000","last_unstale":"2024-03-14T12:13:54.478606+0000","last_undegraded":"2024-03-14T12:13:54.478606+0000","last_fullsized":"2024-03-14T12:13:54.478606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:27:27.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777610+0000","last_change":"2024-03-14T12:13:50.557053+0000","last_active":"2024-03-14T12:13:52.777610+0000","last_peered":"2024-03-14T12:13:52.777610+0000","last_clean":"2024-03-14T12:13:52.777610+0000","last_became_active":"2024-03-14T12:13:50.556737+0000","last_became_peered":"2024-03-14T12:13:50.556737+0000","last_unstale":"2024-03-14T12:13:52.777610+0000","last_undegraded":"2024-03-14T12:13:52.777610+0000","last_fullsized":"2024-03-14T12:13:52.777610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777653+0000","last_change":"2024-03-14T12:13:51.581665+0000","last_active":"2024-03-14T12:13:52.777653+0000","last_peered":"2024-03-14T12:13:52.777653+0000","last_clean":"2024-03-14T12:13:52.777653+0000","last_became_active":"2024-03-14T12:13:51.580980+0000","last_became_peered":"2024-03-14T12:13:51.580980+0000","last_unstale":"2024-03-14T12:13:52.777653+0000","last_undegraded":"2024-03-14T12:13:52.777653+0000","last_fullsized":"2024-03-14T12:13:52.777653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580677+0000","last_change":"2024-03-14T12:13:50.566403+0000","last_active":"2024-03-14T12:13:52.580677+0000","last_peered":"2024-03-14T12:13:52.580677+0000","last_clean":"2024-03-14T12:13:52.580677+0000","last_became_active":"2024-03-14T12:13:50.565782+0000","last_became_peered":"2024-03-14T12:13:50.565782+0000","last_unstale":"2024-03-14T12:13:52.580677+0000","last_undegraded":"2024-03-14T12:13:52.580677+0000","last_fullsized":"2024-03-14T12:13:52.580677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478689+0000","last_change":"2024-03-14T12:13:51.592512+0000","last_active":"2024-03-14T12:13:54.478689+0000","last_peered":"2024-03-14T12:13:54.478689+0000","last_clean":"2024-03-14T12:13:54.478689+0000","last_became_active":"2024-03-14T12:13:51.592273+0000","last_became_peered":"2024-03-14T12:13:51.592273+0000","last_unstale":"2024-03-14T12:13:54.478689+0000","last_undegraded":"2024-03-14T12:13:54.478689+0000","last_fullsized":"2024-03-14T12:13:54.478689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581835+0000","last_change":"2024-03-14T12:13:50.564239+0000","last_active":"2024-03-14T12:13:52.581835+0000","last_peered":"2024-03-14T12:13:52.581835+0000","last_clean":"2024-03-14T12:13:52.581835+0000","last_became_active":"2024-03-14T12:13:50.563920+0000","last_became_peered":"2024-03-14T12:13:50.563920+0000","last_unstale":"2024-03-14T12:13:52.581835+0000","last_undegraded":"2024-03-14T12:13:52.581835+0000","last_fullsized":"2024-03-14T12:13:52.581835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581784+0000","last_change":"2024-03-14T12:13:51.576682+0000","last_active":"2024-03-14T12:13:52.581784+0000","last_peered":"2024-03-14T12:13:52.581784+0000","last_clean":"2024-03-14T12:13:52.581784+0000","last_became_active":"2024-03-14T12:13:51.576420+0000","last_became_peered":"2024-03-14T12:13:51.576420+0000","last_unstale":"2024-03-14T12:13:52.581784+0000","last_undegraded":"2024-03-14T12:13:52.581784+0000","last_fullsized":"2024-03-14T12:13:52.581784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:47.181854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477551+0000","last_change":"2024-03-14T12:13:50.564352+0000","last_active":"2024-03-14T12:13:54.477551+0000","last_peered":"2024-03-14T12:13:54.477551+0000","last_clean":"2024-03-14T12:13:54.477551+0000","last_became_active":"2024-03-14T12:13:50.563979+0000","last_became_peered":"2024-03-14T12:13:50.563979+0000","last_unstale":"2024-03-14T12:13:54.477551+0000","last_undegraded":"2024-03-14T12:13:54.477551+0000","last_fullsized":"2024-03-14T12:13:54.477551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478395+0000","last_change":"2024-03-14T12:13:51.592382+0000","last_active":"2024-03-14T12:13:54.478395+0000","last_peered":"2024-03-14T12:13:54.478395+0000","last_clean":"2024-03-14T12:13:54.478395+0000","last_became_active":"2024-03-14T12:13:51.592109+0000","last_became_peered":"2024-03-14T12:13:51.592109+0000","last_unstale":"2024-03-14T12:13:54.478395+0000","last_undegraded":"2024-03-14T12:13:54.478395+0000","last_fullsized":"2024-03-14T12:13:54.478395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579929+0000","last_change":"2024-03-14T12:13:50.565493+0000","last_active":"2024-03-14T12:13:52.579929+0000","last_peered":"2024-03-14T12:13:52.579929+0000","last_clean":"2024-03-14T12:13:52.579929+0000","last_became_active":"2024-03-14T12:13:50.565247+0000","last_became_peered":"2024-03-14T12:13:50.565247+0000","last_unstale":"2024-03-14T12:13:52.579929+0000","last_undegraded":"2024-03-14T12:13:52.579929+0000","last_fullsized":"2024-03-14T12:13:52.579929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:39:10.529585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478754+0000","last_change":"2024-03-14T12:13:51.591007+0000","last_active":"2024-03-14T12:13:54.478754+0000","last_peered":"2024-03-14T12:13:54.478754+0000","last_clean":"2024-03-14T12:13:54.478754+0000","last_became_active":"2024-03-14T12:13:51.590750+0000","last_became_peered":"2024-03-14T12:13:51.590750+0000","last_unstale":"2024-03-14T12:13:54.478754+0000","last_undegraded":"2024-03-14T12:13:54.478754+0000","last_fullsized":"2024-03-14T12:13:54.478754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:06:25.678798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477792+0000","last_change":"2024-03-14T12:13:50.568050+0000","last_active":"2024-03-14T12:13:54.477792+0000","last_peered":"2024-03-14T12:13:54.477792+0000","last_clean":"2024-03-14T12:13:54.477792+0000","last_became_active":"2024-03-14T12:13:50.567713+0000","last_became_peered":"2024-03-14T12:13:50.567713+0000","last_unstale":"2024-03-14T12:13:54.477792+0000","last_undegraded":"2024-03-14T12:13:54.477792+0000","last_fullsized":"2024-03-14T12:13:54.477792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:58.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478989+0000","last_change":"2024-03-14T12:13:51.592562+0000","last_active":"2024-03-14T12:13:54.478989+0000","last_peered":"2024-03-14T12:13:54.478989+0000","last_clean":"2024-03-14T12:13:54.478989+0000","last_became_active":"2024-03-14T12:13:51.592255+0000","last_became_peered":"2024-03-14T12:13:51.592255+0000","last_unstale":"2024-03-14T12:13:54.478989+0000","last_undegraded":"2024-03-14T12:13:54.478989+0000","last_fullsized":"2024-03-14T12:13:54.478989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.574041+0000","last_change":"2024-03-14T12:13:50.566339+0000","last_active":"2024-03-14T12:13:52.574041+0000","last_peered":"2024-03-14T12:13:52.574041+0000","last_clean":"2024-03-14T12:13:52.574041+0000","last_became_active":"2024-03-14T12:13:50.566058+0000","last_became_peered":"2024-03-14T12:13:50.566058+0000","last_unstale":"2024-03-14T12:13:52.574041+0000","last_undegraded":"2024-03-14T12:13:52.574041+0000","last_fullsized":"2024-03-14T12:13:52.574041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579993+0000","last_change":"2024-03-14T12:13:51.581453+0000","last_active":"2024-03-14T12:13:52.579993+0000","last_peered":"2024-03-14T12:13:52.579993+0000","last_clean":"2024-03-14T12:13:52.579993+0000","last_became_active":"2024-03-14T12:13:51.581168+0000","last_became_peered":"2024-03-14T12:13:51.581168+0000","last_unstale":"2024-03-14T12:13:52.579993+0000","last_undegraded":"2024-03-14T12:13:52.579993+0000","last_fullsized":"2024-03-14T12:13:52.579993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477730+0000","last_change":"2024-03-14T12:13:50.568172+0000","last_active":"2024-03-14T12:13:54.477730+0000","last_peered":"2024-03-14T12:13:54.477730+0000","last_clean":"2024-03-14T12:13:54.477730+0000","last_became_active":"2024-03-14T12:13:50.567825+0000","last_became_peered":"2024-03-14T12:13:50.567825+0000","last_unstale":"2024-03-14T12:13:54.477730+0000","last_undegraded":"2024-03-14T12:13:54.477730+0000","last_fullsized":"2024-03-14T12:13:54.477730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478581+0000","last_change":"2024-03-14T12:13:51.593995+0000","last_active":"2024-03-14T12:13:54.478581+0000","last_peered":"2024-03-14T12:13:54.478581+0000","last_clean":"2024-03-14T12:13:54.478581+0000","last_became_active":"2024-03-14T12:13:51.593473+0000","last_became_peered":"2024-03-14T12:13:51.593473+0000","last_unstale":"2024-03-14T12:13:54.478581+0000","last_undegraded":"2024-03-14T12:13:54.478581+0000","last_fullsized":"2024-03-14T12:13:54.478581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:36:04.434847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478363+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:13:54.478363+0000","last_peered":"2024-03-14T12:13:54.478363+0000","last_clean":"2024-03-14T12:13:54.478363+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:13:54.478363+0000","last_undegraded":"2024-03-14T12:13:54.478363+0000","last_fullsized":"2024-03-14T12:13:54.478363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580336+0000","last_change":"2024-03-14T12:13:50.565229+0000","last_active":"2024-03-14T12:13:52.580336+0000","last_peered":"2024-03-14T12:13:52.580336+0000","last_clean":"2024-03-14T12:13:52.580336+0000","last_became_active":"2024-03-14T12:13:50.564952+0000","last_became_peered":"2024-03-14T12:13:50.564952+0000","last_unstale":"2024-03-14T12:13:52.580336+0000","last_undegraded":"2024-03-14T12:13:52.580336+0000","last_fullsized":"2024-03-14T12:13:52.580336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:22:30.506594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478072+0000","last_change":"2024-03-14T12:13:51.573189+0000","last_active":"2024-03-14T12:13:54.478072+0000","last_peered":"2024-03-14T12:13:54.478072+0000","last_clean":"2024-03-14T12:13:54.478072+0000","last_became_active":"2024-03-14T12:13:51.572900+0000","last_became_peered":"2024-03-14T12:13:51.572900+0000","last_unstale":"2024-03-14T12:13:54.478072+0000","last_undegraded":"2024-03-14T12:13:54.478072+0000","last_fullsized":"2024-03-14T12:13:54.478072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:37:29.600479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478221+0000","last_change":"2024-03-14T12:13:50.571689+0000","last_active":"2024-03-14T12:13:54.478221+0000","last_peered":"2024-03-14T12:13:54.478221+0000","last_clean":"2024-03-14T12:13:54.478221+0000","last_became_active":"2024-03-14T12:13:50.571292+0000","last_became_peered":"2024-03-14T12:13:50.571292+0000","last_unstale":"2024-03-14T12:13:54.478221+0000","last_undegraded":"2024-03-14T12:13:54.478221+0000","last_fullsized":"2024-03-14T12:13:54.478221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777395+0000","last_change":"2024-03-14T12:13:51.582845+0000","last_active":"2024-03-14T12:13:52.777395+0000","last_peered":"2024-03-14T12:13:52.777395+0000","last_clean":"2024-03-14T12:13:52.777395+0000","last_became_active":"2024-03-14T12:13:51.582580+0000","last_became_peered":"2024-03-14T12:13:51.582580+0000","last_unstale":"2024-03-14T12:13:52.777395+0000","last_undegraded":"2024-03-14T12:13:52.777395+0000","last_fullsized":"2024-03-14T12:13:52.777395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:17:12.149219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477686+0000","last_change":"2024-03-14T12:13:50.568041+0000","last_active":"2024-03-14T12:13:54.477686+0000","last_peered":"2024-03-14T12:13:54.477686+0000","last_clean":"2024-03-14T12:13:54.477686+0000","last_became_active":"2024-03-14T12:13:50.567698+0000","last_became_peered":"2024-03-14T12:13:50.567698+0000","last_unstale":"2024-03-14T12:13:54.477686+0000","last_undegraded":"2024-03-14T12:13:54.477686+0000","last_fullsized":"2024-03-14T12:13:54.477686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573917+0000","last_change":"2024-03-14T12:13:51.572247+0000","last_active":"2024-03-14T12:13:52.573917+0000","last_peered":"2024-03-14T12:13:52.573917+0000","last_clean":"2024-03-14T12:13:52.573917+0000","last_became_active":"2024-03-14T12:13:51.572000+0000","last_became_peered":"2024-03-14T12:13:51.572000+0000","last_unstale":"2024-03-14T12:13:52.573917+0000","last_undegraded":"2024-03-14T12:13:52.573917+0000","last_fullsized":"2024-03-14T12:13:52.573917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580774+0000","last_change":"2024-03-14T12:13:50.562695+0000","last_active":"2024-03-14T12:13:52.580774+0000","last_peered":"2024-03-14T12:13:52.580774+0000","last_clean":"2024-03-14T12:13:52.580774+0000","last_became_active":"2024-03-14T12:13:50.562419+0000","last_became_peered":"2024-03-14T12:13:50.562419+0000","last_unstale":"2024-03-14T12:13:52.580774+0000","last_undegraded":"2024-03-14T12:13:52.580774+0000","last_fullsized":"2024-03-14T12:13:52.580774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580029+0000","last_change":"2024-03-14T12:13:51.578692+0000","last_active":"2024-03-14T12:13:52.580029+0000","last_peered":"2024-03-14T12:13:52.580029+0000","last_clean":"2024-03-14T12:13:52.580029+0000","last_became_active":"2024-03-14T12:13:51.578375+0000","last_became_peered":"2024-03-14T12:13:51.578375+0000","last_unstale":"2024-03-14T12:13:52.580029+0000","last_undegraded":"2024-03-14T12:13:52.580029+0000","last_fullsized":"2024-03-14T12:13:52.580029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:50.543756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573848+0000","last_change":"2024-03-14T12:13:50.568583+0000","last_active":"2024-03-14T12:13:52.573848+0000","last_peered":"2024-03-14T12:13:52.573848+0000","last_clean":"2024-03-14T12:13:52.573848+0000","last_became_active":"2024-03-14T12:13:50.568267+0000","last_became_peered":"2024-03-14T12:13:50.568267+0000","last_unstale":"2024-03-14T12:13:52.573848+0000","last_undegraded":"2024-03-14T12:13:52.573848+0000","last_fullsized":"2024-03-14T12:13:52.573848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580655+0000","last_change":"2024-03-14T12:13:51.579205+0000","last_active":"2024-03-14T12:13:52.580655+0000","last_peered":"2024-03-14T12:13:52.580655+0000","last_clean":"2024-03-14T12:13:52.580655+0000","last_became_active":"2024-03-14T12:13:51.578901+0000","last_became_peered":"2024-03-14T12:13:51.578901+0000","last_unstale":"2024-03-14T12:13:52.580655+0000","last_undegraded":"2024-03-14T12:13:52.580655+0000","last_fullsized":"2024-03-14T12:13:52.580655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:57.743071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.479092+0000","last_change":"2024-03-14T12:13:50.571020+0000","last_active":"2024-03-14T12:13:54.479092+0000","last_peered":"2024-03-14T12:13:54.479092+0000","last_clean":"2024-03-14T12:13:54.479092+0000","last_became_active":"2024-03-14T12:13:50.570744+0000","last_became_peered":"2024-03-14T12:13:50.570744+0000","last_unstale":"2024-03-14T12:13:54.479092+0000","last_undegraded":"2024-03-14T12:13:54.479092+0000","last_fullsized":"2024-03-14T12:13:54.479092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573864+0000","last_change":"2024-03-14T12:13:51.574187+0000","last_active":"2024-03-14T12:13:52.573864+0000","last_peered":"2024-03-14T12:13:52.573864+0000","last_clean":"2024-03-14T12:13:52.573864+0000","last_became_active":"2024-03-14T12:13:51.573916+0000","last_became_peered":"2024-03-14T12:13:51.573916+0000","last_unstale":"2024-03-14T12:13:52.573864+0000","last_undegraded":"2024-03-14T12:13:52.573864+0000","last_fullsized":"2024-03-14T12:13:52.573864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580829+0000","last_change":"2024-03-14T12:13:50.569678+0000","last_active":"2024-03-14T12:13:52.580829+0000","last_peered":"2024-03-14T12:13:52.580829+0000","last_clean":"2024-03-14T12:13:52.580829+0000","last_became_active":"2024-03-14T12:13:50.569377+0000","last_became_peered":"2024-03-14T12:13:50.569377+0000","last_unstale":"2024-03-14T12:13:52.580829+0000","last_undegraded":"2024-03-14T12:13:52.580829+0000","last_fullsized":"2024-03-14T12:13:52.580829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:35.936763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478237+0000","last_change":"2024-03-14T12:13:51.579167+0000","last_active":"2024-03-14T12:13:54.478237+0000","last_peered":"2024-03-14T12:13:54.478237+0000","last_clean":"2024-03-14T12:13:54.478237+0000","last_became_active":"2024-03-14T12:13:51.578971+0000","last_became_peered":"2024-03-14T12:13:51.578971+0000","last_unstale":"2024-03-14T12:13:54.478237+0000","last_undegraded":"2024-03-14T12:13:54.478237+0000","last_fullsized":"2024-03-14T12:13:54.478237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:16:13.886718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.582080+0000","last_change":"2024-03-14T12:13:50.570044+0000","last_active":"2024-03-14T12:13:52.582080+0000","last_peered":"2024-03-14T12:13:52.582080+0000","last_clean":"2024-03-14T12:13:52.582080+0000","last_became_active":"2024-03-14T12:13:50.569759+0000","last_became_peered":"2024-03-14T12:13:50.569759+0000","last_unstale":"2024-03-14T12:13:52.582080+0000","last_undegraded":"2024-03-14T12:13:52.582080+0000","last_fullsized":"2024-03-14T12:13:52.582080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777595+0000","last_change":"2024-03-14T12:13:51.581169+0000","last_active":"2024-03-14T12:13:52.777595+0000","last_peered":"2024-03-14T12:13:52.777595+0000","last_clean":"2024-03-14T12:13:52.777595+0000","last_became_active":"2024-03-14T12:13:51.580757+0000","last_became_peered":"2024-03-14T12:13:51.580757+0000","last_unstale":"2024-03-14T12:13:52.777595+0000","last_undegraded":"2024-03-14T12:13:52.777595+0000","last_fullsized":"2024-03-14T12:13:52.777595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478220+0000","last_change":"2024-03-14T12:13:50.571439+0000","last_active":"2024-03-14T12:13:54.478220+0000","last_peered":"2024-03-14T12:13:54.478220+0000","last_clean":"2024-03-14T12:13:54.478220+0000","last_became_active":"2024-03-14T12:13:50.571184+0000","last_became_peered":"2024-03-14T12:13:50.571184+0000","last_unstale":"2024-03-14T12:13:54.478220+0000","last_undegraded":"2024-03-14T12:13:54.478220+0000","last_fullsized":"2024-03-14T12:13:54.478220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:21.281841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579859+0000","last_change":"2024-03-14T12:13:51.573261+0000","last_active":"2024-03-14T12:13:52.579859+0000","last_peered":"2024-03-14T12:13:52.579859+0000","last_clean":"2024-03-14T12:13:52.579859+0000","last_became_active":"2024-03-14T12:13:51.572866+0000","last_became_peered":"2024-03-14T12:13:51.572866+0000","last_unstale":"2024-03-14T12:13:52.579859+0000","last_undegraded":"2024-03-14T12:13:52.579859+0000","last_fullsized":"2024-03-14T12:13:52.579859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:39.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573916+0000","last_change":"2024-03-14T12:13:50.570820+0000","last_active":"2024-03-14T12:13:52.573916+0000","last_peered":"2024-03-14T12:13:52.573916+0000","last_clean":"2024-03-14T12:13:52.573916+0000","last_became_active":"2024-03-14T12:13:50.570423+0000","last_became_peered":"2024-03-14T12:13:50.570423+0000","last_unstale":"2024-03-14T12:13:52.573916+0000","last_undegraded":"2024-03-14T12:13:52.573916+0000","last_fullsized":"2024-03-14T12:13:52.573916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.477998+0000","last_change":"2024-03-14T12:13:51.575973+0000","last_active":"2024-03-14T12:13:54.477998+0000","last_peered":"2024-03-14T12:13:54.477998+0000","last_clean":"2024-03-14T12:13:54.477998+0000","last_became_active":"2024-03-14T12:13:51.575695+0000","last_became_peered":"2024-03-14T12:13:51.575695+0000","last_unstale":"2024-03-14T12:13:54.477998+0000","last_undegraded":"2024-03-14T12:13:54.477998+0000","last_fullsized":"2024-03-14T12:13:54.477998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:34.290756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.573877+0000","last_change":"2024-03-14T12:13:50.565398+0000","last_active":"2024-03-14T12:13:52.573877+0000","last_peered":"2024-03-14T12:13:52.573877+0000","last_clean":"2024-03-14T12:13:52.573877+0000","last_became_active":"2024-03-14T12:13:50.565109+0000","last_became_peered":"2024-03-14T12:13:50.565109+0000","last_unstale":"2024-03-14T12:13:52.573877+0000","last_undegraded":"2024-03-14T12:13:52.573877+0000","last_fullsized":"2024-03-14T12:13:52.573877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:54.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478222+0000","last_change":"2024-03-14T12:13:51.576916+0000","last_active":"2024-03-14T12:13:54.478222+0000","last_peered":"2024-03-14T12:13:54.478222+0000","last_clean":"2024-03-14T12:13:54.478222+0000","last_became_active":"2024-03-14T12:13:51.576639+0000","last_became_peered":"2024-03-14T12:13:51.576639+0000","last_unstale":"2024-03-14T12:13:54.478222+0000","last_undegraded":"2024-03-14T12:13:54.478222+0000","last_fullsized":"2024-03-14T12:13:54.478222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:03:05.733065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478704+0000","last_change":"2024-03-14T12:13:50.575585+0000","last_active":"2024-03-14T12:13:54.478704+0000","last_peered":"2024-03-14T12:13:54.478704+0000","last_clean":"2024-03-14T12:13:54.478704+0000","last_became_active":"2024-03-14T12:13:50.575288+0000","last_became_peered":"2024-03-14T12:13:50.575288+0000","last_unstale":"2024-03-14T12:13:54.478704+0000","last_undegraded":"2024-03-14T12:13:54.478704+0000","last_fullsized":"2024-03-14T12:13:54.478704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:38.043320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.581882+0000","last_change":"2024-03-14T12:13:51.588022+0000","last_active":"2024-03-14T12:13:52.581882+0000","last_peered":"2024-03-14T12:13:52.581882+0000","last_clean":"2024-03-14T12:13:52.581882+0000","last_became_active":"2024-03-14T12:13:51.587766+0000","last_became_peered":"2024-03-14T12:13:51.587766+0000","last_unstale":"2024-03-14T12:13:52.581882+0000","last_undegraded":"2024-03-14T12:13:52.581882+0000","last_fullsized":"2024-03-14T12:13:52.581882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:03:42.575475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.580894+0000","last_change":"2024-03-14T12:13:51.572504+0000","last_active":"2024-03-14T12:13:52.580894+0000","last_peered":"2024-03-14T12:13:52.580894+0000","last_clean":"2024-03-14T12:13:52.580894+0000","last_became_active":"2024-03-14T12:13:51.572219+0000","last_became_peered":"2024-03-14T12:13:51.572219+0000","last_unstale":"2024-03-14T12:13:52.580894+0000","last_undegraded":"2024-03-14T12:13:52.580894+0000","last_fullsized":"2024-03-14T12:13:52.580894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:48:21.729314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478474+0000","last_change":"2024-03-14T12:13:50.572648+0000","last_active":"2024-03-14T12:13:54.478474+0000","last_peered":"2024-03-14T12:13:54.478474+0000","last_clean":"2024-03-14T12:13:54.478474+0000","last_became_active":"2024-03-14T12:13:50.572409+0000","last_became_peered":"2024-03-14T12:13:50.572409+0000","last_unstale":"2024-03-14T12:13:54.478474+0000","last_undegraded":"2024-03-14T12:13:54.478474+0000","last_fullsized":"2024-03-14T12:13:54.478474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:01.270108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.579831+0000","last_change":"2024-03-14T12:13:51.580823+0000","last_active":"2024-03-14T12:13:52.579831+0000","last_peered":"2024-03-14T12:13:52.579831+0000","last_clean":"2024-03-14T12:13:52.579831+0000","last_became_active":"2024-03-14T12:13:51.574625+0000","last_became_peered":"2024-03-14T12:13:51.574625+0000","last_unstale":"2024-03-14T12:13:52.579831+0000","last_undegraded":"2024-03-14T12:13:52.579831+0000","last_fullsized":"2024-03-14T12:13:52.579831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:58:19.654431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478728+0000","last_change":"2024-03-14T12:13:50.572321+0000","last_active":"2024-03-14T12:13:54.478728+0000","last_peered":"2024-03-14T12:13:54.478728+0000","last_clean":"2024-03-14T12:13:54.478728+0000","last_became_active":"2024-03-14T12:13:50.572001+0000","last_became_peered":"2024-03-14T12:13:50.572001+0000","last_unstale":"2024-03-14T12:13:54.478728+0000","last_undegraded":"2024-03-14T12:13:54.478728+0000","last_fullsized":"2024-03-14T12:13:54.478728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:49.539889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777630+0000","last_change":"2024-03-14T12:13:51.575693+0000","last_active":"2024-03-14T12:13:52.777630+0000","last_peered":"2024-03-14T12:13:52.777630+0000","last_clean":"2024-03-14T12:13:52.777630+0000","last_became_active":"2024-03-14T12:13:51.574493+0000","last_became_peered":"2024-03-14T12:13:51.574493+0000","last_unstale":"2024-03-14T12:13:52.777630+0000","last_undegraded":"2024-03-14T12:13:52.777630+0000","last_fullsized":"2024-03-14T12:13:52.777630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:43:33.800282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478473+0000","last_change":"2024-03-14T12:13:50.565913+0000","last_active":"2024-03-14T12:13:54.478473+0000","last_peered":"2024-03-14T12:13:54.478473+0000","last_clean":"2024-03-14T12:13:54.478473+0000","last_became_active":"2024-03-14T12:13:50.565652+0000","last_became_peered":"2024-03-14T12:13:50.565652+0000","last_unstale":"2024-03-14T12:13:54.478473+0000","last_undegraded":"2024-03-14T12:13:54.478473+0000","last_fullsized":"2024-03-14T12:13:54.478473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:21:11.132460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:54.478392+0000","last_change":"2024-03-14T12:13:51.581638+0000","last_active":"2024-03-14T12:13:54.478392+0000","last_peered":"2024-03-14T12:13:54.478392+0000","last_clean":"2024-03-14T12:13:54.478392+0000","last_became_active":"2024-03-14T12:13:51.581373+0000","last_became_peered":"2024-03-14T12:13:51.581373+0000","last_unstale":"2024-03-14T12:13:54.478392+0000","last_undegraded":"2024-03-14T12:13:54.478392+0000","last_fullsized":"2024-03-14T12:13:54.478392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:50.546003+0000","last_clean_scrub_stamp":"2024-03-14T12:13:50.546003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:07.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T12:13:52.777698+0000","last_change":"2024-03-14T12:13:50.570080+0000","last_active":"2024-03-14T12:13:52.777698+0000","last_peered":"2024-03-14T12:13:52.777698+0000","last_clean":"2024-03-14T12:13:52.777698+0000","last_became_active":"2024-03-14T12:13:50.569816+0000","last_became_peered":"2024-03-14T12:13:50.569816+0000","last_unstale":"2024-03-14T12:13:52.777698+0000","last_undegraded":"2024-03-14T12:13:52.777698+0000","last_fullsized":"2024-03-14T12:13:52.777698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:49.542136+0000","last_clean_scrub_stamp":"2024-03-14T12:13:49.542136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:55:45.592652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:14:02.441 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-14T12:14:02.441 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-14T12:14:02.442 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-14T12:14:02.442 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-14T12:14:02.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe263843640 1 Processor -- start 2024-03-14T12:14:02.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe263843640 1 -- start start 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe263843640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe25c0d2620 0x7fe25c0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe263843640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe25c0b47c0 0x7fe25c0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe263843640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25c0d4570 con 0x7fe25c0d2a00 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe263843640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25c0cffd0 con 0x7fe25c0d2620 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe263843640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25c0d4850 con 0x7fe25c0b47c0 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe2615df640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe25c0d2620 0x7fe25c0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe2615df640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe25c0d2620 0x7fe25c0b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51192/0 (socket says 172.21.15.82:51192) 2024-03-14T12:14:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe2615df640 1 -- 172.21.15.82:0/2064038053 learned_addr learned my addr 172.21.15.82:0/2064038053 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe261de0640 1 --2- 172.21.15.82:0/2064038053 >> v2:172.21.15.164:3300/0 conn(0x7fe25c0b47c0 0x7fe25c0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe2615df640 1 -- 172.21.15.82:0/2064038053 >> v2:172.21.15.164:3300/0 conn(0x7fe25c0b47c0 msgr2=0x7fe25c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe2615df640 1 --2- 172.21.15.82:0/2064038053 >> v2:172.21.15.164:3300/0 conn(0x7fe25c0b47c0 0x7fe25c0cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe2615df640 1 -- 172.21.15.82:0/2064038053 >> v1:172.21.15.164:6789/0 conn(0x7fe25c0d2a00 legacy=0x7fe25c15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.577+0000 7fe2615df640 1 -- 172.21.15.82:0/2064038053 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c0d4b30 con 0x7fe25c0d2620 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.578+0000 7fe2615df640 1 --2- 172.21.15.82:0/2064038053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0d2620 0x7fe25c0b5260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0d2e90 tx=0x7fe25802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447bcae0a49ad33d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.578+0000 7fe253fff640 1 -- 172.21.15.82:0/2064038053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258034070 con 0x7fe25c0d2620 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.578+0000 7fe253fff640 1 -- 172.21.15.82:0/2064038053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258030860 con 0x7fe25c0d2620 2024-03-14T12:14:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.578+0000 7fe253fff640 1 -- 172.21.15.82:0/2064038053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258030c30 con 0x7fe25c0d2620 2024-03-14T12:14:02.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 -- 172.21.15.82:0/2064038053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0d2620 msgr2=0x7fe25c0b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 --2- 172.21.15.82:0/2064038053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0d2620 0x7fe25c0b5260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0d2e90 tx=0x7fe25802b9a0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 -- 172.21.15.82:0/2064038053 shutdown_connections 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 --2- 172.21.15.82:0/2064038053 >> v2:172.21.15.164:3300/0 conn(0x7fe25c0b47c0 0x7fe25c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 --2- 172.21.15.82:0/2064038053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0d2620 0x7fe25c0b5260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 -- 172.21.15.82:0/2064038053 >> 172.21.15.82:0/2064038053 conn(0x7fe25c09d0b0 msgr2=0x7fe25c0d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 -- 172.21.15.82:0/2064038053 shutdown_connections 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.579+0000 7fe263843640 1 -- 172.21.15.82:0/2064038053 wait complete. 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.580+0000 7fe263843640 1 Processor -- start 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.580+0000 7fe263843640 1 -- start start 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe263843640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0b47c0 0x7fe25c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe263843640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe25c0d2620 0x7fe25c0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe263843640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe25c0d2a00 0x7fe25c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe263843640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c15e760 con 0x7fe25c0b47c0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe263843640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c15e5e0 con 0x7fe25c0d2620 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe263843640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c15e8e0 con 0x7fe25c0d2a00 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0b47c0 0x7fe25c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0b47c0 0x7fe25c158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51200/0 (socket says 172.21.15.82:51200) 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 -- 172.21.15.82:0/3802413617 learned_addr learned my addr 172.21.15.82:0/3802413617 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe261de0640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe25c0d2a00 0x7fe25c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe260dde640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe25c0d2620 0x7fe25c0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 -- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe25c0d2a00 msgr2=0x7fe25c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe25c0d2a00 0x7fe25c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 -- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe25c0d2620 msgr2=0x7fe25c0b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe25c0d2620 0x7fe25c0b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 -- 172.21.15.82:0/3802413617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c0b6d50 con 0x7fe25c0b47c0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe2615df640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0b47c0 0x7fe25c158960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe258000c00 tx=0x7fe2580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe251ffb640 1 -- 172.21.15.82:0/3802413617 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258034040 con 0x7fe25c0b47c0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe251ffb640 1 -- 172.21.15.82:0/3802413617 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258039d90 con 0x7fe25c0b47c0 2024-03-14T12:14:02.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c0b7030 con 0x7fe25c0b47c0 2024-03-14T12:14:02.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.581+0000 7fe251ffb640 1 -- 172.21.15.82:0/3802413617 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258043610 con 0x7fe25c0b47c0 2024-03-14T12:14:02.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.582+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c0b75f0 con 0x7fe25c0b47c0 2024-03-14T12:14:02.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.583+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c04f050 con 0x7fe25c0b47c0 2024-03-14T12:14:02.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.583+0000 7fe251ffb640 1 -- 172.21.15.82:0/3802413617 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe258037070 con 0x7fe25c0b47c0 2024-03-14T12:14:02.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.583+0000 7fe251ffb640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe24006bc00 0x7fe24006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.583+0000 7fe251ffb640 1 -- 172.21.15.82:0/3802413617 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2580aca50 con 0x7fe25c0b47c0 2024-03-14T12:14:02.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.583+0000 7fe260dde640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe24006bc00 0x7fe24006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.584+0000 7fe260dde640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe24006bc00 0x7fe24006e0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe25c158ea0 tx=0x7fe24c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.587+0000 7fe251ffb640 1 -- 172.21.15.82:0/3802413617 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe25803e070 con 0x7fe25c0b47c0 2024-03-14T12:14:02.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.794+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe25c074740 con 0x7fe25c0b47c0 2024-03-14T12:14:02.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.795+0000 7fe251ffb640 1 -- 172.21.15.82:0/3802413617 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe25807af80 con 0x7fe25c0b47c0 2024-03-14T12:14:02.795 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:02.795 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe24006bc00 msgr2=0x7fe24006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe24006bc00 0x7fe24006e0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe25c158ea0 tx=0x7fe24c009040 comp rx=0 tx=0).stop 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0b47c0 msgr2=0x7fe25c158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0b47c0 0x7fe25c158960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe258000c00 tx=0x7fe2580046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 shutdown_connections 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe24006bc00 0x7fe24006e0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe25c0d2a00 0x7fe25c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe25c0d2620 0x7fe25c0b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 --2- 172.21.15.82:0/3802413617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe25c0b47c0 0x7fe25c158960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 >> 172.21.15.82:0/3802413617 conn(0x7fe25c09d0b0 msgr2=0x7fe25c0ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 shutdown_connections 2024-03-14T12:14:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.797+0000 7fe263843640 1 -- 172.21.15.82:0/3802413617 wait complete. 2024-03-14T12:14:02.808 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-14T12:14:02.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:02.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.907+0000 7f8037eab640 1 Processor -- start 2024-03-14T12:14:02.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.907+0000 7f8037eab640 1 -- start start 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8037eab640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f803015e6b0 0x7f80300cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8037eab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f803006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8037eab640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803006cca0 con 0x7f80300b4350 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8037eab640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803006cf80 con 0x7f803015e6b0 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8037eab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803006f1d0 con 0x7f803006c7e0 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8035c47640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f803015e6b0 0x7f80300cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8035446640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f803006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8036448640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f80300b4350 0x7f803015ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:52026/0 (socket says 172.21.15.82:52026) 2024-03-14T12:14:02.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8036448640 1 -- 172.21.15.82:0/437764174 learned_addr learned my addr 172.21.15.82:0/437764174 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:02.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8035c47640 1 -- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 msgr2=0x7f803006f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:02.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8035c47640 1 --2- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f803006f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8035c47640 1 -- 172.21.15.82:0/437764174 >> v1:172.21.15.164:6789/0 conn(0x7f80300b4350 legacy=0x7f803015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8035c47640 1 -- 172.21.15.82:0/437764174 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803006f4b0 con 0x7f803015e6b0 2024-03-14T12:14:02.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.909+0000 7f8035446640 1 --2- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f803006f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:02.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8035c47640 1 --2- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f803015e6b0 0x7f80300cfa60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f802c0097a0 tx=0x7f802c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcfbd7a280274b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8034c45640 1 -- 172.21.15.82:0/437764174 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c035020 con 0x7f803015e6b0 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8034c45640 1 -- 172.21.15.82:0/437764174 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c031830 con 0x7f803015e6b0 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8034c45640 1 -- 172.21.15.82:0/437764174 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c031c00 con 0x7f803015e6b0 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 -- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f803015e6b0 msgr2=0x7f80300cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 --2- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f803015e6b0 0x7f80300cfa60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f802c0097a0 tx=0x7f802c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 -- 172.21.15.82:0/437764174 shutdown_connections 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 --2- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f803006f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 --2- 172.21.15.82:0/437764174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f803015e6b0 0x7f80300cfa60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 -- 172.21.15.82:0/437764174 >> 172.21.15.82:0/437764174 conn(0x7f80300b3670 msgr2=0x7f80300b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 -- 172.21.15.82:0/437764174 shutdown_connections 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.910+0000 7f8037eab640 1 -- 172.21.15.82:0/437764174 wait complete. 2024-03-14T12:14:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 Processor -- start 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 -- start start 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f80301f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80300b4350 0x7f803009ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f803015e6b0 0x7f8030149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80300cffa0 con 0x7f803015e6b0 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803006ee80 con 0x7f80300b4350 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8037eab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300d0120 con 0x7f803006c7e0 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8036448640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f803015e6b0 0x7f8030149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8036448640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f803015e6b0 0x7f8030149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51202/0 (socket says 172.21.15.82:51202) 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8036448640 1 -- 172.21.15.82:0/2640335324 learned_addr learned my addr 172.21.15.82:0/2640335324 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.912+0000 7f8035c47640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f80301f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8035446640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80300b4350 0x7f803009ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8036448640 1 -- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 msgr2=0x7f80301f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8036448640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f80301f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8036448640 1 -- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80300b4350 msgr2=0x7f803009ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8036448640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80300b4350 0x7f803009ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8036448640 1 -- 172.21.15.82:0/2640335324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300b6550 con 0x7f803015e6b0 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8036448640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f803015e6b0 0x7f8030149f10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f802800caf0 tx=0x7f802800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f80267fc640 1 -- 172.21.15.82:0/2640335324 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028017070 con 0x7f803015e6b0 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f80267fc640 1 -- 172.21.15.82:0/2640335324 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802800aca0 con 0x7f803015e6b0 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f80267fc640 1 -- 172.21.15.82:0/2640335324 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028012600 con 0x7f803015e6b0 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80300b6890 con 0x7f803015e6b0 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.914+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80300b70d0 con 0x7f803015e6b0 2024-03-14T12:14:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.915+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80300b6a70 con 0x7f803015e6b0 2024-03-14T12:14:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.915+0000 7f80267fc640 1 -- 172.21.15.82:0/2640335324 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f802801a070 con 0x7f803015e6b0 2024-03-14T12:14:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.916+0000 7f80267fc640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f801406bbd0 0x7f801406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.916+0000 7f8035c47640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f801406bbd0 0x7f801406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.916+0000 7f80267fc640 1 -- 172.21.15.82:0/2640335324 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f802808a660 con 0x7f803015e6b0 2024-03-14T12:14:02.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.916+0000 7f8035c47640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f801406bbd0 0x7f801406e090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f802c036000 tx=0x7f802c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:02.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:02.919+0000 7f80267fc640 1 -- 172.21.15.82:0/2640335324 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f80300b6890 con 0x7f803015e6b0 2024-03-14T12:14:03.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.092+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f803004f050 con 0x7f803015e6b0 2024-03-14T12:14:03.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.093+0000 7f80267fc640 1 -- 172.21.15.82:0/2640335324 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f8028058b90 con 0x7f803015e6b0 2024-03-14T12:14:03.092 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:03.093 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f801406bbd0 msgr2=0x7f801406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f801406bbd0 0x7f801406e090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f802c036000 tx=0x7f802c008030 comp rx=0 tx=0).stop 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f803015e6b0 msgr2=0x7f8030149f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f803015e6b0 0x7f8030149f10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f802800caf0 tx=0x7f802800a060 comp rx=0 tx=0).stop 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 shutdown_connections 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f803015e6b0 0x7f8030149f10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80300b4350 0x7f803009ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f801406bbd0 0x7f801406e090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 --2- 172.21.15.82:0/2640335324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f803006c7e0 0x7f80301f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 >> 172.21.15.82:0/2640335324 conn(0x7f80300b3670 msgr2=0x7f80300aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:03.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 shutdown_connections 2024-03-14T12:14:03.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.095+0000 7f8037eab640 1 -- 172.21.15.82:0/2640335324 wait complete. 2024-03-14T12:14:03.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:03.106 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:03.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.210+0000 7f2401e38640 1 Processor -- start 2024-03-14T12:14:03.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.210+0000 7f2401e38640 1 -- start start 2024-03-14T12:14:03.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f2401e38640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23fc15dd70 0x7f23fc06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f2401e38640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15e150 0x7f23fc0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f2401e38640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc070250 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f2401e38640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23fc06ea20 con 0x7f23fc15dd70 2024-03-14T12:14:03.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f2401e38640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc06f880 con 0x7f23fc15e150 2024-03-14T12:14:03.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23faffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15e150 0x7f23fc0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fb7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23fc15dd70 0x7f23fc06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fbfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f23fc0b6bd0 0x7f23fc0d1b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:52038/0 (socket says 172.21.15.82:52038) 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fbfff640 1 -- 172.21.15.82:0/3080257009 learned_addr learned my addr 172.21.15.82:0/3080257009 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fb7fe640 1 -- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15e150 msgr2=0x7f23fc0b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fb7fe640 1 --2- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15e150 0x7f23fc0b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fb7fe640 1 -- 172.21.15.82:0/3080257009 >> v1:172.21.15.164:6789/0 conn(0x7f23fc0b6bd0 legacy=0x7f23fc0d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fb7fe640 1 -- 172.21.15.82:0/3080257009 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc06ec70 con 0x7f23fc15dd70 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23fa7fc640 1 -- 172.21.15.82:0/3080257009 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864137796 0 0) 0x7f23fc070250 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.211+0000 7f23faffd640 1 --2- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15e150 0x7f23fc0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.212+0000 7f23fb7fe640 1 --2- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15dd70 0x7f23fc06fd10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097a0 tx=0x7f23ec02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf1a35750721cce server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.212+0000 7f23fa7fc640 1 -- 172.21.15.82:0/3080257009 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec035020 con 0x7f23fc15dd70 2024-03-14T12:14:03.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.212+0000 7f23fa7fc640 1 -- 172.21.15.82:0/3080257009 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec031830 con 0x7f23fc15dd70 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.212+0000 7f23fa7fc640 1 -- 172.21.15.82:0/3080257009 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec031c00 con 0x7f23fc15dd70 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 -- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15dd70 msgr2=0x7f23fc06fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 --2- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15dd70 0x7f23fc06fd10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097a0 tx=0x7f23ec02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 -- 172.21.15.82:0/3080257009 shutdown_connections 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 --2- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15e150 0x7f23fc0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 --2- 172.21.15.82:0/3080257009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15dd70 0x7f23fc06fd10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 -- 172.21.15.82:0/3080257009 >> 172.21.15.82:0/3080257009 conn(0x7f23fc072ec0 msgr2=0x7f23fc09a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 -- 172.21.15.82:0/3080257009 shutdown_connections 2024-03-14T12:14:03.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.213+0000 7f2401e38640 1 -- 172.21.15.82:0/3080257009 wait complete. 2024-03-14T12:14:03.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.214+0000 7f2401e38640 1 Processor -- start 2024-03-14T12:14:03.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.214+0000 7f2401e38640 1 -- start start 2024-03-14T12:14:03.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f2401e38640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23fc0b6bd0 0x7f23fc14d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f2401e38640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15dd70 0x7f23fc160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f2401e38640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15e150 0x7f23fc160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f2401e38640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc06f510 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f2401e38640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc06f230 con 0x7f23fc15e150 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f2401e38640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc0d2210 con 0x7f23fc15dd70 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23fc0b6bd0 0x7f23fc14d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23fc0b6bd0 0x7f23fc14d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51210/0 (socket says 172.21.15.82:51210) 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 -- 172.21.15.82:0/467601077 learned_addr learned my addr 172.21.15.82:0/467601077 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23faffd640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15dd70 0x7f23fc160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 -- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15dd70 msgr2=0x7f23fc160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fbfff640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15e150 0x7f23fc160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15dd70 0x7f23fc160720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 -- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15e150 msgr2=0x7f23fc160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15e150 0x7f23fc160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 -- 172.21.15.82:0/467601077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc09c6d0 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fbfff640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15e150 0x7f23fc160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23fb7fe640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23fc0b6bd0 0x7f23fc14d740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23ec02bee0 tx=0x7f23ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.215+0000 7f23e3fff640 1 -- 172.21.15.82:0/467601077 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec035040 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.216+0000 7f23e3fff640 1 -- 172.21.15.82:0/467601077 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec03b920 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.216+0000 7f23e3fff640 1 -- 172.21.15.82:0/467601077 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec03bd10 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.216+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc09c9b0 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.216+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc159de0 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.217+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc159fe0 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.217+0000 7f23e3fff640 1 -- 172.21.15.82:0/467601077 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f23ec038020 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.217+0000 7f23e3fff640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f23e806bbd0 0x7f23e806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.217+0000 7f23e3fff640 1 -- 172.21.15.82:0/467601077 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23ec0acb30 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.217+0000 7f23faffd640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f23e806bbd0 0x7f23e806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.218+0000 7f23faffd640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f23e806bbd0 0x7f23e806e090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0b6450 tx=0x7f23f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.222+0000 7f23e3fff640 1 -- 172.21.15.82:0/467601077 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f23ec07b630 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.393+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23fc0af860 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.393+0000 7f23e3fff640 1 -- 172.21.15.82:0/467601077 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23ec031800 con 0x7f23fc0b6bd0 2024-03-14T12:14:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:03.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f23e806bbd0 msgr2=0x7f23e806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f23e806bbd0 0x7f23e806e090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0b6450 tx=0x7f23f0008040 comp rx=0 tx=0).stop 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23fc0b6bd0 msgr2=0x7f23fc14d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23fc0b6bd0 0x7f23fc14d740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23ec02bee0 tx=0x7f23ec0046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 shutdown_connections 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23fc15e150 0x7f23fc160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f23e806bbd0 0x7f23e806e090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23fc15dd70 0x7f23fc160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 --2- 172.21.15.82:0/467601077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23fc0b6bd0 0x7f23fc14d740 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 >> 172.21.15.82:0/467601077 conn(0x7f23fc072ec0 msgr2=0x7f23fc158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 shutdown_connections 2024-03-14T12:14:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.395+0000 7f2401e38640 1 -- 172.21.15.82:0/467601077 wait complete. 2024-03-14T12:14:03.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:03.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 Processor -- start 2024-03-14T12:14:03.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 -- start start 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9268161580 0x7f92680b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f9268161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92680cfcc0 0x7f92680d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9268162490 con 0x7f92680cfcc0 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926806ee00 con 0x7f9268161580 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.509+0000 7f926cc61640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926806f0e0 con 0x7f9268161b10 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92680cfcc0 0x7f92680d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92680cfcc0 0x7f92680d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51224/0 (socket says 172.21.15.82:51224) 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 -- 172.21.15.82:0/3908400136 learned_addr learned my addr 172.21.15.82:0/3908400136 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f925dd9b640 1 --2- 172.21.15.82:0/3908400136 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f9268161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f926659c640 1 --2- 172.21.15.82:0/3908400136 >> v2:172.21.15.164:3300/0 conn(0x7f9268161580 0x7f92680b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 -- 172.21.15.82:0/3908400136 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 msgr2=0x7f9268161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 --2- 172.21.15.82:0/3908400136 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f9268161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 -- 172.21.15.82:0/3908400136 >> v2:172.21.15.164:3300/0 conn(0x7f9268161580 msgr2=0x7f92680b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 --2- 172.21.15.82:0/3908400136 >> v2:172.21.15.164:3300/0 conn(0x7f9268161580 0x7f92680b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f9266d9d640 1 -- 172.21.15.82:0/3908400136 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926806be20 con 0x7f92680cfcc0 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.510+0000 7f926659c640 1 --2- 172.21.15.82:0/3908400136 >> v2:172.21.15.164:3300/0 conn(0x7f9268161580 0x7f92680b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f9266d9d640 1 --2- 172.21.15.82:0/3908400136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f92680d0110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9268071640 tx=0x7f926002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152e16a834eb1d96 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f9265d9b640 1 -- 172.21.15.82:0/3908400136 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926002cb00 con 0x7f92680cfcc0 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f9265d9b640 1 -- 172.21.15.82:0/3908400136 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926002cca0 con 0x7f92680cfcc0 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f9265d9b640 1 -- 172.21.15.82:0/3908400136 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260034610 con 0x7f92680cfcc0 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f926cc61640 1 -- 172.21.15.82:0/3908400136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 msgr2=0x7f92680d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f926cc61640 1 --2- 172.21.15.82:0/3908400136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f92680d0110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9268071640 tx=0x7f926002bf60 comp rx=0 tx=0).stop 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f926cc61640 1 -- 172.21.15.82:0/3908400136 shutdown_connections 2024-03-14T12:14:03.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f926cc61640 1 --2- 172.21.15.82:0/3908400136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f92680d0110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f926cc61640 1 --2- 172.21.15.82:0/3908400136 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f9268161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f926cc61640 1 --2- 172.21.15.82:0/3908400136 >> v2:172.21.15.164:3300/0 conn(0x7f9268161580 0x7f92680b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.511+0000 7f926cc61640 1 -- 172.21.15.82:0/3908400136 >> 172.21.15.82:0/3908400136 conn(0x7f92680b3680 msgr2=0x7f92680b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:03.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.512+0000 7f926cc61640 1 -- 172.21.15.82:0/3908400136 shutdown_connections 2024-03-14T12:14:03.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.512+0000 7f926cc61640 1 -- 172.21.15.82:0/3908400136 wait complete. 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.513+0000 7f926cc61640 1 Processor -- start 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.513+0000 7f926cc61640 1 -- start start 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926cc61640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f9268156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926cc61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9268161580 0x7f926815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926cc61640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f92680b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926cc61640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926806e9e0 con 0x7f92680cfcc0 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926cc61640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926806c3e0 con 0x7f9268161580 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926cc61640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926806eb60 con 0x7f9268161b10 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f9268156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f9268156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51238/0 (socket says 172.21.15.82:51238) 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 -- 172.21.15.82:0/2114303111 learned_addr learned my addr 172.21.15.82:0/2114303111 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f925dd9b640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9268161580 0x7f926815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f9266d9d640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f92680b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 -- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 msgr2=0x7f92680b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f92680b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 -- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9268161580 msgr2=0x7f926815ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9268161580 0x7f926815ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 -- 172.21.15.82:0/2114303111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926815b8a0 con 0x7f92680cfcc0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.514+0000 7f926659c640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f9268156b90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f925800e7e0 tx=0x7f925800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.515+0000 7f925f7fe640 1 -- 172.21.15.82:0/2114303111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925800f040 con 0x7f92680cfcc0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.515+0000 7f925f7fe640 1 -- 172.21.15.82:0/2114303111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9258004030 con 0x7f92680cfcc0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.515+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926815bb80 con 0x7f92680cfcc0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.515+0000 7f925f7fe640 1 -- 172.21.15.82:0/2114303111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258004400 con 0x7f92680cfcc0 2024-03-14T12:14:03.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.515+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926815c140 con 0x7f92680cfcc0 2024-03-14T12:14:03.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.516+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926804f050 con 0x7f92680cfcc0 2024-03-14T12:14:03.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.516+0000 7f925f7fe640 1 -- 172.21.15.82:0/2114303111 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f925800c6d0 con 0x7f92680cfcc0 2024-03-14T12:14:03.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.516+0000 7f925f7fe640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f924806bbd0 0x7f924806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.517+0000 7f925dd9b640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f924806bbd0 0x7f924806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.517+0000 7f925f7fe640 1 -- 172.21.15.82:0/2114303111 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f925808a7b0 con 0x7f92680cfcc0 2024-03-14T12:14:03.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.517+0000 7f925dd9b640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f924806bbd0 0x7f924806e090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92680cf3b0 tx=0x7f9250009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.521+0000 7f925f7fe640 1 -- 172.21.15.82:0/2114303111 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9258017020 con 0x7f92680cfcc0 2024-03-14T12:14:03.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.693+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f926806e640 con 0x7f92680cfcc0 2024-03-14T12:14:03.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.694+0000 7f925f7fe640 1 -- 172.21.15.82:0/2114303111 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9258058c50 con 0x7f92680cfcc0 2024-03-14T12:14:03.693 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:03.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f924806bbd0 msgr2=0x7f924806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f924806bbd0 0x7f924806e090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92680cf3b0 tx=0x7f9250009190 comp rx=0 tx=0).stop 2024-03-14T12:14:03.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 msgr2=0x7f9268156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f9268156b90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f925800e7e0 tx=0x7f925800eca0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 shutdown_connections 2024-03-14T12:14:03.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f924806bbd0 0x7f924806e090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9268161b10 0x7f92680b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9268161580 0x7f926815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 --2- 172.21.15.82:0/2114303111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92680cfcc0 0x7f9268156b90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 >> 172.21.15.82:0/2114303111 conn(0x7f92680b3680 msgr2=0x7f92680d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:03.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 shutdown_connections 2024-03-14T12:14:03.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.696+0000 7f926cc61640 1 -- 172.21.15.82:0/2114303111 wait complete. 2024-03-14T12:14:03.706 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:03.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.811+0000 7f93ca359640 1 Processor -- start 2024-03-14T12:14:03.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.811+0000 7f93ca359640 1 -- start start 2024-03-14T12:14:03.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93ca359640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93c40b5e60 0x7f93c406ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93ca359640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f93c40b1200 0x7f93c406f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93ca359640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c415c0c0 con 0x7f93c415bce0 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93ca359640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40afc50 con 0x7f93c40b1200 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93ca359640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40afea0 con 0x7f93c40b5e60 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93c3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f93c415bce0 0x7f93c40b5750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54226/0 (socket says 172.21.15.82:54226) 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93c3fff640 1 -- 172.21.15.82:0/3087239739 learned_addr learned my addr 172.21.15.82:0/3087239739 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93c88f6640 1 --2- 172.21.15.82:0/3087239739 >> v2:172.21.15.82:3300/0 conn(0x7f93c40b1200 0x7f93c406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93c88f6640 1 -- 172.21.15.82:0/3087239739 >> v2:172.21.15.164:3300/0 conn(0x7f93c40b5e60 msgr2=0x7f93c406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93c88f6640 1 --2- 172.21.15.82:0/3087239739 >> v2:172.21.15.164:3300/0 conn(0x7f93c40b5e60 0x7f93c406ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93c88f6640 1 -- 172.21.15.82:0/3087239739 >> v1:172.21.15.82:6789/0 conn(0x7f93c415bce0 legacy=0x7f93c40b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.812+0000 7f93c88f6640 1 -- 172.21.15.82:0/3087239739 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40b0180 con 0x7f93c40b1200 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.813+0000 7f93c88f6640 1 --2- 172.21.15.82:0/3087239739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c40b1200 0x7f93c406f240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f93c40afdd0 tx=0x7f93bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c441e8b6752850f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.813+0000 7f93c2ffd640 1 -- 172.21.15.82:0/3087239739 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc039070 con 0x7f93c40b1200 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.813+0000 7f93c2ffd640 1 -- 172.21.15.82:0/3087239739 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc02cc60 con 0x7f93c40b1200 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.813+0000 7f93c2ffd640 1 -- 172.21.15.82:0/3087239739 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc034660 con 0x7f93c40b1200 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 -- 172.21.15.82:0/3087239739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c40b1200 msgr2=0x7f93c406f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 --2- 172.21.15.82:0/3087239739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c40b1200 0x7f93c406f240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f93c40afdd0 tx=0x7f93bc02beb0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 -- 172.21.15.82:0/3087239739 shutdown_connections 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 --2- 172.21.15.82:0/3087239739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c40b1200 0x7f93c406f240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 --2- 172.21.15.82:0/3087239739 >> v2:172.21.15.164:3300/0 conn(0x7f93c40b5e60 0x7f93c406ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 -- 172.21.15.82:0/3087239739 >> 172.21.15.82:0/3087239739 conn(0x7f93c4072ec0 msgr2=0x7f93c409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:03.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 -- 172.21.15.82:0/3087239739 shutdown_connections 2024-03-14T12:14:03.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.814+0000 7f93ca359640 1 -- 172.21.15.82:0/3087239739 wait complete. 2024-03-14T12:14:03.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.815+0000 7f93ca359640 1 Processor -- start 2024-03-14T12:14:03.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.815+0000 7f93ca359640 1 -- start start 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c40b1200 0x7f93c413cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40b5e60 0x7f93c413ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c415bce0 0x7f93c414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40b0a20 con 0x7f93c415bce0 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40b0740 con 0x7f93c40b5e60 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40b0cb0 con 0x7f93c40b1200 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c415bce0 0x7f93c414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c415bce0 0x7f93c414e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51256/0 (socket says 172.21.15.82:51256) 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 -- 172.21.15.82:0/888176305 learned_addr learned my addr 172.21.15.82:0/888176305 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c3fff640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c40b1200 0x7f93c413cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c37fe640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40b5e60 0x7f93c413ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 -- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c40b1200 msgr2=0x7f93c413cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c40b1200 0x7f93c413cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 -- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40b5e60 msgr2=0x7f93c413ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40b5e60 0x7f93c413ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 -- 172.21.15.82:0/888176305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c413d580 con 0x7f93c415bce0 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c3fff640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c40b1200 0x7f93c413cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c37fe640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40b5e60 0x7f93c413ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c88f6640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c415bce0 0x7f93c414e120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f93bc037040 tx=0x7f93bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c0ff9640 1 -- 172.21.15.82:0/888176305 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc039050 con 0x7f93c415bce0 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c0ff9640 1 -- 172.21.15.82:0/888176305 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc045070 con 0x7f93c415bce0 2024-03-14T12:14:03.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c4140510 con 0x7f93c415bce0 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93c0ff9640 1 -- 172.21.15.82:0/888176305 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc03e810 con 0x7f93c415bce0 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.816+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c413dae0 con 0x7f93c415bce0 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.817+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c404f050 con 0x7f93c415bce0 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.818+0000 7f93c0ff9640 1 -- 172.21.15.82:0/888176305 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f93bc03e9b0 con 0x7f93c415bce0 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.818+0000 7f93c0ff9640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93b006bbd0 0x7f93b006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.818+0000 7f93c0ff9640 1 -- 172.21.15.82:0/888176305 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93bc0ad9b0 con 0x7f93c415bce0 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.818+0000 7f93c3fff640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93b006bbd0 0x7f93b006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:03.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.819+0000 7f93c3fff640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93b006bbd0 0x7f93b006e090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000f60 tx=0x7f93b4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:03.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.822+0000 7f93c0ff9640 1 -- 172.21.15.82:0/888176305 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f93c413d580 con 0x7f93c415bce0 2024-03-14T12:14:03.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.994+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93c40d29e0 con 0x7f93c415bce0 2024-03-14T12:14:03.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.994+0000 7f93c0ff9640 1 -- 172.21.15.82:0/888176305 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93bc032e20 con 0x7f93c415bce0 2024-03-14T12:14:03.993 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:03.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93b006bbd0 msgr2=0x7f93b006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93b006bbd0 0x7f93b006e090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000f60 tx=0x7f93b4009040 comp rx=0 tx=0).stop 2024-03-14T12:14:03.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c415bce0 msgr2=0x7f93c414e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:03.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c415bce0 0x7f93c414e120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f93bc037040 tx=0x7f93bc004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:03.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 shutdown_connections 2024-03-14T12:14:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c415bce0 0x7f93c414e120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c40b5e60 0x7f93c413ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93b006bbd0 0x7f93b006e090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 --2- 172.21.15.82:0/888176305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c40b1200 0x7f93c413cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 >> 172.21.15.82:0/888176305 conn(0x7f93c4072ec0 msgr2=0x7f93c40b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 shutdown_connections 2024-03-14T12:14:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:03.996+0000 7f93ca359640 1 -- 172.21.15.82:0/888176305 wait complete. 2024-03-14T12:14:04.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:04.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.107+0000 7fe0c7790640 1 Processor -- start 2024-03-14T12:14:04.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.107+0000 7fe0c7790640 1 -- start start 2024-03-14T12:14:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c7790640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c00b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c7790640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe0c00b5bb0 0x7fe0c0161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c7790640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe0c0161fc0 0x7fe0c015e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c7790640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00b6d00 con 0x7fe0c0161fc0 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c7790640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00b60f0 con 0x7fe0c00b5bb0 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c7790640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00b63d0 con 0x7fe0c01551d0 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe0c0161fc0 0x7fe0c015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe0c0161fc0 0x7fe0c015e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51264/0 (socket says 172.21.15.82:51264) 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/82553540 learned_addr learned my addr 172.21.15.82:0/82553540 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c552c640 1 --2- 172.21.15.82:0/82553540 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c00b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c4d2b640 1 --2- 172.21.15.82:0/82553540 >> v2:172.21.15.164:3300/0 conn(0x7fe0c00b5bb0 0x7fe0c0161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/82553540 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 msgr2=0x7fe0c00b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 --2- 172.21.15.82:0/82553540 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c00b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/82553540 >> v2:172.21.15.164:3300/0 conn(0x7fe0c00b5bb0 msgr2=0x7fe0c0161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 --2- 172.21.15.82:0/82553540 >> v2:172.21.15.164:3300/0 conn(0x7fe0c00b5bb0 0x7fe0c0161a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/82553540 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00b66b0 con 0x7fe0c0161fc0 2024-03-14T12:14:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.108+0000 7fe0c4d2b640 1 --2- 172.21.15.82:0/82553540 >> v2:172.21.15.164:3300/0 conn(0x7fe0c00b5bb0 0x7fe0c0161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.109+0000 7fe0c5d2d640 1 --2- 172.21.15.82:0/82553540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c015e490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00721a0 tx=0x7fe0b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2752182c60b48d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.109+0000 7fe0b7fff640 1 -- 172.21.15.82:0/82553540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b802ca70 con 0x7fe0c0161fc0 2024-03-14T12:14:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.109+0000 7fe0b7fff640 1 -- 172.21.15.82:0/82553540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b802cc10 con 0x7fe0c0161fc0 2024-03-14T12:14:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.109+0000 7fe0b7fff640 1 -- 172.21.15.82:0/82553540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80343f0 con 0x7fe0c0161fc0 2024-03-14T12:14:04.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 -- 172.21.15.82:0/82553540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 msgr2=0x7fe0c015e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 --2- 172.21.15.82:0/82553540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c015e490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00721a0 tx=0x7fe0b802bed0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 -- 172.21.15.82:0/82553540 shutdown_connections 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 --2- 172.21.15.82:0/82553540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c015e490 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 --2- 172.21.15.82:0/82553540 >> v2:172.21.15.164:3300/0 conn(0x7fe0c00b5bb0 0x7fe0c0161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 --2- 172.21.15.82:0/82553540 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c00b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 -- 172.21.15.82:0/82553540 >> 172.21.15.82:0/82553540 conn(0x7fe0c0072ee0 msgr2=0x7fe0c009a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 -- 172.21.15.82:0/82553540 shutdown_connections 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.110+0000 7fe0c7790640 1 -- 172.21.15.82:0/82553540 wait complete. 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.111+0000 7fe0c7790640 1 Processor -- start 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.111+0000 7fe0c7790640 1 -- start start 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c7790640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c00b5bb0 0x7fe0c014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c7790640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c01f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c7790640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c01f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c7790640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015eb50 con 0x7fe0c0161fc0 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c7790640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015e9d0 con 0x7fe0c00b5bb0 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c7790640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015ecd0 con 0x7fe0c01551d0 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c01f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c01f7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51278/0 (socket says 172.21.15.82:51278) 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/1934460375 learned_addr learned my addr 172.21.15.82:0/1934460375 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c4d2b640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c01f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c552c640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c00b5bb0 0x7fe0c014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 msgr2=0x7fe0c01f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c01f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c00b5bb0 msgr2=0x7fe0c014d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c00b5bb0 0x7fe0c014d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 -- 172.21.15.82:0/1934460375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c01e8f50 con 0x7fe0c0161fc0 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c4d2b640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c01f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:04.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.112+0000 7fe0c5d2d640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c01f7c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009f40 tx=0x7fe0b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.113+0000 7fe0b5ffb640 1 -- 172.21.15.82:0/1934460375 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8044020 con 0x7fe0c0161fc0 2024-03-14T12:14:04.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.113+0000 7fe0b5ffb640 1 -- 172.21.15.82:0/1934460375 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b8040670 con 0x7fe0c0161fc0 2024-03-14T12:14:04.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.113+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c01e86b0 con 0x7fe0c0161fc0 2024-03-14T12:14:04.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.113+0000 7fe0b5ffb640 1 -- 172.21.15.82:0/1934460375 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8009510 con 0x7fe0c0161fc0 2024-03-14T12:14:04.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.113+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c01e8c70 con 0x7fe0c0161fc0 2024-03-14T12:14:04.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.114+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c004f050 con 0x7fe0c0161fc0 2024-03-14T12:14:04.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.114+0000 7fe0b5ffb640 1 -- 172.21.15.82:0/1934460375 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe0b8047020 con 0x7fe0c0161fc0 2024-03-14T12:14:04.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.115+0000 7fe0b5ffb640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0a406bc00 0x7fe0a406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.115+0000 7fe0c552c640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0a406bc00 0x7fe0a406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.115+0000 7fe0b5ffb640 1 -- 172.21.15.82:0/1934460375 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0b80ad460 con 0x7fe0c0161fc0 2024-03-14T12:14:04.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.115+0000 7fe0c552c640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0a406bc00 0x7fe0a406e0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc006fd0 tx=0x7fe0bc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.119+0000 7fe0b5ffb640 1 -- 172.21.15.82:0/1934460375 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe0b804e030 con 0x7fe0c0161fc0 2024-03-14T12:14:04.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.327+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0c009b3a0 con 0x7fe0c0161fc0 2024-03-14T12:14:04.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.327+0000 7fe0b5ffb640 1 -- 172.21.15.82:0/1934460375 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fe0b807bf50 con 0x7fe0c0161fc0 2024-03-14T12:14:04.326 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:04.326 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:14:04.327 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:14:04.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0a406bc00 msgr2=0x7fe0a406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0a406bc00 0x7fe0a406e0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc006fd0 tx=0x7fe0bc008040 comp rx=0 tx=0).stop 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 msgr2=0x7fe0c01f7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c01f7c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009f40 tx=0x7fe0b8004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 shutdown_connections 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c0161fc0 0x7fe0c01f7c10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0a406bc00 0x7fe0a406e0c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c01551d0 0x7fe0c01f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 --2- 172.21.15.82:0/1934460375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c00b5bb0 0x7fe0c014d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 >> 172.21.15.82:0/1934460375 conn(0x7fe0c0072ee0 msgr2=0x7fe0c00aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 shutdown_connections 2024-03-14T12:14:04.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.329+0000 7fe0c7790640 1 -- 172.21.15.82:0/1934460375 wait complete. 2024-03-14T12:14:04.339 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:04.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.440+0000 7f6faaa01640 1 Processor -- start 2024-03-14T12:14:04.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.440+0000 7f6faaa01640 1 -- start start 2024-03-14T12:14:04.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6faaa01640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 0x7f6fa415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6faaa01640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa40d1590 con 0x7f6fa415df30 2024-03-14T12:14:04.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6faaa01640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa4161b70 con 0x7f6fa406f7f0 2024-03-14T12:14:04.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6faaa01640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4161e50 con 0x7f6fa406b420 2024-03-14T12:14:04.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6fa415df30 0x7f6fa406ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54232/0 (socket says 172.21.15.82:54232) 2024-03-14T12:14:04.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa3fff640 1 -- 172.21.15.82:0/2764896472 learned_addr learned my addr 172.21.15.82:0/2764896472 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa8f9e640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 0x7f6fa415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163498604 0 0) 0x7f6fa40d1590 con 0x7f6fa415df30 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f8c003520 con 0x7f6fa415df30 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 914773845 0 0) 0x7f6f8c003520 con 0x7f6fa415df30 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 msgr2=0x7f6fa415ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 0x7f6fa415ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 >> v1:172.21.15.164:6789/0 conn(0x7f6fa406f7f0 legacy=0x7f6fa415e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4162130 con 0x7f6fa415df30 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6f94001670 con 0x7f6fa415df30 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 >> v1:172.21.15.82:6789/0 conn(0x7f6fa415df30 legacy=0x7f6fa406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f8c0039e0 0x7f6f8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.441+0000 7f6fa2ffd640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 0x7f6f8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa2ffd640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f8c004d40 0x7f6f8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4161e50 con 0x7f6f8c004d40 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c00baa0 con 0x7f6f8c0039e0 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c00bca0 con 0x7f6fa406b420 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa8f9e640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f8c004d40 0x7f6f8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa3fff640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f8c0039e0 0x7f6f8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa37fe640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 0x7f6f8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa8f9e640 1 -- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 msgr2=0x7f6f8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa8f9e640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa406b420 0x7f6f8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa8f9e640 1 -- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f8c0039e0 msgr2=0x7f6f8c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa8f9e640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f8c0039e0 0x7f6f8c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.442+0000 7f6fa8f9e640 1 -- 172.21.15.82:0/2764896472 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c003520 con 0x7f6f8c004d40 2024-03-14T12:14:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6fa8f9e640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f8c004d40 0x7f6f8c0051c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4071430 tx=0x7f6f8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede6003a769dc692 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c039070 con 0x7f6f8c004d40 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c02cb90 con 0x7f6f8c004d40 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6fa2ffd640 1 -- 172.21.15.82:0/2764896472 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c009c10 con 0x7f6f8c004d40 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6faaa01640 1 -- 172.21.15.82:0/2764896472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f8c004d40 msgr2=0x7f6f8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6faaa01640 1 --2- 172.21.15.82:0/2764896472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f8c004d40 0x7f6f8c0051c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4071430 tx=0x7f6f8c005830 comp rx=0 tx=0).stop 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6fa8f9e640 1 -- 172.21.15.82:0/2764896472 reap_dead start 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6faaa01640 1 -- 172.21.15.82:0/2764896472 shutdown_connections 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6faaa01640 1 -- 172.21.15.82:0/2764896472 >> 172.21.15.82:0/2764896472 conn(0x7f6fa40d09d0 msgr2=0x7f6fa40d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6faaa01640 1 -- 172.21.15.82:0/2764896472 shutdown_connections 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.443+0000 7f6faaa01640 1 -- 172.21.15.82:0/2764896472 wait complete. 2024-03-14T12:14:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.444+0000 7f6faaa01640 1 Processor -- start 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6faaa01640 1 -- start start 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6faaa01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa413fd70 0x7f6fa41585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6faaa01640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa4140150 0x7f6fa4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6faaa01640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40ad850 0x7f6fa41e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6faaa01640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4162650 con 0x7f6fa413fd70 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6faaa01640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa40b3c10 con 0x7f6fa4140150 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6faaa01640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa41627d0 con 0x7f6fa40ad850 2024-03-14T12:14:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa413fd70 0x7f6fa41585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa413fd70 0x7f6fa41585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51292/0 (socket says 172.21.15.82:51292) 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.445+0000 7f6fa3fff640 1 -- 172.21.15.82:0/3263727141 learned_addr learned my addr 172.21.15.82:0/3263727141 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa37fe640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa4140150 0x7f6fa4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa8f9e640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40ad850 0x7f6fa41e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa3fff640 1 -- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40ad850 msgr2=0x7f6fa41e77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa3fff640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40ad850 0x7f6fa41e77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa3fff640 1 -- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa4140150 msgr2=0x7f6fa4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa3fff640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa4140150 0x7f6fa4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa3fff640 1 -- 172.21.15.82:0/3263727141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa41579b0 con 0x7f6fa413fd70 2024-03-14T12:14:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa8f9e640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40ad850 0x7f6fa41e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa3fff640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa413fd70 0x7f6fa41585f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f94007320 tx=0x7f6f94007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa0ff9640 1 -- 172.21.15.82:0/3263727141 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9401a070 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa0ff9640 1 -- 172.21.15.82:0/3263727141 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9400f9a0 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6fa0ff9640 1 -- 172.21.15.82:0/3263727141 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400fd70 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa41408d0 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.446+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa4140e90 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.447+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa404f050 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.447+0000 7f6fa0ff9640 1 -- 172.21.15.82:0/3263727141 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6f9401d070 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.448+0000 7f6fa0ff9640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f9006bbd0 0x7f6f9006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.448+0000 7f6fa37fe640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f9006bbd0 0x7f6f9006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.448+0000 7f6fa0ff9640 1 -- 172.21.15.82:0/3263727141 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f9408a5d0 con 0x7f6fa413fd70 2024-03-14T12:14:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.448+0000 7f6fa37fe640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f9006bbd0 0x7f6f9006e090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6fa406cd40 tx=0x7f6f98009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.452+0000 7f6fa0ff9640 1 -- 172.21.15.82:0/3263727141 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6f94059030 con 0x7f6fa413fd70 2024-03-14T12:14:04.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.626+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fa40ced50 con 0x7f6fa413fd70 2024-03-14T12:14:04.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.627+0000 7f6fa0ff9640 1 -- 172.21.15.82:0/3263727141 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f6f94058930 con 0x7f6fa413fd70 2024-03-14T12:14:04.626 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:04.626 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:04.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f9006bbd0 msgr2=0x7f6f9006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f9006bbd0 0x7f6f9006e090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6fa406cd40 tx=0x7f6f98009040 comp rx=0 tx=0).stop 2024-03-14T12:14:04.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa413fd70 msgr2=0x7f6fa41585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa413fd70 0x7f6fa41585f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f94007320 tx=0x7f6f94007720 comp rx=0 tx=0).stop 2024-03-14T12:14:04.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 shutdown_connections 2024-03-14T12:14:04.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f9006bbd0 0x7f6f9006e090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fa40ad850 0x7f6fa41e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fa4140150 0x7f6fa4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 --2- 172.21.15.82:0/3263727141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fa413fd70 0x7f6fa41585f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 >> 172.21.15.82:0/3263727141 conn(0x7f6fa40d09d0 msgr2=0x7f6fa40b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:04.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 shutdown_connections 2024-03-14T12:14:04.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.629+0000 7f6faaa01640 1 -- 172.21.15.82:0/3263727141 wait complete. 2024-03-14T12:14:04.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:04.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:04.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.743+0000 7f02795e1640 1 Processor -- start 2024-03-14T12:14:04.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.743+0000 7f02795e1640 1 -- start start 2024-03-14T12:14:04.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02795e1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f02740d0e10 0x7f0274168c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02795e1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740d25a0 con 0x7f027415dff0 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02795e1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740d2880 con 0x7f02740d12a0 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02795e1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02740d2350 con 0x7f02740d0e10 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f0272ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f027415dff0 0x7f02740d4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54248/0 (socket says 172.21.15.82:54248) 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f0272ffd640 1 -- 172.21.15.82:0/823365127 learned_addr learned my addr 172.21.15.82:0/823365127 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02737fe640 1 --2- 172.21.15.82:0/823365127 >> v2:172.21.15.82:3300/0 conn(0x7f02740d0e10 0x7f0274168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02737fe640 1 -- 172.21.15.82:0/823365127 >> v1:172.21.15.164:6789/0 conn(0x7f02740d12a0 legacy=0x7f02740d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02737fe640 1 -- 172.21.15.82:0/823365127 >> v1:172.21.15.82:6789/0 conn(0x7f027415dff0 legacy=0x7f02740d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f0271ffb640 1 -- 172.21.15.82:0/823365127 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008649880 0 0) 0x7f02740d25a0 con 0x7f027415dff0 2024-03-14T12:14:04.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.744+0000 7f02737fe640 1 -- 172.21.15.82:0/823365127 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740d2b60 con 0x7f02740d0e10 2024-03-14T12:14:04.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.745+0000 7f02737fe640 1 --2- 172.21.15.82:0/823365127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f0274168c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02740d24d0 tx=0x7f026c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb09b91b4ac8490 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.745+0000 7f0271ffb640 1 -- 172.21.15.82:0/823365127 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c039070 con 0x7f02740d0e10 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.745+0000 7f0271ffb640 1 -- 172.21.15.82:0/823365127 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c02cce0 con 0x7f02740d0e10 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.745+0000 7f0271ffb640 1 -- 172.21.15.82:0/823365127 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c034660 con 0x7f02740d0e10 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.746+0000 7f02795e1640 1 -- 172.21.15.82:0/823365127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 msgr2=0x7f0274168c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.746+0000 7f02795e1640 1 --2- 172.21.15.82:0/823365127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f0274168c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02740d24d0 tx=0x7f026c02bf30 comp rx=0 tx=0).stop 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.746+0000 7f02795e1640 1 -- 172.21.15.82:0/823365127 shutdown_connections 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.746+0000 7f02795e1640 1 --2- 172.21.15.82:0/823365127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f0274168c90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.746+0000 7f02795e1640 1 -- 172.21.15.82:0/823365127 >> 172.21.15.82:0/823365127 conn(0x7f027409d0d0 msgr2=0x7f02740d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.746+0000 7f02795e1640 1 -- 172.21.15.82:0/823365127 shutdown_connections 2024-03-14T12:14:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.746+0000 7f02795e1640 1 -- 172.21.15.82:0/823365127 wait complete. 2024-03-14T12:14:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.747+0000 7f02795e1640 1 Processor -- start 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.747+0000 7f02795e1640 1 -- start start 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02795e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f027415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02795e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02740d12a0 0x7f02740b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02795e1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027415dff0 0x7f02740c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02795e1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740cc4b0 con 0x7f02740d0e10 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02795e1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740cc330 con 0x7f02740d12a0 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02795e1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02740cc630 con 0x7f027415dff0 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f027415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f027415a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51308/0 (socket says 172.21.15.82:51308) 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 -- 172.21.15.82:0/3778134365 learned_addr learned my addr 172.21.15.82:0/3778134365 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02727fc640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02740d12a0 0x7f02740b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02737fe640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027415dff0 0x7f02740c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 -- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027415dff0 msgr2=0x7f02740c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027415dff0 0x7f02740c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 -- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02740d12a0 msgr2=0x7f02740b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02740d12a0 0x7f02740b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 -- 172.21.15.82:0/3778134365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02741f7070 con 0x7f02740d0e10 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f0272ffd640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f027415a5e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f026400c420 tx=0x7f026400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f025b7fe640 1 -- 172.21.15.82:0/3778134365 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640074e0 con 0x7f02740d0e10 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f025b7fe640 1 -- 172.21.15.82:0/3778134365 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640047e0 con 0x7f02740d0e10 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f025b7fe640 1 -- 172.21.15.82:0/3778134365 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264004c10 con 0x7f02740d0e10 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.748+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02741f7350 con 0x7f02740d0e10 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.749+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741f7910 con 0x7f02740d0e10 2024-03-14T12:14:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.749+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02740685f0 con 0x7f02740d0e10 2024-03-14T12:14:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.750+0000 7f025b7fe640 1 -- 172.21.15.82:0/3778134365 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0264011070 con 0x7f02740d0e10 2024-03-14T12:14:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.750+0000 7f025b7fe640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026006bbd0 0x7f026006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.750+0000 7f025b7fe640 1 -- 172.21.15.82:0/3778134365 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f026408a6a0 con 0x7f02740d0e10 2024-03-14T12:14:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.750+0000 7f02727fc640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026006bbd0 0x7f026006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.751+0000 7f02727fc640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026006bbd0 0x7f026006e090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0268000f60 tx=0x7f0268009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.755+0000 7f025b7fe640 1 -- 172.21.15.82:0/3778134365 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f02740685f0 con 0x7f02740d0e10 2024-03-14T12:14:04.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.925+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f027404f050 con 0x7f02740d0e10 2024-03-14T12:14:04.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.926+0000 7f025b7fe640 1 -- 172.21.15.82:0/3778134365 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f026400e280 con 0x7f02740d0e10 2024-03-14T12:14:04.925 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:04.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026006bbd0 msgr2=0x7f026006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026006bbd0 0x7f026006e090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0268000f60 tx=0x7f0268009040 comp rx=0 tx=0).stop 2024-03-14T12:14:04.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 msgr2=0x7f027415a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:04.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f027415a5e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f026400c420 tx=0x7f026400c8e0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 shutdown_connections 2024-03-14T12:14:04.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026006bbd0 0x7f026006e090 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027415dff0 0x7f02740c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02740d12a0 0x7f02740b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 --2- 172.21.15.82:0/3778134365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02740d0e10 0x7f027415a5e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:04.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 >> 172.21.15.82:0/3778134365 conn(0x7f027409d0d0 msgr2=0x7f027406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:04.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 shutdown_connections 2024-03-14T12:14:04.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:04.928+0000 7f02795e1640 1 -- 172.21.15.82:0/3778134365 wait complete. 2024-03-14T12:14:04.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.042+0000 7f63052a7640 1 Processor -- start 2024-03-14T12:14:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.042+0000 7f63052a7640 1 -- start start 2024-03-14T12:14:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f63052a7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f630006f390 0x7f63000b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f63052a7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f630006b420 0x7f630006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f63052a7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6300162a30 con 0x7f63001477c0 2024-03-14T12:14:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f63052a7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630006f770 con 0x7f630006f390 2024-03-14T12:14:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f63052a7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63001470a0 con 0x7f630006b420 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f62feffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f630006f390 0x7f63000b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f62feffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f630006f390 0x7f63000b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51318/0 (socket says 172.21.15.82:51318) 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.043+0000 7f62feffd640 1 -- 172.21.15.82:0/2678183572 learned_addr learned my addr 172.21.15.82:0/2678183572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.044+0000 7f62feffd640 1 -- 172.21.15.82:0/2678183572 >> v2:172.21.15.164:3300/0 conn(0x7f630006b420 msgr2=0x7f630006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.044+0000 7f62feffd640 1 --2- 172.21.15.82:0/2678183572 >> v2:172.21.15.164:3300/0 conn(0x7f630006b420 0x7f630006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.044+0000 7f62feffd640 1 -- 172.21.15.82:0/2678183572 >> v1:172.21.15.164:6789/0 conn(0x7f63001477c0 legacy=0x7f630006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.044+0000 7f62feffd640 1 -- 172.21.15.82:0/2678183572 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6300161c70 con 0x7f630006f390 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.044+0000 7f62feffd640 1 --2- 172.21.15.82:0/2678183572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f630006f390 0x7f63000b5a30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6300147220 tx=0x7f62f002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257849c7e28713e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.044+0000 7f62fdffb640 1 -- 172.21.15.82:0/2678183572 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f002c9e0 con 0x7f630006f390 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f62fdffb640 1 -- 172.21.15.82:0/2678183572 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f002cb80 con 0x7f630006f390 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f62fdffb640 1 -- 172.21.15.82:0/2678183572 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f00343f0 con 0x7f630006f390 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 -- 172.21.15.82:0/2678183572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f630006f390 msgr2=0x7f63000b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 --2- 172.21.15.82:0/2678183572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f630006f390 0x7f63000b5a30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6300147220 tx=0x7f62f002be40 comp rx=0 tx=0).stop 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 -- 172.21.15.82:0/2678183572 shutdown_connections 2024-03-14T12:14:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 --2- 172.21.15.82:0/2678183572 >> v2:172.21.15.164:3300/0 conn(0x7f630006b420 0x7f630006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 --2- 172.21.15.82:0/2678183572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f630006f390 0x7f63000b5a30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 -- 172.21.15.82:0/2678183572 >> 172.21.15.82:0/2678183572 conn(0x7f63000d0b40 msgr2=0x7f63000d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 -- 172.21.15.82:0/2678183572 shutdown_connections 2024-03-14T12:14:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.045+0000 7f63052a7640 1 -- 172.21.15.82:0/2678183572 wait complete. 2024-03-14T12:14:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.046+0000 7f63052a7640 1 Processor -- start 2024-03-14T12:14:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.046+0000 7f63052a7640 1 -- start start 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f63052a7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f630006b420 0x7f63001f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f63052a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006f390 0x7f6300160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f63052a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63001477c0 0x7f63000ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f63052a7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6300162230 con 0x7f63001477c0 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f63052a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001475e0 con 0x7f630006f390 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f63052a7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6300162510 con 0x7f630006b420 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63001477c0 0x7f63000ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63001477c0 0x7f63000ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51320/0 (socket says 172.21.15.82:51320) 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f62ff7fe640 1 -- 172.21.15.82:0/1370216854 learned_addr learned my addr 172.21.15.82:0/1370216854 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f62feffd640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f630006b420 0x7f63001f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.047+0000 7f62fe7fc640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006f390 0x7f6300160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62ff7fe640 1 -- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f630006b420 msgr2=0x7f63001f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62ff7fe640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f630006b420 0x7f63001f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62ff7fe640 1 -- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006f390 msgr2=0x7f6300160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62ff7fe640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006f390 0x7f6300160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62ff7fe640 1 -- 172.21.15.82:0/1370216854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63001f4b40 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62ff7fe640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63001477c0 0x7f63000ae9b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009fd0 tx=0x7f62f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62e77fe640 1 -- 172.21.15.82:0/1370216854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8011830 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62e77fe640 1 -- 172.21.15.82:0/1370216854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8016070 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f62e77fe640 1 -- 172.21.15.82:0/1370216854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8011bf0 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63000cdeb0 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.048+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63000ce470 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.049+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63000b4a50 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.049+0000 7f62e77fe640 1 -- 172.21.15.82:0/1370216854 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f62f801b460 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.050+0000 7f62e77fe640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f62ec06bbd0 0x7f62ec06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.050+0000 7f62e77fe640 1 -- 172.21.15.82:0/1370216854 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f808a770 con 0x7f63001477c0 2024-03-14T12:14:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.050+0000 7f62feffd640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f62ec06bbd0 0x7f62ec06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.050+0000 7f62feffd640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f62ec06bbd0 0x7f62ec06e090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62f0035000 tx=0x7f62f0030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.054+0000 7f62e77fe640 1 -- 172.21.15.82:0/1370216854 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f63000b4a50 con 0x7f63001477c0 2024-03-14T12:14:05.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.224+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f630004f050 con 0x7f63001477c0 2024-03-14T12:14:05.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.224+0000 7f62e77fe640 1 -- 172.21.15.82:0/1370216854 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62f80592f0 con 0x7f63001477c0 2024-03-14T12:14:05.223 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:05.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f62ec06bbd0 msgr2=0x7f62ec06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f62ec06bbd0 0x7f62ec06e090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62f0035000 tx=0x7f62f0030000 comp rx=0 tx=0).stop 2024-03-14T12:14:05.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63001477c0 msgr2=0x7f63000ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63001477c0 0x7f63000ae9b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009fd0 tx=0x7f62f800b8e0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 shutdown_connections 2024-03-14T12:14:05.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63001477c0 0x7f63000ae9b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f630006f390 0x7f6300160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f62ec06bbd0 0x7f62ec06e090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 --2- 172.21.15.82:0/1370216854 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f630006b420 0x7f63001f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 >> 172.21.15.82:0/1370216854 conn(0x7f63000d0b40 msgr2=0x7f6300145e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:05.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 shutdown_connections 2024-03-14T12:14:05.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.226+0000 7f63052a7640 1 -- 172.21.15.82:0/1370216854 wait complete. 2024-03-14T12:14:05.236 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:05.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.340+0000 7ff33e412640 1 Processor -- start 2024-03-14T12:14:05.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.340+0000 7ff33e412640 1 -- start start 2024-03-14T12:14:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff33e412640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff33815dec0 0x7ff3380d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff33e412640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 0x7ff3380d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff33e412640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3380d1420 con 0x7ff3380ca540 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff33e412640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3380d1700 con 0x7ff33815dec0 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff33e412640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380c80a0 con 0x7ff33815e2a0 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff3377fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff3380ca540 0x7ff3380d2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54254/0 (socket says 172.21.15.82:54254) 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff3377fe640 1 -- 172.21.15.82:0/772394565 learned_addr learned my addr 172.21.15.82:0/772394565 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff33c9af640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 0x7ff3380d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164956850 0 0) 0x7ff3380d1420 con 0x7ff3380ca540 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff320003520 con 0x7ff3380ca540 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff337fff640 1 --2- 172.21.15.82:0/772394565 >> v2:172.21.15.164:3300/0 conn(0x7ff33815dec0 0x7ff3380d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2140278690 0 0) 0x7ff320003520 con 0x7ff3380ca540 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 msgr2=0x7ff3380d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 0x7ff3380d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 >> v2:172.21.15.164:3300/0 conn(0x7ff33815dec0 msgr2=0x7ff3380d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 --2- 172.21.15.82:0/772394565 >> v2:172.21.15.164:3300/0 conn(0x7ff33815dec0 0x7ff3380d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3380c7a50 con 0x7ff3380ca540 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7ff32c001670 con 0x7ff3380ca540 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 >> v1:172.21.15.82:6789/0 conn(0x7ff3380ca540 legacy=0x7ff3380d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 0x7ff3200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff320004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff320004d40 0x7ff3200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380d1700 con 0x7ff320003f30 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380c80a0 con 0x7ff320004d40 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32000ba10 con 0x7ff33815e2a0 2024-03-14T12:14:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.341+0000 7ff3377fe640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff320004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff337fff640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 0x7ff3200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff33c9af640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff320004d40 0x7ff3200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff3377fe640 1 -- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 msgr2=0x7ff3200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff3377fe640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff33815e2a0 0x7ff3200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff3377fe640 1 -- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff320004d40 msgr2=0x7ff3200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff3377fe640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff320004d40 0x7ff3200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff3377fe640 1 -- 172.21.15.82:0/772394565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320003520 con 0x7ff320003f30 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.342+0000 7ff3377fe640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff320004800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3380ca920 tx=0x7ff320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bd0e5c0f008cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c007d70 con 0x7ff320003f30 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c030070 con 0x7ff320003f30 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff336ffd640 1 -- 172.21.15.82:0/772394565 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c005d30 con 0x7ff320003f30 2024-03-14T12:14:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff33e412640 1 -- 172.21.15.82:0/772394565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 msgr2=0x7ff320004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff33e412640 1 --2- 172.21.15.82:0/772394565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff320004800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3380ca920 tx=0x7ff320005830 comp rx=0 tx=0).stop 2024-03-14T12:14:05.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff33c9af640 1 -- 172.21.15.82:0/772394565 reap_dead start 2024-03-14T12:14:05.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff33e412640 1 -- 172.21.15.82:0/772394565 shutdown_connections 2024-03-14T12:14:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff33e412640 1 -- 172.21.15.82:0/772394565 >> 172.21.15.82:0/772394565 conn(0x7ff3380d3dd0 msgr2=0x7ff3380d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff33e412640 1 -- 172.21.15.82:0/772394565 shutdown_connections 2024-03-14T12:14:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.343+0000 7ff33e412640 1 -- 172.21.15.82:0/772394565 wait complete. 2024-03-14T12:14:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.344+0000 7ff33e412640 1 Processor -- start 2024-03-14T12:14:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.344+0000 7ff33e412640 1 -- start start 2024-03-14T12:14:05.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff33e412640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff3381f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff33e412640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff33815c380 0x7ff3380b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff33e412640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380b8f40 0x7ff3380cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff33e412640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380d1d30 con 0x7ff320003f30 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff33e412640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380d1bb0 con 0x7ff33815c380 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff33e412640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380d1eb0 con 0x7ff3380b8f40 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff337fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff3381f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff337fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff3381f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51332/0 (socket says 172.21.15.82:51332) 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.345+0000 7ff337fff640 1 -- 172.21.15.82:0/3992774007 learned_addr learned my addr 172.21.15.82:0/3992774007 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff33c9af640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380b8f40 0x7ff3380cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff337fff640 1 -- 172.21.15.82:0/3992774007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380b8f40 msgr2=0x7ff3380cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff337fff640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380b8f40 0x7ff3380cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff337fff640 1 -- 172.21.15.82:0/3992774007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff33815c380 msgr2=0x7ff3380b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff337fff640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff33815c380 0x7ff3380b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff337fff640 1 -- 172.21.15.82:0/3992774007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338160bb0 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff337fff640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff3381f4550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff328011a00 tx=0x7ff328011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff334ff9640 1 -- 172.21.15.82:0/3992774007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800fb50 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff334ff9640 1 -- 172.21.15.82:0/3992774007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32800fcf0 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3380ccdb0 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff334ff9640 1 -- 172.21.15.82:0/3992774007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800fea0 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.346+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3381f3a70 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.347+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33804f050 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.347+0000 7ff334ff9640 1 -- 172.21.15.82:0/3992774007 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff328012040 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.348+0000 7ff334ff9640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff32406bc00 0x7ff32406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.348+0000 7ff334ff9640 1 -- 172.21.15.82:0/3992774007 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff32808b050 con 0x7ff320003f30 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.348+0000 7ff3377fe640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff32406bc00 0x7ff32406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.348+0000 7ff3377fe640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff32406bc00 0x7ff32406e0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff32c034000 tx=0x7ff32c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.352+0000 7ff334ff9640 1 -- 172.21.15.82:0/3992774007 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff338160bb0 con 0x7ff320003f30 2024-03-14T12:14:05.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.523+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3380b9320 con 0x7ff320003f30 2024-03-14T12:14:05.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.524+0000 7ff334ff9640 1 -- 172.21.15.82:0/3992774007 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff328019030 con 0x7ff320003f30 2024-03-14T12:14:05.523 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:05.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff32406bc00 msgr2=0x7ff32406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff32406bc00 0x7ff32406e0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff32c034000 tx=0x7ff32c02e000 comp rx=0 tx=0).stop 2024-03-14T12:14:05.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 msgr2=0x7ff3381f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff3381f4550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff328011a00 tx=0x7ff328011ec0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 shutdown_connections 2024-03-14T12:14:05.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff32406bc00 0x7ff32406e0c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380b8f40 0x7ff3380cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff33815c380 0x7ff3380b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 --2- 172.21.15.82:0/3992774007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff320003f30 0x7ff3381f4550 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 >> 172.21.15.82:0/3992774007 conn(0x7ff3380d3dd0 msgr2=0x7ff33806cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:05.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 shutdown_connections 2024-03-14T12:14:05.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.526+0000 7ff33e412640 1 -- 172.21.15.82:0/3992774007 wait complete. 2024-03-14T12:14:05.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:05.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.639+0000 7f5c3c8c0640 1 Processor -- start 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.639+0000 7f5c3c8c0640 1 -- start start 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c3c8c0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3806f7f0 0x7f5c3815e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c3c8c0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5c3806b420 0x7f5c3815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c3c8c0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380d1590 con 0x7f5c3815df30 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c3c8c0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c38161b70 con 0x7f5c3806b420 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c3c8c0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38161e50 con 0x7f5c3806f7f0 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c3659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5c3815df30 0x7f5c3806ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54258/0 (socket says 172.21.15.82:54258) 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c3659c640 1 -- 172.21.15.82:0/3115838669 learned_addr learned my addr 172.21.15.82:0/3115838669 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3806f7f0 0x7f5c3815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c36d9d640 1 --2- 172.21.15.82:0/3115838669 >> v2:172.21.15.164:3300/0 conn(0x7f5c3806b420 0x7f5c3815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779536676 0 0) 0x7f5c380d1590 con 0x7f5c3815df30 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c24003520 con 0x7f5c3815df30 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609685943 0 0) 0x7f5c24003520 con 0x7f5c3815df30 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3806f7f0 msgr2=0x7f5c3815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3806f7f0 0x7f5c3815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 >> v2:172.21.15.164:3300/0 conn(0x7f5c3806b420 msgr2=0x7f5c3815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 --2- 172.21.15.82:0/3115838669 >> v2:172.21.15.164:3300/0 conn(0x7f5c3806b420 0x7f5c3815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38162130 con 0x7f5c3815df30 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f5c28001670 con 0x7f5c3815df30 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 >> v1:172.21.15.82:6789/0 conn(0x7f5c3815df30 legacy=0x7f5c3806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c240039b0 0x7f5c24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c24004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c24004ef0 0x7f5c2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38161b70 con 0x7f5c240043a0 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38161e50 con 0x7f5c24004ef0 2024-03-14T12:14:05.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2400bf00 con 0x7f5c240039b0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.640+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3806f7f0 0x7f5c3815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c24004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c36d9d640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c24004ef0 0x7f5c2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c3659c640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c240039b0 0x7f5c24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c2ffff640 1 -- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c240039b0 msgr2=0x7f5c24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c240039b0 0x7f5c24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c2ffff640 1 -- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c24004ef0 msgr2=0x7f5c2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c24004ef0 0x7f5c2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.641+0000 7f5c2ffff640 1 -- 172.21.15.82:0/3115838669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24003520 con 0x7f5c240043a0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c24004880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c380b5e90 tx=0x7f5c24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ab77716db0f7bb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200097d0 con 0x7f5c240043a0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c20004030 con 0x7f5c240043a0 2024-03-14T12:14:05.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c35d9b640 1 -- 172.21.15.82:0/3115838669 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20004400 con 0x7f5c240043a0 2024-03-14T12:14:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/3115838669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 msgr2=0x7f5c24004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c3c8c0640 1 --2- 172.21.15.82:0/3115838669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c24004880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c380b5e90 tx=0x7f5c24005830 comp rx=0 tx=0).stop 2024-03-14T12:14:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c36d9d640 1 -- 172.21.15.82:0/3115838669 reap_dead start 2024-03-14T12:14:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/3115838669 shutdown_connections 2024-03-14T12:14:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/3115838669 >> 172.21.15.82:0/3115838669 conn(0x7f5c380d09d0 msgr2=0x7f5c380d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/3115838669 shutdown_connections 2024-03-14T12:14:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.642+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/3115838669 wait complete. 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 Processor -- start 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 -- start start 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c381574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380af870 0x7f5c381493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3815c6a0 0x7f5c3815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38162650 con 0x7f5c240043a0 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380b3bd0 con 0x7f5c380af870 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3c8c0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381627d0 con 0x7f5c3815c6a0 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c381574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c381574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51354/0 (socket says 172.21.15.82:51354) 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.644+0000 7f5c3659c640 1 -- 172.21.15.82:0/757301333 learned_addr learned my addr 172.21.15.82:0/757301333 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c36d9d640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3815c6a0 0x7f5c3815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380af870 0x7f5c381493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3659c640 1 -- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3815c6a0 msgr2=0x7f5c3815cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3659c640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3815c6a0 0x7f5c3815cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3659c640 1 -- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380af870 msgr2=0x7f5c381493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3659c640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380af870 0x7f5c381493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3659c640 1 -- 172.21.15.82:0/757301333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38157a20 con 0x7f5c240043a0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380af870 0x7f5c381493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3659c640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c381574e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004fa0 tx=0x7f5c280074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c2effd640 1 -- 172.21.15.82:0/757301333 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2801a070 con 0x7f5c240043a0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c2effd640 1 -- 172.21.15.82:0/757301333 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c28007740 con 0x7f5c240043a0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c38149930 con 0x7f5c240043a0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c2effd640 1 -- 172.21.15.82:0/757301333 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800fbd0 con 0x7f5c240043a0 2024-03-14T12:14:05.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.645+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c380aeab0 con 0x7f5c240043a0 2024-03-14T12:14:05.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.646+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3804f050 con 0x7f5c240043a0 2024-03-14T12:14:05.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.646+0000 7f5c2effd640 1 -- 172.21.15.82:0/757301333 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5c2801d070 con 0x7f5c240043a0 2024-03-14T12:14:05.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.647+0000 7f5c2effd640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5c1806bbd0 0x7f5c1806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.647+0000 7f5c2effd640 1 -- 172.21.15.82:0/757301333 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c2808a450 con 0x7f5c240043a0 2024-03-14T12:14:05.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.647+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5c1806bbd0 0x7f5c1806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.647+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5c1806bbd0 0x7f5c1806e090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c20035000 tx=0x7f5c20030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.651+0000 7f5c2effd640 1 -- 172.21.15.82:0/757301333 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5c28058eb0 con 0x7f5c240043a0 2024-03-14T12:14:05.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.861+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c380b5ae0 con 0x7f5c240043a0 2024-03-14T12:14:05.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.861+0000 7f5c2effd640 1 -- 172.21.15.82:0/757301333 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f5c280587b0 con 0x7f5c240043a0 2024-03-14T12:14:05.861 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:05.861 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:14:05.861 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.863+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5c1806bbd0 msgr2=0x7f5c1806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.863+0000 7f5c3c8c0640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5c1806bbd0 0x7f5c1806e090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c20035000 tx=0x7f5c20030000 comp rx=0 tx=0).stop 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.863+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 msgr2=0x7f5c381574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.863+0000 7f5c3c8c0640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c381574e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004fa0 tx=0x7f5c280074f0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.863+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 shutdown_connections 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.863+0000 7f5c3c8c0640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5c1806bbd0 0x7f5c1806e090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.864+0000 7f5c3c8c0640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c3815c6a0 0x7f5c3815cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.864+0000 7f5c3c8c0640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380af870 0x7f5c381493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.864+0000 7f5c3c8c0640 1 --2- 172.21.15.82:0/757301333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c240043a0 0x7f5c381574e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.864+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 >> 172.21.15.82:0/757301333 conn(0x7f5c380d09d0 msgr2=0x7f5c3806ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:05.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.864+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 shutdown_connections 2024-03-14T12:14:05.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.864+0000 7f5c3c8c0640 1 -- 172.21.15.82:0/757301333 wait complete. 2024-03-14T12:14:05.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:05.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.974+0000 7f37eb59e640 1 Processor -- start 2024-03-14T12:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.974+0000 7f37eb59e640 1 -- start start 2024-03-14T12:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37eb59e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37ec1582d0 0x7f37ec15eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37eb59e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37ec0b5540 0x7f37ec162070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37eb59e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec0b3910 con 0x7f37ec1625b0 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37eb59e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec161900 con 0x7f37ec0b5540 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37eb59e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec0b3bf0 con 0x7f37ec1582d0 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37e1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37ec0b5540 0x7f37ec162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37e1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37ec0b5540 0x7f37ec162070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51370/0 (socket says 172.21.15.82:51370) 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37e1d9b640 1 -- 172.21.15.82:0/1486851092 learned_addr learned my addr 172.21.15.82:0/1486851092 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.975+0000 7f37ea59c640 1 --2- 172.21.15.82:0/1486851092 >> v2:172.21.15.164:3300/0 conn(0x7f37ec1582d0 0x7f37ec15eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.976+0000 7f37e1d9b640 1 -- 172.21.15.82:0/1486851092 >> v2:172.21.15.164:3300/0 conn(0x7f37ec1582d0 msgr2=0x7f37ec15eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.976+0000 7f37e1d9b640 1 --2- 172.21.15.82:0/1486851092 >> v2:172.21.15.164:3300/0 conn(0x7f37ec1582d0 0x7f37ec15eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.976+0000 7f37e1d9b640 1 -- 172.21.15.82:0/1486851092 >> v1:172.21.15.82:6789/0 conn(0x7f37ec1625b0 legacy=0x7f37ec06d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.976+0000 7f37e1d9b640 1 -- 172.21.15.82:0/1486851092 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0b3ed0 con 0x7f37ec0b5540 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.976+0000 7f37e1d9b640 1 --2- 172.21.15.82:0/1486851092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec0b5540 0x7f37ec162070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0d1a00 tx=0x7f37d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672f73881bd7af66 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.976+0000 7f37e9d9b640 1 -- 172.21.15.82:0/1486851092 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4035020 con 0x7f37ec0b5540 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37e9d9b640 1 -- 172.21.15.82:0/1486851092 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d4031870 con 0x7f37ec0b5540 2024-03-14T12:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37e9d9b640 1 -- 172.21.15.82:0/1486851092 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4031c40 con 0x7f37ec0b5540 2024-03-14T12:14:05.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 -- 172.21.15.82:0/1486851092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec0b5540 msgr2=0x7f37ec162070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 --2- 172.21.15.82:0/1486851092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec0b5540 0x7f37ec162070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0d1a00 tx=0x7f37d402b910 comp rx=0 tx=0).stop 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 -- 172.21.15.82:0/1486851092 shutdown_connections 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 --2- 172.21.15.82:0/1486851092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec0b5540 0x7f37ec162070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 --2- 172.21.15.82:0/1486851092 >> v2:172.21.15.164:3300/0 conn(0x7f37ec1582d0 0x7f37ec15eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 -- 172.21.15.82:0/1486851092 >> 172.21.15.82:0/1486851092 conn(0x7f37ec0d0900 msgr2=0x7f37ec0d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 -- 172.21.15.82:0/1486851092 shutdown_connections 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.977+0000 7f37eb59e640 1 -- 172.21.15.82:0/1486851092 wait complete. 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 Processor -- start 2024-03-14T12:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 -- start start 2024-03-14T12:14:05.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ec0b5540 0x7f37ec157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec1582d0 0x7f37ec0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ec1e93b0 0x7f37ec1e9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec158830 con 0x7f37ec1582d0 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1586b0 con 0x7f37ec0b5540 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37eb59e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0b59c0 con 0x7f37ec1e93b0 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37e1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec1582d0 0x7f37ec0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37e1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec1582d0 0x7f37ec0afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51384/0 (socket says 172.21.15.82:51384) 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.979+0000 7f37e1d9b640 1 -- 172.21.15.82:0/3969027564 learned_addr learned my addr 172.21.15.82:0/3969027564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37ead9d640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ec1e93b0 0x7f37ec1e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e1d9b640 1 -- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ec1e93b0 msgr2=0x7f37ec1e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e1d9b640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ec1e93b0 0x7f37ec1e9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e1d9b640 1 -- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ec0b5540 msgr2=0x7f37ec157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37ea59c640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ec0b5540 0x7f37ec157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e1d9b640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ec0b5540 0x7f37ec157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e1d9b640 1 -- 172.21.15.82:0/3969027564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec1366f0 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e1d9b640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec1582d0 0x7f37ec0afc80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f37d4000c00 tx=0x7f37d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e37fe640 1 -- 172.21.15.82:0/3969027564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4035040 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e37fe640 1 -- 172.21.15.82:0/3969027564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d4030d00 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec0b05f0 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37e37fe640 1 -- 172.21.15.82:0/3969027564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40456a0 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.980+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec0b0bb0 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.981+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec075210 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.981+0000 7f37e37fe640 1 -- 172.21.15.82:0/3969027564 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f37d4038020 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.982+0000 7f37e37fe640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f37cc06bbd0 0x7f37cc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.982+0000 7f37ea59c640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f37cc06bbd0 0x7f37cc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.982+0000 7f37e37fe640 1 -- 172.21.15.82:0/3969027564 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37d40acd20 con 0x7f37ec1582d0 2024-03-14T12:14:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.982+0000 7f37ea59c640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f37cc06bbd0 0x7f37cc06e090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009a10 tx=0x7f37dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:05.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:05.986+0000 7f37e37fe640 1 -- 172.21.15.82:0/3969027564 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f37d407b8a0 con 0x7f37ec1582d0 2024-03-14T12:14:06.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.162+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37ec0d17e0 con 0x7f37ec1582d0 2024-03-14T12:14:06.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.163+0000 7f37e37fe640 1 -- 172.21.15.82:0/3969027564 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f37d407b1a0 con 0x7f37ec1582d0 2024-03-14T12:14:06.162 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:06.162 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:06.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f37cc06bbd0 msgr2=0x7f37cc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f37cc06bbd0 0x7f37cc06e090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009a10 tx=0x7f37dc008040 comp rx=0 tx=0).stop 2024-03-14T12:14:06.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec1582d0 msgr2=0x7f37ec0afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec1582d0 0x7f37ec0afc80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f37d4000c00 tx=0x7f37d40046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 shutdown_connections 2024-03-14T12:14:06.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f37cc06bbd0 0x7f37cc06e090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ec1e93b0 0x7f37ec1e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37ec1582d0 0x7f37ec0afc80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 --2- 172.21.15.82:0/3969027564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ec0b5540 0x7f37ec157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 >> 172.21.15.82:0/3969027564 conn(0x7f37ec0d0900 msgr2=0x7f37ec146050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:06.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 shutdown_connections 2024-03-14T12:14:06.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.165+0000 7f37eb59e640 1 -- 172.21.15.82:0/3969027564 wait complete. 2024-03-14T12:14:06.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:06.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.277+0000 7f6e5b4a1640 1 Processor -- start 2024-03-14T12:14:06.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.277+0000 7f6e5b4a1640 1 -- start start 2024-03-14T12:14:06.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5b4a1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6e541551d0 0x7f6e540b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:06.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5b4a1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e5406f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:06.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5b4a1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540b60f0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5b4a1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540b6d00 con 0x7f6e541551d0 2024-03-14T12:14:06.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5b4a1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540b63d0 con 0x7f6e54162190 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5923d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6e541551d0 0x7f6e540b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5923d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6e541551d0 0x7f6e540b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51388/0 (socket says 172.21.15.82:51388) 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5923d640 1 -- 172.21.15.82:0/298262603 learned_addr learned my addr 172.21.15.82:0/298262603 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e59a3e640 1 --2- 172.21.15.82:0/298262603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e5406f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5923d640 1 -- 172.21.15.82:0/298262603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 msgr2=0x7f6e5406f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5923d640 1 --2- 172.21.15.82:0/298262603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e5406f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5923d640 1 -- 172.21.15.82:0/298262603 >> v1:172.21.15.82:6789/0 conn(0x7f6e540b5bb0 legacy=0x7f6e54161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e5923d640 1 -- 172.21.15.82:0/298262603 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540b66b0 con 0x7f6e541551d0 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.278+0000 7f6e59a3e640 1 --2- 172.21.15.82:0/298262603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e5406f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.279+0000 7f6e5923d640 1 --2- 172.21.15.82:0/298262603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e541551d0 0x7f6e540b5670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009870 tx=0x7f6e5002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e160be5ba8a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.279+0000 7f6e4bfff640 1 -- 172.21.15.82:0/298262603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50034070 con 0x7f6e541551d0 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.279+0000 7f6e4bfff640 1 -- 172.21.15.82:0/298262603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e50030840 con 0x7f6e541551d0 2024-03-14T12:14:06.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.279+0000 7f6e4bfff640 1 -- 172.21.15.82:0/298262603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50030c10 con 0x7f6e541551d0 2024-03-14T12:14:06.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/298262603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e541551d0 msgr2=0x7f6e540b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/298262603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e541551d0 0x7f6e540b5670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009870 tx=0x7f6e5002b980 comp rx=0 tx=0).stop 2024-03-14T12:14:06.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/298262603 shutdown_connections 2024-03-14T12:14:06.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/298262603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e5406f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/298262603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e541551d0 0x7f6e540b5670 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/298262603 >> 172.21.15.82:0/298262603 conn(0x7f6e54072ee0 msgr2=0x7f6e5409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/298262603 shutdown_connections 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.280+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/298262603 wait complete. 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.281+0000 7f6e5b4a1640 1 Processor -- start 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.281+0000 7f6e5b4a1640 1 -- start start 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5b4a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e540b5bb0 0x7f6e54152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5b4a1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e541551d0 0x7f6e54153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5b4a1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e540b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5b4a1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5406f9f0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5b4a1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5406f870 con 0x7f6e541551d0 2024-03-14T12:14:06.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5b4a1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5406fb70 con 0x7f6e54162190 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e540b5bb0 0x7f6e54152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e540b5bb0 0x7f6e54152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51392/0 (socket says 172.21.15.82:51392) 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 -- 172.21.15.82:0/3613122046 learned_addr learned my addr 172.21.15.82:0/3613122046 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e59a3e640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e540b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e58a3c640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e541551d0 0x7f6e54153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 -- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 msgr2=0x7f6e540b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e540b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 -- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e541551d0 msgr2=0x7f6e54153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e541551d0 0x7f6e54153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 -- 172.21.15.82:0/3613122046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540cb2c0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e59a3e640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e540b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e58a3c640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e541551d0 0x7f6e54153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5923d640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e540b5bb0 0x7f6e54152c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000c00 tx=0x7f6e50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e49ffb640 1 -- 172.21.15.82:0/3613122046 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50048070 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e49ffb640 1 -- 172.21.15.82:0/3613122046 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e50030640 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e540cb5a0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.282+0000 7f6e49ffb640 1 -- 172.21.15.82:0/3613122046 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50034040 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.283+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e540cbb60 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.284+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5404f050 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.284+0000 7f6e49ffb640 1 -- 172.21.15.82:0/3613122046 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6e5004b070 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.284+0000 7f6e49ffb640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e3806bbd0 0x7f6e3806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.284+0000 7f6e49ffb640 1 -- 172.21.15.82:0/3613122046 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e500acbd0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.284+0000 7f6e58a3c640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e3806bbd0 0x7f6e3806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:06.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.285+0000 7f6e58a3c640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e3806bbd0 0x7f6e3806e090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e5415d9c0 tx=0x7f6e44009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:06.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.289+0000 7f6e49ffb640 1 -- 172.21.15.82:0/3613122046 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6e5007b6c0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.497+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e5409b3a0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.498+0000 7f6e49ffb640 1 -- 172.21.15.82:0/3613122046 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f6e5007afc0 con 0x7f6e540b5bb0 2024-03-14T12:14:06.497 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:06.497 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:14:06.498 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:14:06.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e3806bbd0 msgr2=0x7f6e3806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e3806bbd0 0x7f6e3806e090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e5415d9c0 tx=0x7f6e44009040 comp rx=0 tx=0).stop 2024-03-14T12:14:06.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e540b5bb0 msgr2=0x7f6e54152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:06.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e540b5bb0 0x7f6e54152c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000c00 tx=0x7f6e50004730 comp rx=0 tx=0).stop 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 shutdown_connections 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e3806bbd0 0x7f6e3806e090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e54162190 0x7f6e540b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e541551d0 0x7f6e54153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 --2- 172.21.15.82:0/3613122046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e540b5bb0 0x7f6e54152c50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 >> 172.21.15.82:0/3613122046 conn(0x7f6e54072ee0 msgr2=0x7f6e5406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 shutdown_connections 2024-03-14T12:14:06.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:06.500+0000 7f6e5b4a1640 1 -- 172.21.15.82:0/3613122046 wait complete. 2024-03-14T12:14:06.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T12:13:50.902698+0000', 'modified': '2024-03-14T12:13:53.462023+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6834/3117714765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 3117714765}, {'type': 'v1', 'addr': '172.21.15.164:6834', 'nonce': 3117714765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T12:14:06.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T12:14:06.914 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-14T12:14:07.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.017+0000 7f21c9380640 1 Processor -- start 2024-03-14T12:14:07.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.017+0000 7f21c9380640 1 -- start start 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c9380640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f21c40af150 0x7f21c40af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c9380640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c9380640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21c40b58f0 0x7f21c406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c9380640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40b01d0 con 0x7f21c40b58f0 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c9380640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c415e710 con 0x7f21c40af150 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c9380640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c415e9f0 con 0x7f21c415f1f0 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21c40b58f0 0x7f21c406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21c40b58f0 0x7f21c406b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51394/0 (socket says 172.21.15.82:51394) 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c37fe640 1 -- 172.21.15.82:0/1581172531 learned_addr learned my addr 172.21.15.82:0/1581172531 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:07.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c27fc640 1 --2- 172.21.15.82:0/1581172531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.018+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/1581172531 >> v2:172.21.15.164:3300/0 conn(0x7f21c40af150 0x7f21c40af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c37fe640 1 -- 172.21.15.82:0/1581172531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 msgr2=0x7f21c406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1581172531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c37fe640 1 -- 172.21.15.82:0/1581172531 >> v2:172.21.15.164:3300/0 conn(0x7f21c40af150 msgr2=0x7f21c40af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1581172531 >> v2:172.21.15.164:3300/0 conn(0x7f21c40af150 0x7f21c40af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c37fe640 1 -- 172.21.15.82:0/1581172531 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c415ecd0 con 0x7f21c40b58f0 2024-03-14T12:14:07.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1581172531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c406b250 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21c404f5e0 tx=0x7f21bc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9605b9afcbaf224e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c1ffb640 1 -- 172.21.15.82:0/1581172531 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc02e070 con 0x7f21c40b58f0 2024-03-14T12:14:07.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c1ffb640 1 -- 172.21.15.82:0/1581172531 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc037070 con 0x7f21c40b58f0 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.019+0000 7f21c1ffb640 1 -- 172.21.15.82:0/1581172531 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc031a00 con 0x7f21c40b58f0 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 -- 172.21.15.82:0/1581172531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 msgr2=0x7f21c406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 --2- 172.21.15.82:0/1581172531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c406b250 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21c404f5e0 tx=0x7f21bc02b880 comp rx=0 tx=0).stop 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 -- 172.21.15.82:0/1581172531 shutdown_connections 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 --2- 172.21.15.82:0/1581172531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c406b250 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 --2- 172.21.15.82:0/1581172531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c406ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 --2- 172.21.15.82:0/1581172531 >> v2:172.21.15.164:3300/0 conn(0x7f21c40af150 0x7f21c40af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 -- 172.21.15.82:0/1581172531 >> 172.21.15.82:0/1581172531 conn(0x7f21c40d0a20 msgr2=0x7f21c40d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 -- 172.21.15.82:0/1581172531 shutdown_connections 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.020+0000 7f21c9380640 1 -- 172.21.15.82:0/1581172531 wait complete. 2024-03-14T12:14:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.021+0000 7f21c9380640 1 Processor -- start 2024-03-14T12:14:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.021+0000 7f21c9380640 1 -- start start 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c9380640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c40af150 0x7f21c413cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c9380640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c4140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c9380640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c9380640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40aff40 con 0x7f21c40b58f0 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c9380640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40afdc0 con 0x7f21c40af150 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c9380640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4157cd0 con 0x7f21c415f1f0 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c4140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c4140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51404/0 (socket says 172.21.15.82:51404) 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 -- 172.21.15.82:0/3098719935 learned_addr learned my addr 172.21.15.82:0/3098719935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c40af150 0x7f21c413cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c37fe640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 -- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 msgr2=0x7f21c414e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c414e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 -- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c40af150 msgr2=0x7f21c413cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c40af150 0x7f21c413cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.022+0000 7f21c27fc640 1 -- 172.21.15.82:0/3098719935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4140550 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.023+0000 7f21c27fc640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c4140010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d5f0 tx=0x7f21b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.023+0000 7f21ab7fe640 1 -- 172.21.15.82:0/3098719935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80125f0 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.023+0000 7f21ab7fe640 1 -- 172.21.15.82:0/3098719935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8019070 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.023+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c414c930 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.023+0000 7f21ab7fe640 1 -- 172.21.15.82:0/3098719935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80129d0 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.023+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c4150b50 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.025+0000 7f21ab7fe640 1 -- 172.21.15.82:0/3098719935 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f21b8012b70 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.025+0000 7f21ab7fe640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f21b006bbd0 0x7f21b006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.025+0000 7f21ab7fe640 1 -- 172.21.15.82:0/3098719935 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21b808ace0 con 0x7f21c40b58f0 2024-03-14T12:14:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.025+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f21b006bbd0 0x7f21b006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.025+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f21b006bbd0 0x7f21b006e090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21b4003aa0 tx=0x7f21b4009830 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.026+0000 7f21cc035740 1 Processor -- start 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.026+0000 7f21cc035740 1 -- start start 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920f9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920f9d650 0x556920f75620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21cc035740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920f75b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21cc035740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556920eda7c0 con 0x556920f769c0 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556920e83b50 con 0x556920f9d650 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x556920ed6860 con 0x556920f765e0 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920f75b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920f75b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51418/0 (socket says 172.21.15.82:51418) 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 -- 172.21.15.82:0/1386451804 learned_addr learned my addr 172.21.15.82:0/1386451804 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 -- 172.21.15.82:0/1386451804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 msgr2=0x556920f9d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1386451804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920f9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 -- 172.21.15.82:0/1386451804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920f9d650 msgr2=0x556920f75620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1386451804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920f9d650 0x556920f75620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 -- 172.21.15.82:0/1386451804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556920e979a0 con 0x556920f769c0 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1386451804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920f75b60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006c80 tx=0x7f21bc008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e2ed7eb122ba05 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1386451804 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc031a20 con 0x556920f769c0 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1386451804 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc04a070 con 0x556920f769c0 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.027+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1386451804 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc031dc0 con 0x556920f769c0 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 -- 172.21.15.82:0/1386451804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 msgr2=0x556920f75b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 --2- 172.21.15.82:0/1386451804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920f75b60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006c80 tx=0x7f21bc008030 comp rx=0 tx=0).stop 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 -- 172.21.15.82:0/1386451804 shutdown_connections 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 --2- 172.21.15.82:0/1386451804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920f75b60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 --2- 172.21.15.82:0/1386451804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920f9d650 0x556920f75620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 --2- 172.21.15.82:0/1386451804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920f9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 -- 172.21.15.82:0/1386451804 >> 172.21.15.82:0/1386451804 conn(0x556920f3d400 msgr2=0x556920fbb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 -- 172.21.15.82:0/1386451804 shutdown_connections 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 -- 172.21.15.82:0/1386451804 wait complete. 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.028+0000 7f21cc035740 1 -- start start 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 1 Processor -- start 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920f944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fbdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fcb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556920f9cc80 con 0x556920f769c0 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556920e96f90 con 0x556920fbe320 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x556920f95a00 con 0x556920f765e0 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21cc035740 10 client.0 _ll_register_callbacks cb 0x556920ed2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fbdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fbdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51428/0 (socket says 172.21.15.82:51428) 2024-03-14T12:14:07.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21c27fc640 1 -- 172.21.15.82:0/3726367798 learned_addr learned my addr 172.21.15.82:0/3726367798 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920f944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.029+0000 7f21c37fe640 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fcb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21c27fc640 1 -- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 msgr2=0x556920f944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21c27fc640 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920f944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21c27fc640 1 -- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 msgr2=0x556920fcb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21c27fc640 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fcb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21c27fc640 1 -- 172.21.15.82:0/3726367798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556920bef2d0 con 0x556920f769c0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21c27fc640 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fbdde0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d5c0 tx=0x7f21b8059370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765828c052580ac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21a1ffb640 1 -- 172.21.15.82:0/3726367798 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8055880 con 0x556920f769c0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21a1ffb640 1 -- 172.21.15.82:0/3726367798 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21b8058660 con 0x556920f769c0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21bc04a070 con 0x556920f769c0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21a1ffb640 1 -- 172.21.15.82:0/3726367798 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21b8055a20 con 0x556920f769c0 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 10 client.4638 fetch_fsmap learned FSMap version 7 2024-03-14T12:14:07.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x556920cb96b0 con 0x556920f769c0 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21a1ffb640 1 -- 172.21.15.82:0/3726367798 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc04a070 con 0x556920f769c0 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21a1ffb640 1 -- 172.21.15.82:0/3726367798 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21b8013070 con 0x556920f769c0 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 10 client.4638 fetch_fsmap finished waiting for FSMap version 7 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 20 client.4638 populate_metadata read hostname 'smithi082' 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fcec30 0x556920fcf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 4 client.4638 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.030+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 --> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556920ee7d00 con 0x556920fcec30 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.031+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fcec30 0x556920fcf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.031+0000 7f21a1ffb640 10 client.4638 ms_handle_connect on v2:172.21.15.164:6832/3117714765 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.031+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fcec30 0x556920fcf010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a66183647e6f2068 server_cookie=f61c96ce457311c7 in_seq=0 out_seq=0 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21a1ffb640 1 -- 172.21.15.82:0/3726367798 <== mds.0 v2:172.21.15.164:6832/3117714765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556920ee7d00 con 0x556920fcec30 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21a1ffb640 10 client.4638 handle_command_reply: tid=0 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 client.4638 shutdown 2024-03-14T12:14:07.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 msgr2=0x556920fbdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fbdde0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d5c0 tx=0x7f21b8059370 comp rx=0 tx=0).stop 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 shutdown_connections 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fcec30 0x556920fcf010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fcb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fbdde0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21a1ffb640 0 client.4638 ms_handle_reset on v2:172.21.15.164:6832/3117714765 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 --2- 172.21.15.82:0/3726367798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920f944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 >> 172.21.15.82:0/3726367798 conn(0x556920f3d400 msgr2=0x556920f9e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 shutdown_connections 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 1 -- 172.21.15.82:0/3726367798 wait complete. 2024-03-14T12:14:07.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.032+0000 7f21cc035740 20 client.4638 trim_cache size 0 max 16384 2024-03-14T12:14:07.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 Processor -- start 2024-03-14T12:14:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 -- start start 2024-03-14T12:14:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920fba9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fb0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b15a0 con 0x556920f769c0 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41f7e30 con 0x556920fbe320 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.048+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b0fa0 con 0x556920f765e0 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fb0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fb0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51442/0 (socket says 172.21.15.82:51442) 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 -- 172.21.15.82:0/1293998933 learned_addr learned my addr 172.21.15.82:0/1293998933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920fba9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 -- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 msgr2=0x556920fba9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920fba9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 -- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 msgr2=0x556920fb11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 -- 172.21.15.82:0/1293998933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556920f72030 con 0x556920f769c0 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21c27fc640 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fb0c90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009020 tx=0x7f21b8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dad4acf2161e9b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21a3fff640 1 -- 172.21.15.82:0/1293998933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8055880 con 0x556920f769c0 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21a3fff640 1 -- 172.21.15.82:0/1293998933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8089170 con 0x556920f769c0 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.049+0000 7f21a3fff640 1 -- 172.21.15.82:0/1293998933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8058860 con 0x556920f769c0 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 -- 172.21.15.82:0/1293998933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 msgr2=0x556920fb0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fb0c90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009020 tx=0x7f21b8020000 comp rx=0 tx=0).stop 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 -- 172.21.15.82:0/1293998933 shutdown_connections 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f769c0 0x556920fb0c90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 --2- 172.21.15.82:0/1293998933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f765e0 0x556920fba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 -- 172.21.15.82:0/1293998933 >> 172.21.15.82:0/1293998933 conn(0x556920f3d400 msgr2=0x556920f9e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 -- 172.21.15.82:0/1293998933 shutdown_connections 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 -- 172.21.15.82:0/1293998933 wait complete. 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 -- start start 2024-03-14T12:14:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.050+0000 7f21cc035740 1 Processor -- start 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21cc035740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f765e0 0x556921025ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f769c0 0x556920fb5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21cc035740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21cc035740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556920fbc510 con 0x556920f765e0 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556920fbbeb0 con 0x556920fbe320 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21cc035740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x556920fbca50 con 0x556920f769c0 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21cc035740 10 client.0 _ll_register_callbacks cb 0x7f21bc006bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:14:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f765e0 0x556921025ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f765e0 0x556921025ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51448/0 (socket says 172.21.15.82:51448) 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 -- 172.21.15.82:0/286562362 learned_addr learned my addr 172.21.15.82:0/286562362 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c27fc640 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f769c0 0x556920fb5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c37fe640 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 -- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f769c0 msgr2=0x556920fb5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f769c0 0x556920fb5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 -- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 msgr2=0x556920fb7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 -- 172.21.15.82:0/286562362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556920fb88a0 con 0x556920f765e0 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f765e0 0x556921025ba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556920fbbde0 tx=0x7f21b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1accc9eb8a863db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.051+0000 7f21a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21a1ffb640 1 -- 172.21.15.82:0/286562362 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b40278c0 con 0x556920f765e0 2024-03-14T12:14:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21a1ffb640 1 -- 172.21.15.82:0/286562362 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21b4012040 con 0x556920f765e0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21b8089170 con 0x556920f765e0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21a1ffb640 1 -- 172.21.15.82:0/286562362 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21b4017430 con 0x556920f765e0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 10 client.4644 fetch_fsmap learned FSMap version 7 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x556920fb5ae0 con 0x556920f765e0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21a1ffb640 1 -- 172.21.15.82:0/286562362 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8089170 con 0x556920f765e0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21a1ffb640 1 -- 172.21.15.82:0/286562362 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21b4038050 con 0x556920f765e0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 10 client.4644 fetch_fsmap finished waiting for FSMap version 7 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 10 client.4644 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 20 client.4644 populate_metadata read hostname 'smithi082' 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fa36f0 0x556920fb8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 4 client.4644 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 --> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556920ee2050 con 0x556920fa36f0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.052+0000 7f21c27fc640 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fa36f0 0x556920fb8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21a1ffb640 10 client.4644 ms_handle_connect on v2:172.21.15.164:6832/3117714765 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21c27fc640 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fa36f0 0x556920fb8fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb09f9876bc9f78 server_cookie=9f14e406acbbedbf in_seq=0 out_seq=0 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21a1ffb640 1 -- 172.21.15.82:0/286562362 <== mds.0 v2:172.21.15.164:6832/3117714765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556920ee2050 con 0x556920fa36f0 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21a1ffb640 10 client.4644 handle_command_reply: tid=0 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 client.4644 shutdown 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f765e0 msgr2=0x556921025ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f765e0 0x556921025ba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556920fbbde0 tx=0x7f21b4004ab0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 shutdown_connections 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:6832/3117714765,v1:172.21.15.164:6834/3117714765] conn(0x556920fa36f0 0x556920fb8fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556920fbe320 0x556920fb7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21a1ffb640 0 client.4644 ms_handle_reset on v2:172.21.15.164:6832/3117714765 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x556920f769c0 0x556920fb5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 --2- 172.21.15.82:0/286562362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556920f765e0 0x556921025ba0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 >> 172.21.15.82:0/286562362 conn(0x556920f3d400 msgr2=0x556920fb2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 shutdown_connections 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.053+0000 7f21cc035740 1 -- 172.21.15.82:0/286562362 wait complete. 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21cc035740 20 client.4644 trim_cache size 0 max 16384 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2", 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 4306, 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 15.481210076, 2024-03-14T12:14:07.059 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 7, 2024-03-14T12:14:07.060 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 21, 2024-03-14T12:14:07.060 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 21, 2024-03-14T12:14:07.060 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 21.972651972000001 2024-03-14T12:14:07.060 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f21b006bbd0 msgr2=0x7f21b006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f21b006bbd0 0x7f21b006e090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21b4003aa0 tx=0x7f21b4009830 comp rx=0 tx=0).stop 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 msgr2=0x7f21c4140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c4140010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d5f0 tx=0x7f21b800dab0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 shutdown_connections 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f21b006bbd0 0x7f21b006e090 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c415f1f0 0x7f21c414e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c40b58f0 0x7f21c4140010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 --2- 172.21.15.82:0/3098719935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c40af150 0x7f21c413cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 >> 172.21.15.82:0/3098719935 conn(0x7f21c40d0a20 msgr2=0x7f21c406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 shutdown_connections 2024-03-14T12:14:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.054+0000 7f21c9380640 1 -- 172.21.15.82:0/3098719935 wait complete. 2024-03-14T12:14:07.064 INFO:teuthology.run_tasks:Running task kclient... 2024-03-14T12:14:07.088 INFO:tasks.kclient:Mounting kernel clients... 2024-03-14T12:14:07.088 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-14T12:14:07.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.088 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:14:07.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.103 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-14T12:14:07.172 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-03-14T12:14:07.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:14:07.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.174 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:14:07.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.228 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-14T12:14:07.296 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-03-14T12:14:07.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:14:07.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.298 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:14:07.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.317 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-03-14T12:14:07.391 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-03-14T12:14:07.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:14:07.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.393 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:14:07.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.450 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-03-14T12:14:07.521 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-03-14T12:14:07.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:14:07.522 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-14T12:14:07.522 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-14T12:14:07.522 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:07.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.627+0000 7fb0f7f7d640 1 Processor -- start 2024-03-14T12:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.627+0000 7fb0f7f7d640 1 -- start start 2024-03-14T12:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f7f7d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0f0066b00 0x7fb0f006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f7f7d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00ccb40 con 0x7fb0f015be40 2024-03-14T12:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f7f7d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00cce20 con 0x7fb0f015c220 2024-03-14T12:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f7f7d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00cd750 con 0x7fb0f0066b00 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f5d19640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb0f015be40 0x7fb0f00d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54290/0 (socket says 172.21.15.82:54290) 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f5d19640 1 -- 172.21.15.82:0/1121311918 learned_addr learned my addr 172.21.15.82:0/1121311918 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f5518640 1 --2- 172.21.15.82:0/1121311918 >> v2:172.21.15.164:3300/0 conn(0x7fb0f0066b00 0x7fb0f006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f4d17640 1 -- 172.21.15.82:0/1121311918 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820109064 0 0) 0x7fb0f00ccb40 con 0x7fb0f015be40 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f4d17640 1 -- 172.21.15.82:0/1121311918 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0dc003520 con 0x7fb0f015be40 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f5518640 1 -- 172.21.15.82:0/1121311918 >> v1:172.21.15.164:6789/0 conn(0x7fb0f015c220 legacy=0x7fb0f00cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f5518640 1 -- 172.21.15.82:0/1121311918 >> v1:172.21.15.82:6789/0 conn(0x7fb0f015be40 legacy=0x7fb0f00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.628+0000 7fb0f5518640 1 -- 172.21.15.82:0/1121311918 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00cd100 con 0x7fb0f0066b00 2024-03-14T12:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.629+0000 7fb0f5518640 1 --2- 172.21.15.82:0/1121311918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f0066b00 0x7fb0f006ad10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00097a0 tx=0x7fb0e002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4215d39c45cb8ca server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.630+0000 7fb0f4d17640 1 -- 172.21.15.82:0/1121311918 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0034070 con 0x7fb0f0066b00 2024-03-14T12:14:07.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f4d17640 1 -- 172.21.15.82:0/1121311918 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e0030610 con 0x7fb0f0066b00 2024-03-14T12:14:07.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f4d17640 1 -- 172.21.15.82:0/1121311918 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00309e0 con 0x7fb0f0066b00 2024-03-14T12:14:07.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/1121311918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f0066b00 msgr2=0x7fb0f006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/1121311918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f0066b00 0x7fb0f006ad10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00097a0 tx=0x7fb0e002b960 comp rx=0 tx=0).stop 2024-03-14T12:14:07.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/1121311918 shutdown_connections 2024-03-14T12:14:07.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/1121311918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f0066b00 0x7fb0f006ad10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/1121311918 >> 172.21.15.82:0/1121311918 conn(0x7fb0f0072310 msgr2=0x7fb0f0072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/1121311918 shutdown_connections 2024-03-14T12:14:07.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.631+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/1121311918 wait complete. 2024-03-14T12:14:07.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.632+0000 7fb0f7f7d640 1 Processor -- start 2024-03-14T12:14:07.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.632+0000 7fb0f7f7d640 1 -- start start 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f7f7d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0f0066b00 0x7fb0f015e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f7f7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f015be40 0x7fb0f01f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f7f7d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f015c220 0x7fb0f01f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f7f7d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b1310 con 0x7fb0f015be40 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f7f7d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b1190 con 0x7fb0f015c220 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f7f7d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f006b250 con 0x7fb0f0066b00 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f015be40 0x7fb0f01f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f015be40 0x7fb0f01f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51458/0 (socket says 172.21.15.82:51458) 2024-03-14T12:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 -- 172.21.15.82:0/3372215121 learned_addr learned my addr 172.21.15.82:0/3372215121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5d19640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0f0066b00 0x7fb0f015e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f651a640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f015c220 0x7fb0f01f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 -- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0f0066b00 msgr2=0x7fb0f015e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0f0066b00 0x7fb0f015e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 -- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f015c220 msgr2=0x7fb0f01f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f015c220 0x7fb0f01f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5518640 1 -- 172.21.15.82:0/3372215121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00c7520 con 0x7fb0f015be40 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f5d19640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0f0066b00 0x7fb0f015e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.633+0000 7fb0f651a640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f015c220 0x7fb0f01f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.634+0000 7fb0f5518640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f015be40 0x7fb0f01f4c10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0000c00 tx=0x7fb0e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.634+0000 7fb0e67fc640 1 -- 172.21.15.82:0/3372215121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0034040 con 0x7fb0f015be40 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.634+0000 7fb0e67fc640 1 -- 172.21.15.82:0/3372215121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e0030410 con 0x7fb0f015be40 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.634+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f00c7800 con 0x7fb0f015be40 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.634+0000 7fb0e67fc640 1 -- 172.21.15.82:0/3372215121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0039c70 con 0x7fb0f015be40 2024-03-14T12:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.634+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f00c7dc0 con 0x7fb0f015be40 2024-03-14T12:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.635+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f004f050 con 0x7fb0f015be40 2024-03-14T12:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.635+0000 7fb0e67fc640 1 -- 172.21.15.82:0/3372215121 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb0e0037070 con 0x7fb0f015be40 2024-03-14T12:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.636+0000 7fb0e67fc640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb0d406bbd0 0x7fb0d406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.636+0000 7fb0f5d19640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb0d406bbd0 0x7fb0d406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.636+0000 7fb0e67fc640 1 -- 172.21.15.82:0/3372215121 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0e00ac960 con 0x7fb0f015be40 2024-03-14T12:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.636+0000 7fb0f5d19640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb0d406bbd0 0x7fb0d406e090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8001560 tx=0x7fb0e8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:07.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.640+0000 7fb0e67fc640 1 -- 172.21.15.82:0/3372215121 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb0e007b600 con 0x7fb0f015be40 2024-03-14T12:14:07.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.848+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb0f006d060 con 0x7fb0f015be40 2024-03-14T12:14:07.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.848+0000 7fb0e67fc640 1 -- 172.21.15.82:0/3372215121 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0e007af00 con 0x7fb0f015be40 2024-03-14T12:14:07.848 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:14:07.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb0d406bbd0 msgr2=0x7fb0d406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb0d406bbd0 0x7fb0d406e090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8001560 tx=0x7fb0e8009040 comp rx=0 tx=0).stop 2024-03-14T12:14:07.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f015be40 msgr2=0x7fb0f01f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:07.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f015be40 0x7fb0f01f4c10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0000c00 tx=0x7fb0e00046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 shutdown_connections 2024-03-14T12:14:07.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f015c220 0x7fb0f01f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f015be40 0x7fb0f01f4c10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb0d406bbd0 0x7fb0d406e090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 --2- 172.21.15.82:0/3372215121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0f0066b00 0x7fb0f015e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:07.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 >> 172.21.15.82:0/3372215121 conn(0x7fb0f0072310 msgr2=0x7fb0f006e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:07.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 shutdown_connections 2024-03-14T12:14:07.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:07.850+0000 7fb0f7f7d640 1 -- 172.21.15.82:0/3372215121 wait complete. 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:14:07.860 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:14:07.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.861 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42672sec preferred_lft 42672sec 2024-03-14T12:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:07.877 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-14T12:14:07.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:07.877 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:07.877 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:07.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-03-14T12:14:07.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-03-14T12:14:07.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-03-14T12:14:07.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-14T12:14:07.878 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:08.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.054 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:14:08.080 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:14:08.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.082 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:14:08.150 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:14:08.150 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:14:08.150 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:14:08.150 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:14:08.150 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:14:08.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.151 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:08.151 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:08.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:14:08.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:14:08.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:14:08.151 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:08.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.420 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:14:08.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.434 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-03-14T12:14:08.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.489 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:08.490 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:08.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:14:08.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-14T12:14:08.490 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:08.572 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-14T12:14:08.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.572 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:08.572 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:08.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-14T12:14:08.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-14T12:14:08.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-14T12:14:08.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-14T12:14:08.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-14T12:14:08.573 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:08.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.722 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:08.722 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:08.722 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-14T12:14:08.722 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-03-14T12:14:08.722 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:08.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:08.763 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:08.818 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:14:08.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:08.818 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:08.872 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T12:14:08.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:08.872 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:08.957 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:08.957 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:09.070 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:09.071 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:09.071 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:14:09.071 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:14:09.071 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:14:09.073 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T12:14:09.074 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:14:09.074 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:14:09.074 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:14:09.074 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:14:09.074 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:14:09.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.074 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:09.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.103 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:09.165 INFO:teuthology.orchestra.run.smithi082.stdout:client4573 2024-03-14T12:14:09.166 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:09.166 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:09.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.221 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:09.284 INFO:teuthology.orchestra.run.smithi082.stdout:client4573 2024-03-14T12:14:09.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.284 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:09.345 INFO:teuthology.orchestra.run.smithi082.stdout:client4573 2024-03-14T12:14:09.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.345 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.client4573/status') 2024-03-14T12:14:09.416 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.4573 (3)172.21.15.82:0/27549187 2024-03-14T12:14:09.416 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-03-14T12:14:09.417 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-03-14T12:14:09.417 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-03-14T12:14:09.417 INFO:teuthology.orchestra.run.smithi082.stderr:68 bytes copied, 2.0548e-05 s, 3.3 MB/s 2024-03-14T12:14:09.418 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-14T12:14:09.418 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-14T12:14:09.418 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-14T12:14:09.418 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:14:09.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.557+0000 7fb860d9e640 1 Processor -- start 2024-03-14T12:14:09.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.557+0000 7fb860d9e640 1 -- start start 2024-03-14T12:14:09.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb860d9e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb85c0b4190 0x7fb85c160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:09.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb860d9e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb85c0cc770 0x7fb85c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb860d9e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb85c06fc70 con 0x7fb85c06ed90 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb860d9e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c15d340 con 0x7fb85c0cc770 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb860d9e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c0ce610 con 0x7fb85c0b4190 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb85ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb85c0cc770 0x7fb85c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb85ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb85c0cc770 0x7fb85c0ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51462/0 (socket says 172.21.15.82:51462) 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.558+0000 7fb85ad9d640 1 -- 172.21.15.82:0/4122598231 learned_addr learned my addr 172.21.15.82:0/4122598231 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.559+0000 7fb859d9b640 1 --2- 172.21.15.82:0/4122598231 >> v2:172.21.15.164:3300/0 conn(0x7fb85c0b4190 0x7fb85c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.559+0000 7fb85ad9d640 1 -- 172.21.15.82:0/4122598231 >> v2:172.21.15.164:3300/0 conn(0x7fb85c0b4190 msgr2=0x7fb85c160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.559+0000 7fb85ad9d640 1 --2- 172.21.15.82:0/4122598231 >> v2:172.21.15.164:3300/0 conn(0x7fb85c0b4190 0x7fb85c160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.559+0000 7fb85ad9d640 1 -- 172.21.15.82:0/4122598231 >> v1:172.21.15.82:6789/0 conn(0x7fb85c06ed90 legacy=0x7fb85c06f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.559+0000 7fb85ad9d640 1 -- 172.21.15.82:0/4122598231 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c0ce8f0 con 0x7fb85c0cc770 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.559+0000 7fb85ad9d640 1 --2- 172.21.15.82:0/4122598231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c0cc770 0x7fb85c0ccbe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb85c070b30 tx=0x7fb85402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b677c77c09b6e633 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.559+0000 7fb85959a640 1 -- 172.21.15.82:0/4122598231 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854034070 con 0x7fb85c0cc770 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb85959a640 1 -- 172.21.15.82:0/4122598231 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb854030860 con 0x7fb85c0cc770 2024-03-14T12:14:09.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb85959a640 1 -- 172.21.15.82:0/4122598231 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854030c30 con 0x7fb85c0cc770 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 -- 172.21.15.82:0/4122598231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c0cc770 msgr2=0x7fb85c0ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 --2- 172.21.15.82:0/4122598231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c0cc770 0x7fb85c0ccbe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb85c070b30 tx=0x7fb85402b9a0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 -- 172.21.15.82:0/4122598231 shutdown_connections 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 --2- 172.21.15.82:0/4122598231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c0cc770 0x7fb85c0ccbe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 --2- 172.21.15.82:0/4122598231 >> v2:172.21.15.164:3300/0 conn(0x7fb85c0b4190 0x7fb85c160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 -- 172.21.15.82:0/4122598231 >> 172.21.15.82:0/4122598231 conn(0x7fb85c0b3670 msgr2=0x7fb85c0b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 -- 172.21.15.82:0/4122598231 shutdown_connections 2024-03-14T12:14:09.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.560+0000 7fb860d9e640 1 -- 172.21.15.82:0/4122598231 wait complete. 2024-03-14T12:14:09.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.561+0000 7fb860d9e640 1 Processor -- start 2024-03-14T12:14:09.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb860d9e640 1 -- start start 2024-03-14T12:14:09.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb860d9e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06ed90 0x7fb85c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:09.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb860d9e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0b4190 0x7fb85c155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:09.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb860d9e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c0cc770 0x7fb85c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:09.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb860d9e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0cf190 con 0x7fb85c06ed90 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb860d9e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0ceeb0 con 0x7fb85c0cc770 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb860d9e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0cd120 con 0x7fb85c0b4190 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb85a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06ed90 0x7fb85c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb85a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06ed90 0x7fb85c156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51464/0 (socket says 172.21.15.82:51464) 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.562+0000 7fb85a59c640 1 -- 172.21.15.82:0/700491492 learned_addr learned my addr 172.21.15.82:0/700491492 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb859d9b640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0b4190 0x7fb85c155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85ad9d640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c0cc770 0x7fb85c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85a59c640 1 -- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0b4190 msgr2=0x7fb85c155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85a59c640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0b4190 0x7fb85c155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85a59c640 1 -- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c0cc770 msgr2=0x7fb85c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85a59c640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c0cc770 0x7fb85c155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85a59c640 1 -- 172.21.15.82:0/700491492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c1e4260 con 0x7fb85c06ed90 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85ad9d640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c0cc770 0x7fb85c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb85a59c640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06ed90 0x7fb85c156ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c790 tx=0x7fb84c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:09.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb84affd640 1 -- 172.21.15.82:0/700491492 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c016020 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb84affd640 1 -- 172.21.15.82:0/700491492 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c006d50 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c1e6fe0 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb84affd640 1 -- 172.21.15.82:0/700491492 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c003bb0 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.563+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c1e47e0 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.564+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c04f050 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.564+0000 7fb84affd640 1 -- 172.21.15.82:0/700491492 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb84c00a850 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.565+0000 7fb84affd640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb83c06bbd0 0x7fb83c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.565+0000 7fb859d9b640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb83c06bbd0 0x7fb83c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.565+0000 7fb84affd640 1 -- 172.21.15.82:0/700491492 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb84c08a710 con 0x7fb85c06ed90 2024-03-14T12:14:09.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.565+0000 7fb859d9b640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb83c06bbd0 0x7fb83c06e090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb85c1f5cb0 tx=0x7fb850008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:09.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.569+0000 7fb84affd640 1 -- 172.21.15.82:0/700491492 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb84c020020 con 0x7fb85c06ed90 2024-03-14T12:14:09.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.778+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb85c06fa50 con 0x7fb85c06ed90 2024-03-14T12:14:09.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.778+0000 7fb84affd640 1 -- 172.21.15.82:0/700491492 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb85c06fc70 con 0x7fb85c06ed90 2024-03-14T12:14:09.777 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:14:09.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb83c06bbd0 msgr2=0x7fb83c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:09.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb83c06bbd0 0x7fb83c06e090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb85c1f5cb0 tx=0x7fb850008040 comp rx=0 tx=0).stop 2024-03-14T12:14:09.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06ed90 msgr2=0x7fb85c156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:09.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06ed90 0x7fb85c156ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c790 tx=0x7fb84c00cc50 comp rx=0 tx=0).stop 2024-03-14T12:14:09.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 shutdown_connections 2024-03-14T12:14:09.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c0cc770 0x7fb85c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb83c06bbd0 0x7fb83c06e090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0b4190 0x7fb85c155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 --2- 172.21.15.82:0/700491492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06ed90 0x7fb85c156ab0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:09.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 >> 172.21.15.82:0/700491492 conn(0x7fb85c0b3670 msgr2=0x7fb85c158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:09.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 shutdown_connections 2024-03-14T12:14:09.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:09.780+0000 7fb860d9e640 1 -- 172.21.15.82:0/700491492 wait complete. 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:14:09.790 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-14T12:14:09.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.791 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:09.806 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42670sec preferred_lft 42670sec 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:14:09.807 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:09.808 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c443:69ff:fec0:3488/64 scope link tentative 2024-03-14T12:14:09.808 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:09.808 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:14:09.808 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:14:09.808 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::887e:60ff:fe47:4333/64 scope link tentative 2024-03-14T12:14:09.808 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:09.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.808 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:14:09.871 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:14:09.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.873 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:14:09.928 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:14:09.928 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:14:09.928 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:14:09.928 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:14:09.928 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:14:09.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:09.929 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:09.929 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:09.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:14:09.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:14:09.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:14:09.929 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:10.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.024 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:14:10.038 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:14:10.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.039 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-03-14T12:14:10.093 INFO:teuthology.orchestra.run.smithi082.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-14T12:14:10.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.094 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:10.094 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:10.094 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:14:10.094 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-14T12:14:10.094 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:10.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8e:f1:b8:2c:f6:3b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::8cf1:b8ff:fe2c:f63b/64 scope link tentative 2024-03-14T12:14:10.241 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.243 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-14T12:14:10.310 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:10.310 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:10.310 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:10.310 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.310 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:10.311 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.311 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:14:10.311 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8e:f1:b8:2c:f6:3b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-14T12:14:10.311 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-14T12:14:10.311 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.311 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::8cf1:b8ff:fe2c:f63b/64 scope link tentative 2024-03-14T12:14:10.311 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:10.312 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-14T12:14:10.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.312 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:10.313 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:10.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-14T12:14:10.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-14T12:14:10.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-14T12:14:10.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-14T12:14:10.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-14T12:14:10.313 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:10.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.437 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:10.437 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:10.437 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.1 up 2024-03-14T12:14:10.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.1 master ceph-brx 2024-03-14T12:14:10.438 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:10.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:10.529 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-14T12:14:10.544 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-14T12:14:10.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:10.544 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-14T12:14:10.598 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-14T12:14:10.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.599 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:10.683 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:10.684 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:10.690 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:10.690 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:10.690 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:14:10.691 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 2@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:14:10.691 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:14:10.693 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-14T12:14:10.693 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:14:10.693 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:14:10.693 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:14:10.693 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:14:10.693 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:14:10.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.694 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-14T12:14:10.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.726 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:10.783 INFO:teuthology.orchestra.run.smithi082.stdout:client4662 2024-03-14T12:14:10.784 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:10.784 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:10.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.839 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:10.895 INFO:teuthology.orchestra.run.smithi082.stdout:client4662 2024-03-14T12:14:10.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.895 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:10.951 INFO:teuthology.orchestra.run.smithi082.stdout:client4662 2024-03-14T12:14:10.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:10.951 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.client4662/status') 2024-03-14T12:14:11.018 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.4662 (3)192.168.0.2:0/128075343 2024-03-14T12:14:11.018 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-03-14T12:14:11.018 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-03-14T12:14:11.019 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-03-14T12:14:11.019 INFO:teuthology.orchestra.run.smithi082.stderr:68 bytes copied, 3.7663e-05 s, 1.8 MB/s 2024-03-14T12:14:11.020 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-14T12:14:11.020 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-14T12:14:11.020 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T12:14:11.020 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:14:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.160+0000 7fbc94e20640 1 Processor -- start 2024-03-14T12:14:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.160+0000 7fbc94e20640 1 -- start start 2024-03-14T12:14:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc94e20640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc9015f1f0 0x7fbc9006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc94e20640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc900b58f0 0x7fbc9006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc94e20640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900b01d0 con 0x7fbc900af150 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc94e20640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc900aff90 con 0x7fbc9015f1f0 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc94e20640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015e710 con 0x7fbc900b58f0 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc8dd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc9015f1f0 0x7fbc9006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc8dd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc9015f1f0 0x7fbc9006ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51476/0 (socket says 172.21.15.82:51476) 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/2153159708 learned_addr learned my addr 172.21.15.82:0/2153159708 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.161+0000 7fbc8ed9d640 1 --2- 172.21.15.82:0/2153159708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc900b58f0 0x7fbc9006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.162+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/2153159708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc900b58f0 msgr2=0x7fbc9006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.162+0000 7fbc8dd9b640 1 --2- 172.21.15.82:0/2153159708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc900b58f0 0x7fbc9006b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.162+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/2153159708 >> v1:172.21.15.164:6789/0 conn(0x7fbc900af150 legacy=0x7fbc900af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.162+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/2153159708 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9015e9f0 con 0x7fbc9015f1f0 2024-03-14T12:14:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.162+0000 7fbc8ed9d640 1 --2- 172.21.15.82:0/2153159708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc900b58f0 0x7fbc9006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.162+0000 7fbc8dd9b640 1 --2- 172.21.15.82:0/2153159708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc9015f1f0 0x7fbc9006ad10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009e50 tx=0x7fbc8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bf9e8e1588d6dc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc8d59a640 1 -- 172.21.15.82:0/2153159708 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8402ca70 con 0x7fbc9015f1f0 2024-03-14T12:14:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc8d59a640 1 -- 172.21.15.82:0/2153159708 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8402cc10 con 0x7fbc9015f1f0 2024-03-14T12:14:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc8d59a640 1 -- 172.21.15.82:0/2153159708 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc840343f0 con 0x7fbc9015f1f0 2024-03-14T12:14:11.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 -- 172.21.15.82:0/2153159708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc9015f1f0 msgr2=0x7fbc9006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 --2- 172.21.15.82:0/2153159708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc9015f1f0 0x7fbc9006ad10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009e50 tx=0x7fbc8402bed0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 -- 172.21.15.82:0/2153159708 shutdown_connections 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 --2- 172.21.15.82:0/2153159708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc900b58f0 0x7fbc9006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 --2- 172.21.15.82:0/2153159708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc9015f1f0 0x7fbc9006ad10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 -- 172.21.15.82:0/2153159708 >> 172.21.15.82:0/2153159708 conn(0x7fbc900d0a20 msgr2=0x7fbc900d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 -- 172.21.15.82:0/2153159708 shutdown_connections 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.163+0000 7fbc94e20640 1 -- 172.21.15.82:0/2153159708 wait complete. 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.164+0000 7fbc94e20640 1 Processor -- start 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc94e20640 1 -- start start 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc94e20640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc900af150 0x7fbc900cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc94e20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc900b58f0 0x7fbc901f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc94e20640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc9015f1f0 0x7fbc901f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc94e20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90157cd0 con 0x7fbc900b58f0 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc94e20640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015e400 con 0x7fbc900af150 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc94e20640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90157e50 con 0x7fbc9015f1f0 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc8dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc900b58f0 0x7fbc901f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc8dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc900b58f0 0x7fbc901f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51492/0 (socket says 172.21.15.82:51492) 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/3460597763 learned_addr learned my addr 172.21.15.82:0/3460597763 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.165+0000 7fbc8ed9d640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc9015f1f0 0x7fbc901f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8e59c640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc900af150 0x7fbc900cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc9015f1f0 msgr2=0x7fbc901f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8dd9b640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc9015f1f0 0x7fbc901f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc900af150 msgr2=0x7fbc900cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8dd9b640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc900af150 0x7fbc900cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8dd9b640 1 -- 172.21.15.82:0/3460597763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900cf720 con 0x7fbc900b58f0 2024-03-14T12:14:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8ed9d640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc9015f1f0 0x7fbc901f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc8dd9b640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc900b58f0 0x7fbc901f12b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc84000f80 tx=0x7fbc84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc7effd640 1 -- 172.21.15.82:0/3460597763 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84044020 con 0x7fbc900b58f0 2024-03-14T12:14:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc7effd640 1 -- 172.21.15.82:0/3460597763 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc840406b0 con 0x7fbc900b58f0 2024-03-14T12:14:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc900cfa00 con 0x7fbc900b58f0 2024-03-14T12:14:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc7effd640 1 -- 172.21.15.82:0/3460597763 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84009510 con 0x7fbc900b58f0 2024-03-14T12:14:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.166+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc901e74c0 con 0x7fbc900b58f0 2024-03-14T12:14:11.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.167+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9004f050 con 0x7fbc900b58f0 2024-03-14T12:14:11.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.168+0000 7fbc7effd640 1 -- 172.21.15.82:0/3460597763 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbc84047020 con 0x7fbc900b58f0 2024-03-14T12:14:11.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.168+0000 7fbc7effd640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbc7006bbd0 0x7fbc7006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:11.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.168+0000 7fbc7effd640 1 -- 172.21.15.82:0/3460597763 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc840ad520 con 0x7fbc900b58f0 2024-03-14T12:14:11.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.168+0000 7fbc8e59c640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbc7006bbd0 0x7fbc7006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:11.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.169+0000 7fbc8e59c640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbc7006bbd0 0x7fbc7006e090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc88001640 tx=0x7fbc88009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:11.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.172+0000 7fbc7effd640 1 -- 172.21.15.82:0/3460597763 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbc8404e030 con 0x7fbc900b58f0 2024-03-14T12:14:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.383+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc901e7690 con 0x7fbc900b58f0 2024-03-14T12:14:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.383+0000 7fbc7effd640 1 -- 172.21.15.82:0/3460597763 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc8407c0a0 con 0x7fbc900b58f0 2024-03-14T12:14:11.382 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:14:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbc7006bbd0 msgr2=0x7fbc7006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbc7006bbd0 0x7fbc7006e090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc88001640 tx=0x7fbc88009040 comp rx=0 tx=0).stop 2024-03-14T12:14:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc900b58f0 msgr2=0x7fbc901f12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc900b58f0 0x7fbc901f12b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc84000f80 tx=0x7fbc84004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 shutdown_connections 2024-03-14T12:14:11.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbc7006bbd0 0x7fbc7006e090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc9015f1f0 0x7fbc901f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc900b58f0 0x7fbc901f12b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 --2- 172.21.15.82:0/3460597763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc900af150 0x7fbc900cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:11.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 >> 172.21.15.82:0/3460597763 conn(0x7fbc900d0a20 msgr2=0x7fbc900721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:11.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 shutdown_connections 2024-03-14T12:14:11.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:11.385+0000 7fbc94e20640 1 -- 172.21.15.82:0/3460597763 wait complete. 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:14:11.395 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:14:11.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:11.395 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:11.415 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-14T12:14:11.416 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:11.416 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-14T12:14:11.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:11.417 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:11.417 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:11.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add name ceph-brx type bridge 2024-03-14T12:14:11.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr flush dev ceph-brx 2024-03-14T12:14:11.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set ceph-brx up 2024-03-14T12:14:11.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-14T12:14:11.417 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:11.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:11.669 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:14:11.696 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-14T12:14:11.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:11.699 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-14T12:14:11.772 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-14T12:14:11.772 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:14:11.772 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:14:11.772 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:14:11.772 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:14:11.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:11.773 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:11.773 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:11.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:14:11.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:14:11.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:14:11.773 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:12.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.042 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:14:12.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.057 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-03-14T12:14:12.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.112 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:12.112 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:12.113 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T12:14:12.113 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-14T12:14:12.113 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:12.196 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-14T12:14:12.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-14T12:14:12.197 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:12.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.338 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:12.338 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:12.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-03-14T12:14:12.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.0 master ceph-brx 2024-03-14T12:14:12.338 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:12.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:12.380 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T12:14:12.435 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:14:12.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:12.435 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:14:12.490 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T12:14:12.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.490 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:12.575 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:12.575 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:12.697 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:12.697 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:12.697 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:14:12.697 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with new device syntax: 1@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:14:12.697 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:14:12.698 INFO:teuthology.orchestra.run.smithi164.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-14T12:14:12.698 INFO:teuthology.orchestra.run.smithi164.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:14:12.698 INFO:teuthology.orchestra.run.smithi164.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:14:12.698 INFO:teuthology.orchestra.run.smithi164.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:14:12.698 INFO:teuthology.orchestra.run.smithi164.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:14:12.700 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:14:12.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.700 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:14:12.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.727 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:12.783 INFO:teuthology.orchestra.run.smithi164.stdout:client4448 2024-03-14T12:14:12.783 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:12.783 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:12.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.838 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:12.894 INFO:teuthology.orchestra.run.smithi164.stdout:client4448 2024-03-14T12:14:12.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.894 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:12.950 INFO:teuthology.orchestra.run.smithi164.stdout:client4448 2024-03-14T12:14:12.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:12.950 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.client4448/status') 2024-03-14T12:14:13.017 INFO:teuthology.orchestra.run.smithi164.stdout:instance: client.4448 (3)192.168.0.1:0/1604686846 2024-03-14T12:14:13.017 INFO:teuthology.orchestra.run.smithi164.stdout:blocklisted: false 2024-03-14T12:14:13.017 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records in 2024-03-14T12:14:13.017 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records out 2024-03-14T12:14:13.017 INFO:teuthology.orchestra.run.smithi164.stderr:69 bytes copied, 2.0697e-05 s, 3.3 MB/s 2024-03-14T12:14:13.018 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-14T12:14:13.018 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-14T12:14:13.018 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-14T12:14:13.019 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:14:13.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.131+0000 7efea3c4d640 1 Processor -- start 2024-03-14T12:14:13.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.131+0000 7efea3c4d640 1 -- start start 2024-03-14T12:14:13.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea3c4d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe9c15e0e0 0x7efe9c0d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea3c4d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe9c0b3e10 0x7efe9c0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea3c4d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9c15e4c0 con 0x7efe9c0d4a20 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea3c4d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0d1640 con 0x7efe9c0b3e10 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea3c4d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0cf2f0 con 0x7efe9c15e0e0 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea21ea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe9c0b3e10 0x7efe9c0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea21ea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe9c0b3e10 0x7efe9c0b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47144/0 (socket says 172.21.15.82:47144) 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea21ea640 1 -- 172.21.15.82:0/3722203296 learned_addr learned my addr 172.21.15.82:0/3722203296 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea19e9640 1 --2- 172.21.15.82:0/3722203296 >> v2:172.21.15.164:3300/0 conn(0x7efe9c15e0e0 0x7efe9c0d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea21ea640 1 -- 172.21.15.82:0/3722203296 >> v2:172.21.15.164:3300/0 conn(0x7efe9c15e0e0 msgr2=0x7efe9c0d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea21ea640 1 --2- 172.21.15.82:0/3722203296 >> v2:172.21.15.164:3300/0 conn(0x7efe9c15e0e0 0x7efe9c0d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea21ea640 1 -- 172.21.15.82:0/3722203296 >> v1:172.21.15.164:6789/0 conn(0x7efe9c0d4a20 legacy=0x7efe9c0d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:13.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.132+0000 7efea21ea640 1 -- 172.21.15.82:0/3722203296 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0cf5d0 con 0x7efe9c0b3e10 2024-03-14T12:14:13.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.133+0000 7efea21ea640 1 --2- 172.21.15.82:0/3722203296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c0b3e10 0x7efe9c0b4240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe9c08dca0 tx=0x7efe9402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11195a62eacfd65 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.133+0000 7efea09e7640 1 -- 172.21.15.82:0/3722203296 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9402cb20 con 0x7efe9c0b3e10 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.133+0000 7efea09e7640 1 -- 172.21.15.82:0/3722203296 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9402ccc0 con 0x7efe9c0b3e10 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.133+0000 7efea09e7640 1 -- 172.21.15.82:0/3722203296 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94034610 con 0x7efe9c0b3e10 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 -- 172.21.15.82:0/3722203296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c0b3e10 msgr2=0x7efe9c0b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3722203296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c0b3e10 0x7efe9c0b4240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe9c08dca0 tx=0x7efe9402bf80 comp rx=0 tx=0).stop 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 -- 172.21.15.82:0/3722203296 shutdown_connections 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3722203296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c0b3e10 0x7efe9c0b4240 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3722203296 >> v2:172.21.15.164:3300/0 conn(0x7efe9c15e0e0 0x7efe9c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 -- 172.21.15.82:0/3722203296 >> 172.21.15.82:0/3722203296 conn(0x7efe9c0d3d40 msgr2=0x7efe9c0d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 -- 172.21.15.82:0/3722203296 shutdown_connections 2024-03-14T12:14:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.134+0000 7efea3c4d640 1 -- 172.21.15.82:0/3722203296 wait complete. 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.135+0000 7efea3c4d640 1 Processor -- start 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.135+0000 7efea3c4d640 1 -- start start 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea3c4d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe9c0b3e10 0x7efe9c1f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea3c4d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe9c0d4a20 0x7efe9c0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea3c4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c15e0e0 0x7efe9c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea3c4d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b3610 con 0x7efe9c15e0e0 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea3c4d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0cfb90 con 0x7efe9c0b3e10 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea3c4d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b38f0 con 0x7efe9c0d4a20 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c15e0e0 0x7efe9c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c15e0e0 0x7efe9c0c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47156/0 (socket says 172.21.15.82:47156) 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 -- 172.21.15.82:0/3135878846 learned_addr learned my addr 172.21.15.82:0/3135878846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea19e9640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe9c0b3e10 0x7efe9c1f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea11e8640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe9c0d4a20 0x7efe9c0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 -- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe9c0d4a20 msgr2=0x7efe9c0c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe9c0d4a20 0x7efe9c0c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 -- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe9c0b3e10 msgr2=0x7efe9c1f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe9c0b3e10 0x7efe9c1f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.136+0000 7efea21ea640 1 -- 172.21.15.82:0/3135878846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0ca680 con 0x7efe9c15e0e0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.137+0000 7efea21ea640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c15e0e0 0x7efe9c0c9dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe9402c590 tx=0x7efe94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.137+0000 7efe927fc640 1 -- 172.21.15.82:0/3135878846 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94043020 con 0x7efe9c15e0e0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.137+0000 7efe927fc640 1 -- 172.21.15.82:0/3135878846 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9403e450 con 0x7efe9c15e0e0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.137+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c1f7590 con 0x7efe9c15e0e0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.137+0000 7efe927fc640 1 -- 172.21.15.82:0/3135878846 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94009510 con 0x7efe9c15e0e0 2024-03-14T12:14:13.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.137+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c1f7b50 con 0x7efe9c15e0e0 2024-03-14T12:14:13.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.138+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c04f050 con 0x7efe9c15e0e0 2024-03-14T12:14:13.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.138+0000 7efe927fc640 1 -- 172.21.15.82:0/3135878846 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7efe94046020 con 0x7efe9c15e0e0 2024-03-14T12:14:13.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.139+0000 7efe927fc640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe8006bbd0 0x7efe8006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:13.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.139+0000 7efe927fc640 1 -- 172.21.15.82:0/3135878846 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe940ace60 con 0x7efe9c15e0e0 2024-03-14T12:14:13.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.139+0000 7efea19e9640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe8006bbd0 0x7efe8006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:13.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.139+0000 7efea19e9640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe8006bbd0 0x7efe8006e090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efe98006fd0 tx=0x7efe98008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:13.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.143+0000 7efe927fc640 1 -- 172.21.15.82:0/3135878846 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7efe9404d070 con 0x7efe9c15e0e0 2024-03-14T12:14:13.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.351+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efe9c06b760 con 0x7efe9c15e0e0 2024-03-14T12:14:13.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.351+0000 7efe927fc640 1 -- 172.21.15.82:0/3135878846 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7efe9407b300 con 0x7efe9c15e0e0 2024-03-14T12:14:13.351 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe8006bbd0 msgr2=0x7efe8006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe8006bbd0 0x7efe8006e090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efe98006fd0 tx=0x7efe98008040 comp rx=0 tx=0).stop 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c15e0e0 msgr2=0x7efe9c0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c15e0e0 0x7efe9c0c9dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe9402c590 tx=0x7efe94004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 shutdown_connections 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c15e0e0 0x7efe9c0c9dc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe8006bbd0 0x7efe8006e090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe9c0d4a20 0x7efe9c0c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 --2- 172.21.15.82:0/3135878846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe9c0b3e10 0x7efe9c1f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.353+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 >> 172.21.15.82:0/3135878846 conn(0x7efe9c0d3d40 msgr2=0x7efe9c06c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.354+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 shutdown_connections 2024-03-14T12:14:13.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:13.354+0000 7efea3c4d640 1 -- 172.21.15.82:0/3135878846 wait complete. 2024-03-14T12:14:13.363 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-14T12:14:13.363 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:14:13.363 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-14T12:14:13.363 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-14T12:14:13.364 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-14T12:14:13.364 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-14T12:14:13.364 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-14T12:14:13.364 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:14:13.364 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-14T12:14:13.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.364 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-14T12:14:13.380 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42703sec preferred_lft 42703sec 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::882:aaff:fed8:c2be/64 scope link 2024-03-14T12:14:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.382 INFO:teuthology.orchestra.run.smithi164.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:14:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T12:14:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::dc5a:2ff:fe9f:88ac/64 scope link tentative 2024-03-14T12:14:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.382 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:14:13.446 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-14T12:14:13.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.448 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-14T12:14:13.504 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-14T12:14:13.504 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:14:13.504 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:14:13.504 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:14:13.504 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:14:13.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.504 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:13.505 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:13.505 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:14:13.505 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:14:13.505 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:14:13.505 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:13.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.600 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:14:13.614 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T12:14:13.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.615 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-03-14T12:14:13.669 INFO:teuthology.orchestra.run.smithi164.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-14T12:14:13.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.670 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:13.670 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:13.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-14T12:14:13.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-14T12:14:13.670 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:13.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-14T12:14:13.819 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:13.819 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:13.819 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:13.819 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.819 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:13.819 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.820 INFO:teuthology.orchestra.run.smithi164.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:14:13.820 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 8e:d8:6f:74:52:b3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-14T12:14:13.820 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-14T12:14:13.820 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.820 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::8cd8:6fff:fe74:52b3/64 scope link tentative 2024-03-14T12:14:13.820 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.821 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-14T12:14:13.887 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 8e:d8:6f:74:52:b3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::8cd8:6fff:fe74:52b3/64 scope link 2024-03-14T12:14:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:13.889 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-14T12:14:13.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:13.889 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:13.889 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:13.889 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-14T12:14:13.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-14T12:14:13.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-14T12:14:13.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-14T12:14:13.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-14T12:14:13.890 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:14.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:14.015 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:14:14.015 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:14:14.015 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.1 up 2024-03-14T12:14:14.015 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.1 master ceph-brx 2024-03-14T12:14:14.015 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:14:14.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:14.057 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-14T12:14:14.111 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-14T12:14:14.112 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:14.112 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-14T12:14:14.166 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-14T12:14:14.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:14.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:14.250 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:14.250 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:14.257 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:14.257 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:14.257 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:14:14.257 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with new device syntax: 3@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:14:14.257 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:14:14.258 INFO:teuthology.orchestra.run.smithi164.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-14T12:14:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:14:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:14:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:14:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:14:14.260 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:14:14.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:14.260 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-14T12:14:14.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:14.287 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:14.343 INFO:teuthology.orchestra.run.smithi164.stdout:client4591 2024-03-14T12:14:14.343 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:14.343 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:14.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:14.399 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:14.454 INFO:teuthology.orchestra.run.smithi164.stdout:client4591 2024-03-14T12:14:14.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:14.455 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:14.511 INFO:teuthology.orchestra.run.smithi164.stdout:client4591 2024-03-14T12:14:14.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:14.511 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.client4591/status') 2024-03-14T12:14:14.578 INFO:teuthology.orchestra.run.smithi164.stdout:instance: client.4591 (3)192.168.0.2:0/98544970 2024-03-14T12:14:14.578 INFO:teuthology.orchestra.run.smithi164.stdout:blocklisted: false 2024-03-14T12:14:14.580 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records in 2024-03-14T12:14:14.580 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records out 2024-03-14T12:14:14.580 INFO:teuthology.orchestra.run.smithi164.stderr:67 bytes copied, 2.0257e-05 s, 3.3 MB/s 2024-03-14T12:14:14.580 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-14T12:14:14.598 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.711+0000 7f86c08fc640 1 Processor -- start 2024-03-14T12:14:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.711+0000 7f86c08fc640 1 -- start start 2024-03-14T12:14:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.711+0000 7f86c08fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f86bc0cf370 0x7f86bc0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.711+0000 7f86c08fc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0cf8c0 con 0x7f86bc0d0760 2024-03-14T12:14:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.711+0000 7f86c08fc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0cfba0 con 0x7f86bc0b7720 2024-03-14T12:14:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.711+0000 7f86c08fc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc0b6c40 con 0x7f86bc0cf370 2024-03-14T12:14:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.711+0000 7f86b3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f86bc0cf370 0x7f86bc0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.712+0000 7f86ba59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f86bc0d0760 0x7f86bc0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51644/0 (socket says 172.21.15.82:51644) 2024-03-14T12:14:14.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.712+0000 7f86ba59c640 1 -- 172.21.15.82:0/2905600052 learned_addr learned my addr 172.21.15.82:0/2905600052 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:14.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.712+0000 7f86b3fff640 1 -- 172.21.15.82:0/2905600052 >> v1:172.21.15.164:6789/0 conn(0x7f86bc0b7720 legacy=0x7f86bc0b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:14.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.712+0000 7f86b3fff640 1 -- 172.21.15.82:0/2905600052 >> v1:172.21.15.82:6789/0 conn(0x7f86bc0d0760 legacy=0x7f86bc0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:14.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.712+0000 7f86b3fff640 1 -- 172.21.15.82:0/2905600052 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0b6f20 con 0x7f86bc0cf370 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86b3fff640 1 --2- 172.21.15.82:0/2905600052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0d4660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86a40097d0 tx=0x7f86a402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2ee0e416a304b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86b9d9b640 1 -- 172.21.15.82:0/2905600052 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4034070 con 0x7f86bc0cf370 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86b9d9b640 1 -- 172.21.15.82:0/2905600052 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a4030610 con 0x7f86bc0cf370 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86b9d9b640 1 -- 172.21.15.82:0/2905600052 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a40309e0 con 0x7f86bc0cf370 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86c08fc640 1 -- 172.21.15.82:0/2905600052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 msgr2=0x7f86bc0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86c08fc640 1 --2- 172.21.15.82:0/2905600052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0d4660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86a40097d0 tx=0x7f86a402b960 comp rx=0 tx=0).stop 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86c08fc640 1 -- 172.21.15.82:0/2905600052 shutdown_connections 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86c08fc640 1 --2- 172.21.15.82:0/2905600052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0d4660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86c08fc640 1 -- 172.21.15.82:0/2905600052 >> 172.21.15.82:0/2905600052 conn(0x7f86bc09d030 msgr2=0x7f86bc0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86c08fc640 1 -- 172.21.15.82:0/2905600052 shutdown_connections 2024-03-14T12:14:14.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.713+0000 7f86c08fc640 1 -- 172.21.15.82:0/2905600052 wait complete. 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 Processor -- start 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 -- start start 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86bc0b7720 0x7f86bc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86bc0d0760 0x7f86bc0cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1617a0 con 0x7f86bc0cf370 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0d4ba0 con 0x7f86bc0b7720 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86c08fc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc161920 con 0x7f86bc0d0760 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86b3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.715+0000 7f86b3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47164/0 (socket says 172.21.15.82:47164) 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b3fff640 1 -- 172.21.15.82:0/273343471 learned_addr learned my addr 172.21.15.82:0/273343471 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86ba59c640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86bc0b7720 0x7f86bc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:14.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86bad9d640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86bc0d0760 0x7f86bc0cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b3fff640 1 -- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86bc0d0760 msgr2=0x7f86bc0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b3fff640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86bc0d0760 0x7f86bc0cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b3fff640 1 -- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86bc0b7720 msgr2=0x7f86bc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b3fff640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86bc0b7720 0x7f86bc15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b3fff640 1 -- 172.21.15.82:0/273343471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0b4780 con 0x7f86bc0cf370 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b3fff640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0ccbb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86a40098f0 tx=0x7f86a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b2ffd640 1 -- 172.21.15.82:0/273343471 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4034040 con 0x7f86bc0cf370 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b2ffd640 1 -- 172.21.15.82:0/273343471 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a4039a10 con 0x7f86bc0cf370 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86b2ffd640 1 -- 172.21.15.82:0/273343471 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4039e00 con 0x7f86bc0cf370 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc1f43b0 con 0x7f86bc0cf370 2024-03-14T12:14:14.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.716+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc1f4970 con 0x7f86bc0cf370 2024-03-14T12:14:14.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.717+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc04f050 con 0x7f86bc0cf370 2024-03-14T12:14:14.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.718+0000 7f86b2ffd640 1 -- 172.21.15.82:0/273343471 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f86a4037070 con 0x7f86bc0cf370 2024-03-14T12:14:14.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.718+0000 7f86b2ffd640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f869c06bbd0 0x7f869c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:14.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.718+0000 7f86b2ffd640 1 -- 172.21.15.82:0/273343471 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a40acae0 con 0x7f86bc0cf370 2024-03-14T12:14:14.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.718+0000 7f86ba59c640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f869c06bbd0 0x7f869c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:14.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.718+0000 7f86ba59c640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f869c06bbd0 0x7f869c06e090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86bc06c160 tx=0x7f86ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:14.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.722+0000 7f86b2ffd640 1 -- 172.21.15.82:0/273343471 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f86bc0b4780 con 0x7f86bc0cf370 2024-03-14T12:14:14.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.896+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86bc0d0540 con 0x7f86bc0cf370 2024-03-14T12:14:14.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.897+0000 7f86b2ffd640 1 -- 172.21.15.82:0/273343471 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f86a407b660 con 0x7f86bc0cf370 2024-03-14T12:14:14.896 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:14.896 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f869c06bbd0 msgr2=0x7f869c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f869c06bbd0 0x7f869c06e090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86bc06c160 tx=0x7f86ac009040 comp rx=0 tx=0).stop 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 msgr2=0x7f86bc0ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0ccbb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86a40098f0 tx=0x7f86a40046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 shutdown_connections 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f869c06bbd0 0x7f869c06e090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86bc0d0760 0x7f86bc0cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86bc0cf370 0x7f86bc0ccbb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 --2- 172.21.15.82:0/273343471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86bc0b7720 0x7f86bc15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:14.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 >> 172.21.15.82:0/273343471 conn(0x7f86bc09d030 msgr2=0x7f86bc0d2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:14.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 shutdown_connections 2024-03-14T12:14:14.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:14.899+0000 7f86c08fc640 1 -- 172.21.15.82:0/273343471 wait complete. 2024-03-14T12:14:14.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:14.909 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:15.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.013+0000 7f6a6244b640 1 Processor -- start 2024-03-14T12:14:15.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.013+0000 7f6a6244b640 1 -- start start 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a6244b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6a5c15dd70 0x7f6a5c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a6244b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a5c0d1b00 0x7f6a5c0d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a6244b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c070250 con 0x7f6a5c15e150 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a6244b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d2490 con 0x7f6a5c15dd70 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a6244b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c06ea20 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6a5c15dd70 0x7f6a5c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a5c15e150 0x7f6a5c0b6690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51660/0 (socket says 172.21.15.82:51660) 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5b7fe640 1 -- 172.21.15.82:0/202679096 learned_addr learned my addr 172.21.15.82:0/202679096 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6a5c15dd70 0x7f6a5c06fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47170/0 (socket says 172.21.15.82:47170) 2024-03-14T12:14:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a609e8640 1 --2- 172.21.15.82:0/202679096 >> v2:172.21.15.164:3300/0 conn(0x7f6a5c0d1b00 0x7f6a5c0d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5bfff640 1 -- 172.21.15.82:0/202679096 >> v2:172.21.15.164:3300/0 conn(0x7f6a5c0d1b00 msgr2=0x7f6a5c0d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5bfff640 1 --2- 172.21.15.82:0/202679096 >> v2:172.21.15.164:3300/0 conn(0x7f6a5c0d1b00 0x7f6a5c0d1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5bfff640 1 -- 172.21.15.82:0/202679096 >> v1:172.21.15.82:6789/0 conn(0x7f6a5c15e150 legacy=0x7f6a5c0b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.014+0000 7f6a5bfff640 1 -- 172.21.15.82:0/202679096 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c06ec70 con 0x7f6a5c15dd70 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.015+0000 7f6a5bfff640 1 --2- 172.21.15.82:0/202679096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c15dd70 0x7f6a5c06fd10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c072910 tx=0x7f6a4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1ce25afe2f52f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.015+0000 7f6a5affd640 1 -- 172.21.15.82:0/202679096 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c035020 con 0x7f6a5c15dd70 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.015+0000 7f6a5affd640 1 -- 172.21.15.82:0/202679096 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c031830 con 0x7f6a5c15dd70 2024-03-14T12:14:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.015+0000 7f6a5affd640 1 -- 172.21.15.82:0/202679096 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c031c00 con 0x7f6a5c15dd70 2024-03-14T12:14:15.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.015+0000 7f6a6244b640 1 -- 172.21.15.82:0/202679096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c15dd70 msgr2=0x7f6a5c06fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.015+0000 7f6a6244b640 1 --2- 172.21.15.82:0/202679096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c15dd70 0x7f6a5c06fd10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c072910 tx=0x7f6a4c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.016+0000 7f6a6244b640 1 -- 172.21.15.82:0/202679096 shutdown_connections 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.016+0000 7f6a6244b640 1 --2- 172.21.15.82:0/202679096 >> v2:172.21.15.164:3300/0 conn(0x7f6a5c0d1b00 0x7f6a5c0d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.016+0000 7f6a6244b640 1 --2- 172.21.15.82:0/202679096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c15dd70 0x7f6a5c06fd10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.016+0000 7f6a6244b640 1 -- 172.21.15.82:0/202679096 >> 172.21.15.82:0/202679096 conn(0x7f6a5c072ec0 msgr2=0x7f6a5c09a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.016+0000 7f6a6244b640 1 -- 172.21.15.82:0/202679096 shutdown_connections 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.016+0000 7f6a6244b640 1 -- 172.21.15.82:0/202679096 wait complete. 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.017+0000 7f6a6244b640 1 Processor -- start 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.017+0000 7f6a6244b640 1 -- start start 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.017+0000 7f6a6244b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c0d1b00 0x7f6a5c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a6244b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5c15dd70 0x7f6a5c0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a6244b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5c15e150 0x7f6a5c0d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a6244b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c06f230 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a6244b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c06f880 con 0x7f6a5c15e150 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a6244b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c06f510 con 0x7f6a5c15dd70 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c0d1b00 0x7f6a5c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c0d1b00 0x7f6a5c1593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47174/0 (socket says 172.21.15.82:47174) 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 -- 172.21.15.82:0/694967650 learned_addr learned my addr 172.21.15.82:0/694967650 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 -- 172.21.15.82:0/694967650 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5c15dd70 msgr2=0x7f6a5c0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5c15dd70 0x7f6a5c0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 -- 172.21.15.82:0/694967650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5c15e150 msgr2=0x7f6a5c0d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5c15e150 0x7f6a5c0d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 -- 172.21.15.82:0/694967650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0b1220 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a5bfff640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c0d1b00 0x7f6a5c1593d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009890 tx=0x7f6a4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a58ff9640 1 -- 172.21.15.82:0/694967650 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c041070 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a58ff9640 1 -- 172.21.15.82:0/694967650 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c03aa10 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a58ff9640 1 -- 172.21.15.82:0/694967650 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c035040 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c0b1500 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.018+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c0b1ac0 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.019+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c04f050 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.020+0000 7f6a58ff9640 1 -- 172.21.15.82:0/694967650 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6a4c038020 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.020+0000 7f6a58ff9640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4806bc00 0x7f6a4806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.020+0000 7f6a58ff9640 1 -- 172.21.15.82:0/694967650 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a4c0ac4d0 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.020+0000 7f6a5b7fe640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4806bc00 0x7f6a4806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.020+0000 7f6a5b7fe640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4806bc00 0x7f6a4806e0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005880 tx=0x7f6a50007aa0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.024+0000 7f6a58ff9640 1 -- 172.21.15.82:0/694967650 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6a4c07b050 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.196+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a5c0d29e0 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.196+0000 7f6a58ff9640 1 -- 172.21.15.82:0/694967650 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a4c0aae20 con 0x7f6a5c0d1b00 2024-03-14T12:14:15.195 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:15.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4806bc00 msgr2=0x7f6a4806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4806bc00 0x7f6a4806e0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005880 tx=0x7f6a50007aa0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c0d1b00 msgr2=0x7f6a5c1593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c0d1b00 0x7f6a5c1593d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009890 tx=0x7f6a4c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 shutdown_connections 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5c15e150 0x7f6a5c0d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4806bc00 0x7f6a4806e0c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5c15dd70 0x7f6a5c0d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 --2- 172.21.15.82:0/694967650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a5c0d1b00 0x7f6a5c1593d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 >> 172.21.15.82:0/694967650 conn(0x7f6a5c072ec0 msgr2=0x7f6a5c15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 shutdown_connections 2024-03-14T12:14:15.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.198+0000 7f6a6244b640 1 -- 172.21.15.82:0/694967650 wait complete. 2024-03-14T12:14:15.208 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:15.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.312+0000 7fc17e48c640 1 Processor -- start 2024-03-14T12:14:15.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.312+0000 7fc17e48c640 1 -- start start 2024-03-14T12:14:15.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.312+0000 7fc17e48c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 0x7fc1780d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.312+0000 7fc17e48c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc1780d12a0 0x7fc1780d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.312+0000 7fc17e48c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1780d25a0 con 0x7fc1780d0c40 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.312+0000 7fc17e48c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780d10a0 con 0x7fc1780d12a0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.312+0000 7fc17e48c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d2350 con 0x7fc17815dff0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc1780d12a0 0x7fc1780d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc1780d12a0 0x7fc1780d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47176/0 (socket says 172.21.15.82:47176) 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 -- 172.21.15.82:0/1692938117 learned_addr learned my addr 172.21.15.82:0/1692938117 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc177fff640 1 --2- 172.21.15.82:0/1692938117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 0x7fc1780d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 -- 172.21.15.82:0/1692938117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 msgr2=0x7fc1780d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 --2- 172.21.15.82:0/1692938117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 0x7fc1780d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 -- 172.21.15.82:0/1692938117 >> v1:172.21.15.164:6789/0 conn(0x7fc1780d0c40 legacy=0x7fc1780ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 -- 172.21.15.82:0/1692938117 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780d2880 con 0x7fc1780d12a0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.313+0000 7fc1777fe640 1 --2- 172.21.15.82:0/1692938117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d12a0 0x7fc1780d0700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009870 tx=0x7fc16c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d498dd192d571ec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc176ffd640 1 -- 172.21.15.82:0/1692938117 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c034070 con 0x7fc1780d12a0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc176ffd640 1 -- 172.21.15.82:0/1692938117 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c030840 con 0x7fc1780d12a0 2024-03-14T12:14:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc176ffd640 1 -- 172.21.15.82:0/1692938117 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c030c10 con 0x7fc1780d12a0 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 -- 172.21.15.82:0/1692938117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d12a0 msgr2=0x7fc1780d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1692938117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d12a0 0x7fc1780d0700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009870 tx=0x7fc16c02b980 comp rx=0 tx=0).stop 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 -- 172.21.15.82:0/1692938117 shutdown_connections 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1692938117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d12a0 0x7fc1780d0700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1692938117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 0x7fc1780d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 -- 172.21.15.82:0/1692938117 >> 172.21.15.82:0/1692938117 conn(0x7fc17809d0d0 msgr2=0x7fc1780d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 -- 172.21.15.82:0/1692938117 shutdown_connections 2024-03-14T12:14:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.314+0000 7fc17e48c640 1 -- 172.21.15.82:0/1692938117 wait complete. 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.315+0000 7fc17e48c640 1 Processor -- start 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc17e48c640 1 -- start start 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc17e48c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d0c40 0x7fc17815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc17e48c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1780d12a0 0x7fc1780c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc17e48c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 0x7fc1780c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc17e48c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d4b40 con 0x7fc1780d0c40 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc17e48c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d49c0 con 0x7fc1780d12a0 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc17e48c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17815e3d0 con 0x7fc17815dff0 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc177fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d0c40 0x7fc17815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc177fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d0c40 0x7fc17815bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47188/0 (socket says 172.21.15.82:47188) 2024-03-14T12:14:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.316+0000 7fc177fff640 1 -- 172.21.15.82:0/1811209407 learned_addr learned my addr 172.21.15.82:0/1811209407 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc177fff640 1 -- 172.21.15.82:0/1811209407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 msgr2=0x7fc1780c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc1777fe640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1780d12a0 0x7fc1780c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc177fff640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 0x7fc1780c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc177fff640 1 -- 172.21.15.82:0/1811209407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1780d12a0 msgr2=0x7fc1780c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc177fff640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1780d12a0 0x7fc1780c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc177fff640 1 -- 172.21.15.82:0/1811209407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780c8930 con 0x7fc1780d0c40 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc177fff640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d0c40 0x7fc17815bdb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc16800d800 tx=0x7fc16800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc174ff9640 1 -- 172.21.15.82:0/1811209407 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16800b6c0 con 0x7fc1780d0c40 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc174ff9640 1 -- 172.21.15.82:0/1811209407 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc168019070 con 0x7fc1780d0c40 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1780c8c10 con 0x7fc1780d0c40 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc174ff9640 1 -- 172.21.15.82:0/1811209407 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16800ba80 con 0x7fc1780d0c40 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.317+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1780c91d0 con 0x7fc1780d0c40 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.318+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17804f050 con 0x7fc1780d0c40 2024-03-14T12:14:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.318+0000 7fc174ff9640 1 -- 172.21.15.82:0/1811209407 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc16800bc20 con 0x7fc1780d0c40 2024-03-14T12:14:15.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.319+0000 7fc174ff9640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc16406bbd0 0x7fc16406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.319+0000 7fc174ff9640 1 -- 172.21.15.82:0/1811209407 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc16808aed0 con 0x7fc1780d0c40 2024-03-14T12:14:15.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.319+0000 7fc1777fe640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc16406bbd0 0x7fc16406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.319+0000 7fc1777fe640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc16406bbd0 0x7fc16406e090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc16c036000 tx=0x7fc16c03b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.323+0000 7fc174ff9640 1 -- 172.21.15.82:0/1811209407 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc168059a50 con 0x7fc1780d0c40 2024-03-14T12:14:15.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.495+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1780cc810 con 0x7fc1780d0c40 2024-03-14T12:14:15.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.495+0000 7fc174ff9640 1 -- 172.21.15.82:0/1811209407 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc168059350 con 0x7fc1780d0c40 2024-03-14T12:14:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:15.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc16406bbd0 msgr2=0x7fc16406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc16406bbd0 0x7fc16406e090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc16c036000 tx=0x7fc16c03b040 comp rx=0 tx=0).stop 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d0c40 msgr2=0x7fc17815bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d0c40 0x7fc17815bdb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc16800d800 tx=0x7fc16800dcc0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 shutdown_connections 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc16406bbd0 0x7fc16406e090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc17815dff0 0x7fc1780c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1780d12a0 0x7fc1780c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 --2- 172.21.15.82:0/1811209407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1780d0c40 0x7fc17815bdb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 >> 172.21.15.82:0/1811209407 conn(0x7fc17809d0d0 msgr2=0x7fc1780cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 shutdown_connections 2024-03-14T12:14:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.497+0000 7fc17e48c640 1 -- 172.21.15.82:0/1811209407 wait complete. 2024-03-14T12:14:15.507 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:15.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.613+0000 7fb6b6a81640 1 Processor -- start 2024-03-14T12:14:15.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.613+0000 7fb6b6a81640 1 -- start start 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b6a81640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6b015dec0 0x7fb6b00d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b6a81640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb6b015e2a0 0x7fb6b00d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b6a81640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b00d1420 con 0x7fb6b00ca540 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b6a81640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b00d1700 con 0x7fb6b015dec0 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b6a81640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b00c80a0 con 0x7fb6b015e2a0 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b481d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6b015dec0 0x7fb6b00d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b481d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6b015dec0 0x7fb6b00d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47198/0 (socket says 172.21.15.82:47198) 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b481d640 1 -- 172.21.15.82:0/2456206763 learned_addr learned my addr 172.21.15.82:0/2456206763 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b501e640 1 --2- 172.21.15.82:0/2456206763 >> v2:172.21.15.164:3300/0 conn(0x7fb6b015e2a0 0x7fb6b00d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b481d640 1 -- 172.21.15.82:0/2456206763 >> v2:172.21.15.164:3300/0 conn(0x7fb6b015e2a0 msgr2=0x7fb6b00d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b481d640 1 --2- 172.21.15.82:0/2456206763 >> v2:172.21.15.164:3300/0 conn(0x7fb6b015e2a0 0x7fb6b00d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b481d640 1 -- 172.21.15.82:0/2456206763 >> v1:172.21.15.164:6789/0 conn(0x7fb6b00ca540 legacy=0x7fb6b00d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.614+0000 7fb6b481d640 1 -- 172.21.15.82:0/2456206763 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b00c7a50 con 0x7fb6b015dec0 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6b481d640 1 --2- 172.21.15.82:0/2456206763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b00d0dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00d3da0 tx=0x7fb6a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778e2a704ed4297 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6af7fe640 1 -- 172.21.15.82:0/2456206763 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a402e070 con 0x7fb6b015dec0 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6af7fe640 1 -- 172.21.15.82:0/2456206763 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a4037070 con 0x7fb6b015dec0 2024-03-14T12:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6af7fe640 1 -- 172.21.15.82:0/2456206763 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4031a00 con 0x7fb6b015dec0 2024-03-14T12:14:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2456206763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 msgr2=0x7fb6b00d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2456206763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b00d0dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00d3da0 tx=0x7fb6a402b880 comp rx=0 tx=0).stop 2024-03-14T12:14:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2456206763 shutdown_connections 2024-03-14T12:14:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.615+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2456206763 >> v2:172.21.15.164:3300/0 conn(0x7fb6b015e2a0 0x7fb6b00d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.616+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2456206763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b00d0dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.616+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2456206763 >> 172.21.15.82:0/2456206763 conn(0x7fb6b00d3dd0 msgr2=0x7fb6b00d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.616+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2456206763 shutdown_connections 2024-03-14T12:14:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.616+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2456206763 wait complete. 2024-03-14T12:14:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.617+0000 7fb6b6a81640 1 Processor -- start 2024-03-14T12:14:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.617+0000 7fb6b6a81640 1 -- start start 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6b00ca540 0x7fb6b015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b01f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6b015e2a0 0x7fb6b00cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00d2270 con 0x7fb6b015dec0 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00d20f0 con 0x7fb6b015e2a0 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00d23f0 con 0x7fb6b00ca540 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b01f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b01f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47204/0 (socket says 172.21.15.82:47204) 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 -- 172.21.15.82:0/2625462135 learned_addr learned my addr 172.21.15.82:0/2625462135 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b501e640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6b015e2a0 0x7fb6b00cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b481d640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6b00ca540 0x7fb6b015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 -- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6b00ca540 msgr2=0x7fb6b015a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6b00ca540 0x7fb6b015a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 -- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6b015e2a0 msgr2=0x7fb6b00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6b015e2a0 0x7fb6b00cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 -- 172.21.15.82:0/2625462135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b00ce080 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b481d640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6b00ca540 0x7fb6b015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6affff640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b01f3de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00b3c0 tx=0x7fb69c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6ad7fa640 1 -- 172.21.15.82:0/2625462135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c011830 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6ad7fa640 1 -- 172.21.15.82:0/2625462135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c016070 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b00b7d50 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6ad7fa640 1 -- 172.21.15.82:0/2625462135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c011c30 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.618+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b00b8310 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.619+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b004f050 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.620+0000 7fb6ad7fa640 1 -- 172.21.15.82:0/2625462135 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb69c01b460 con 0x7fb6b015dec0 2024-03-14T12:14:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.620+0000 7fb6ad7fa640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb69406bbd0 0x7fb69406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.620+0000 7fb6ad7fa640 1 -- 172.21.15.82:0/2625462135 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb69c08a6d0 con 0x7fb6b015dec0 2024-03-14T12:14:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.620+0000 7fb6b481d640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb69406bbd0 0x7fb69406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.621+0000 7fb6b481d640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb69406bbd0 0x7fb69406e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4036000 tx=0x7fb6a4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.625+0000 7fb6ad7fa640 1 -- 172.21.15.82:0/2625462135 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb69c059250 con 0x7fb6b015dec0 2024-03-14T12:14:15.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.797+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6b00d07a0 con 0x7fb6b015dec0 2024-03-14T12:14:15.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.798+0000 7fb6ad7fa640 1 -- 172.21.15.82:0/2625462135 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb69c058b50 con 0x7fb6b015dec0 2024-03-14T12:14:15.797 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:15.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb69406bbd0 msgr2=0x7fb69406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb69406bbd0 0x7fb69406e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4036000 tx=0x7fb6a4008030 comp rx=0 tx=0).stop 2024-03-14T12:14:15.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 msgr2=0x7fb6b01f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b01f3de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00b3c0 tx=0x7fb69c00b880 comp rx=0 tx=0).stop 2024-03-14T12:14:15.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 shutdown_connections 2024-03-14T12:14:15.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6b015e2a0 0x7fb6b00cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6b015dec0 0x7fb6b01f3de0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb69406bbd0 0x7fb69406e090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 --2- 172.21.15.82:0/2625462135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6b00ca540 0x7fb6b015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 >> 172.21.15.82:0/2625462135 conn(0x7fb6b00d3dd0 msgr2=0x7fb6b006cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:15.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 shutdown_connections 2024-03-14T12:14:15.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.800+0000 7fb6b6a81640 1 -- 172.21.15.82:0/2625462135 wait complete. 2024-03-14T12:14:15.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:15.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.911+0000 7fc17c12b640 1 Processor -- start 2024-03-14T12:14:15.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.911+0000 7fc17c12b640 1 -- start start 2024-03-14T12:14:15.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc17c12b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 0x7fc174070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc17c12b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740b3da0 con 0x7fc17415dde0 2024-03-14T12:14:15.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc17c12b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740b4080 con 0x7fc174151230 2024-03-14T12:14:15.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc17c12b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17406c3b0 con 0x7fc1740b4b70 2024-03-14T12:14:15.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc17a6c8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc17415dde0 0x7fc17406cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51674/0 (socket says 172.21.15.82:51674) 2024-03-14T12:14:15.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc17a6c8640 1 -- 172.21.15.82:0/3946840770 learned_addr learned my addr 172.21.15.82:0/3946840770 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894849756 0 0) 0x7fc1740b3da0 con 0x7fc17415dde0 2024-03-14T12:14:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 0x7fc174070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc160004630 con 0x7fc17415dde0 2024-03-14T12:14:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.912+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1157747235 0 0) 0x7fc160004630 con 0x7fc17415dde0 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 msgr2=0x7fc174070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 0x7fc174070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 >> v1:172.21.15.164:6789/0 conn(0x7fc174151230 legacy=0x7fc1740cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740b4360 con 0x7fc17415dde0 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fc16c0019f0 con 0x7fc17415dde0 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 >> v1:172.21.15.82:6789/0 conn(0x7fc17415dde0 legacy=0x7fc17406cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 0x7fc160004af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160005030 0x7fc160005910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc160005e50 0x7fc1600062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17406c3b0 con 0x7fc160005030 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000cbe0 con 0x7fc160005e50 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000cde0 con 0x7fc1740b4b70 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160005030 0x7fc160005910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc179ec7640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 0x7fc160004af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc17a6c8640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc160005e50 0x7fc1600062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc1796c6640 1 -- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 msgr2=0x7fc160004af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 0x7fc160004af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc1796c6640 1 -- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc160005e50 msgr2=0x7fc1600062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc160005e50 0x7fc1600062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc1796c6640 1 -- 172.21.15.82:0/3946840770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160004630 con 0x7fc160005030 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc17a6c8640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc160005e50 0x7fc1600062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.913+0000 7fc179ec7640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1740b4b70 0x7fc160004af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.914+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160005030 0x7fc160005910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc164009ac0 tx=0x7fc160006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3953ac08e33913b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.914+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164039070 con 0x7fc160005030 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.914+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16402c6b0 con 0x7fc160005030 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.914+0000 7fc178ec5640 1 -- 172.21.15.82:0/3946840770 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164009690 con 0x7fc160005030 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.915+0000 7fc17c12b640 1 -- 172.21.15.82:0/3946840770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160005030 msgr2=0x7fc160005910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.915+0000 7fc17c12b640 1 --2- 172.21.15.82:0/3946840770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160005030 0x7fc160005910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc164009ac0 tx=0x7fc160006940 comp rx=0 tx=0).stop 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.915+0000 7fc17a6c8640 1 -- 172.21.15.82:0/3946840770 reap_dead start 2024-03-14T12:14:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.915+0000 7fc17c12b640 1 -- 172.21.15.82:0/3946840770 shutdown_connections 2024-03-14T12:14:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.915+0000 7fc17c12b640 1 -- 172.21.15.82:0/3946840770 >> 172.21.15.82:0/3946840770 conn(0x7fc1740b1c70 msgr2=0x7fc1740cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.915+0000 7fc17c12b640 1 -- 172.21.15.82:0/3946840770 shutdown_connections 2024-03-14T12:14:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.915+0000 7fc17c12b640 1 -- 172.21.15.82:0/3946840770 wait complete. 2024-03-14T12:14:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.916+0000 7fc17c12b640 1 Processor -- start 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.916+0000 7fc17c12b640 1 -- start start 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17c12b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc160005030 0x7fc174150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17c12b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1741f3e00 0x7fc174152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17c12b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc174152f40 0x7fc1741eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17c12b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17406d230 con 0x7fc1741f3e00 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17c12b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17406c690 con 0x7fc174152f40 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17c12b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17406d3b0 con 0x7fc160005030 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1741f3e00 0x7fc174152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1741f3e00 0x7fc174152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47224/0 (socket says 172.21.15.82:47224) 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 -- 172.21.15.82:0/3534590033 learned_addr learned my addr 172.21.15.82:0/3534590033 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc179ec7640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc160005030 0x7fc174150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17a6c8640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc174152f40 0x7fc1741eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 -- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc160005030 msgr2=0x7fc174150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc160005030 0x7fc174150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 -- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc174152f40 msgr2=0x7fc1741eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc174152f40 0x7fc1741eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 -- 172.21.15.82:0/3534590033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1741536a0 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc179ec7640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc160005030 0x7fc174150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc17a6c8640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc174152f40 0x7fc1741eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.917+0000 7fc1796c6640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1741f3e00 0x7fc174152a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc164037040 tx=0x7fc164004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.918+0000 7fc16a7fc640 1 -- 172.21.15.82:0/3534590033 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164042020 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.918+0000 7fc16a7fc640 1 -- 172.21.15.82:0/3534590033 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16403a040 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.918+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1741f3320 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.918+0000 7fc16a7fc640 1 -- 172.21.15.82:0/3534590033 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164039070 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.918+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1741f38e0 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.919+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17404f050 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.919+0000 7fc16a7fc640 1 -- 172.21.15.82:0/3534590033 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc164045020 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.919+0000 7fc16a7fc640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc15806bbd0 0x7fc15806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.919+0000 7fc16a7fc640 1 -- 172.21.15.82:0/3534590033 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1640abf70 con 0x7fc1741f3e00 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.919+0000 7fc179ec7640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc15806bbd0 0x7fc15806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.920+0000 7fc179ec7640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc15806bbd0 0x7fc15806e090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc170001840 tx=0x7fc170009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:15.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:15.924+0000 7fc16a7fc640 1 -- 172.21.15.82:0/3534590033 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc16404c070 con 0x7fc1741f3e00 2024-03-14T12:14:16.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.097+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1740b2f40 con 0x7fc1741f3e00 2024-03-14T12:14:16.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.099+0000 7fc16a7fc640 1 -- 172.21.15.82:0/3534590033 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fc16407aa60 con 0x7fc1741f3e00 2024-03-14T12:14:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:16.098 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.100+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc15806bbd0 msgr2=0x7fc15806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.100+0000 7fc17c12b640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc15806bbd0 0x7fc15806e090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc170001840 tx=0x7fc170009040 comp rx=0 tx=0).stop 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.100+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1741f3e00 msgr2=0x7fc174152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.100+0000 7fc17c12b640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1741f3e00 0x7fc174152a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc164037040 tx=0x7fc164004730 comp rx=0 tx=0).stop 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.100+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 shutdown_connections 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.101+0000 7fc17c12b640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc174152f40 0x7fc1741eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.101+0000 7fc17c12b640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1741f3e00 0x7fc174152a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.101+0000 7fc17c12b640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc15806bbd0 0x7fc15806e090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.101+0000 7fc17c12b640 1 --2- 172.21.15.82:0/3534590033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc160005030 0x7fc174150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.101+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 >> 172.21.15.82:0/3534590033 conn(0x7fc1740b1c70 msgr2=0x7fc17414f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:16.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.101+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 shutdown_connections 2024-03-14T12:14:16.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.101+0000 7fc17c12b640 1 -- 172.21.15.82:0/3534590033 wait complete. 2024-03-14T12:14:16.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:16.111 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:16.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.215+0000 7f7e1b2e9640 1 Processor -- start 2024-03-14T12:14:16.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.215+0000 7f7e1b2e9640 1 -- start start 2024-03-14T12:14:16.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e1b2e9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e1415dd70 0x7f7e1406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e1b2e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1415e150 0x7f7e140b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e1b2e9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e14070250 con 0x7f7e140b6bd0 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e1b2e9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1406ea20 con 0x7f7e1415dd70 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e1b2e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1406f880 con 0x7f7e1415e150 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e19085640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e1415dd70 0x7f7e1406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e19085640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e1415dd70 0x7f7e1406fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47240/0 (socket says 172.21.15.82:47240) 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e19085640 1 -- 172.21.15.82:0/1148731515 learned_addr learned my addr 172.21.15.82:0/1148731515 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e19085640 1 -- 172.21.15.82:0/1148731515 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1415e150 msgr2=0x7f7e140b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e19085640 1 --2- 172.21.15.82:0/1148731515 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1415e150 0x7f7e140b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e19085640 1 -- 172.21.15.82:0/1148731515 >> v1:172.21.15.164:6789/0 conn(0x7f7e140b6bd0 legacy=0x7f7e140d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:16.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.216+0000 7f7e19085640 1 -- 172.21.15.82:0/1148731515 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1406ec70 con 0x7f7e1415dd70 2024-03-14T12:14:16.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.217+0000 7f7e19085640 1 --2- 172.21.15.82:0/1148731515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1406fd10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009870 tx=0x7f7e1002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cd8c78a450bae server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.217+0000 7f7e0bfff640 1 -- 172.21.15.82:0/1148731515 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10035020 con 0x7f7e1415dd70 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.217+0000 7f7e0bfff640 1 -- 172.21.15.82:0/1148731515 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e10031640 con 0x7f7e1415dd70 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.217+0000 7f7e0bfff640 1 -- 172.21.15.82:0/1148731515 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1002d040 con 0x7f7e1415dd70 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/1148731515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 msgr2=0x7f7e1406fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/1148731515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1406fd10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009870 tx=0x7f7e1002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/1148731515 shutdown_connections 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/1148731515 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1415e150 0x7f7e140b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/1148731515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1406fd10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/1148731515 >> 172.21.15.82:0/1148731515 conn(0x7f7e14072ec0 msgr2=0x7f7e1409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/1148731515 shutdown_connections 2024-03-14T12:14:16.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.218+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/1148731515 wait complete. 2024-03-14T12:14:16.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.219+0000 7f7e1b2e9640 1 Processor -- start 2024-03-14T12:14:16.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.219+0000 7f7e1b2e9640 1 -- start start 2024-03-14T12:14:16.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e1b2e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e140b6bd0 0x7f7e1415e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e1b2e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e1b2e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1415e150 0x7f7e14160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e1b2e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1406f510 con 0x7f7e1415dd70 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e1b2e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1406f230 con 0x7f7e1415e150 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e1b2e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140d2210 con 0x7f7e140b6bd0 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1415fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47244/0 (socket says 172.21.15.82:47244) 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 -- 172.21.15.82:0/960297682 learned_addr learned my addr 172.21.15.82:0/960297682 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e19085640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e140b6bd0 0x7f7e1415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e19886640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1415e150 0x7f7e14160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 -- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e140b6bd0 msgr2=0x7f7e1415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e140b6bd0 0x7f7e1415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 -- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1415e150 msgr2=0x7f7e14160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1415e150 0x7f7e14160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 -- 172.21.15.82:0/960297682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1415ebc0 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e18884640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1415fc60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400b410 tx=0x7f7e0400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e09ffb640 1 -- 172.21.15.82:0/960297682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04015020 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.220+0000 7f7e09ffb640 1 -- 172.21.15.82:0/960297682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e04011850 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.221+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1415eea0 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.221+0000 7f7e09ffb640 1 -- 172.21.15.82:0/960297682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04011c40 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.221+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1415f460 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.222+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1404f050 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.222+0000 7f7e09ffb640 1 -- 172.21.15.82:0/960297682 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7e04018020 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.222+0000 7f7e09ffb640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df806bbd0 0x7f7df806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.222+0000 7f7e09ffb640 1 -- 172.21.15.82:0/960297682 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e0408a990 con 0x7f7e1415dd70 2024-03-14T12:14:16.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.223+0000 7f7e19085640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df806bbd0 0x7f7df806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.223+0000 7f7e19085640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df806bbd0 0x7f7df806e090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e10036000 tx=0x7f7e1003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.227+0000 7f7e09ffb640 1 -- 172.21.15.82:0/960297682 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7e0401f020 con 0x7f7e1415dd70 2024-03-14T12:14:16.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.397+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e14159350 con 0x7f7e1415dd70 2024-03-14T12:14:16.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.398+0000 7f7e09ffb640 1 -- 172.21.15.82:0/960297682 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e040172c0 con 0x7f7e1415dd70 2024-03-14T12:14:16.397 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df806bbd0 msgr2=0x7f7df806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df806bbd0 0x7f7df806e090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e10036000 tx=0x7f7e1003c040 comp rx=0 tx=0).stop 2024-03-14T12:14:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 msgr2=0x7f7e1415fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1415fc60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400b410 tx=0x7f7e0400b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 shutdown_connections 2024-03-14T12:14:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1415e150 0x7f7e14160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e1415dd70 0x7f7e1415fc60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df806bbd0 0x7f7df806e090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 --2- 172.21.15.82:0/960297682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e140b6bd0 0x7f7e1415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 >> 172.21.15.82:0/960297682 conn(0x7f7e14072ec0 msgr2=0x7f7e1406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:16.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 shutdown_connections 2024-03-14T12:14:16.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.400+0000 7f7e1b2e9640 1 -- 172.21.15.82:0/960297682 wait complete. 2024-03-14T12:14:16.410 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:16.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.515+0000 7f2ecc84b640 1 Processor -- start 2024-03-14T12:14:16.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.515+0000 7f2ecc84b640 1 -- start start 2024-03-14T12:14:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ecc84b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ec815e0c0 0x7f2ec80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ecc84b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80d08c0 0x7f2ec80ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ecc84b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec80cf440 con 0x7f2ec80b0bc0 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ecc84b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec80cf1e0 con 0x7f2ec815e0c0 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ecc84b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80ccb60 con 0x7f2ec80d08c0 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ec659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ec815e0c0 0x7f2ec80cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ec659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ec815e0c0 0x7f2ec80cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47248/0 (socket says 172.21.15.82:47248) 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.516+0000 7f2ec659c640 1 -- 172.21.15.82:0/1104060117 learned_addr learned my addr 172.21.15.82:0/1104060117 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec659c640 1 -- 172.21.15.82:0/1104060117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80d08c0 msgr2=0x7f2ec80ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec6d9d640 1 --2- 172.21.15.82:0/1104060117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80d08c0 0x7f2ec80ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec659c640 1 --2- 172.21.15.82:0/1104060117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80d08c0 0x7f2ec80ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec659c640 1 -- 172.21.15.82:0/1104060117 >> v1:172.21.15.164:6789/0 conn(0x7f2ec80b0bc0 legacy=0x7f2ec80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec659c640 1 -- 172.21.15.82:0/1104060117 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec80cce40 con 0x7f2ec815e0c0 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec6d9d640 1 --2- 172.21.15.82:0/1104060117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80d08c0 0x7f2ec80ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec659c640 1 --2- 172.21.15.82:0/1104060117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec815e0c0 0x7f2ec80cd920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009e50 tx=0x7f2eb802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fbee8bd9e6ffe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.517+0000 7f2ec559a640 1 -- 172.21.15.82:0/1104060117 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb802ca70 con 0x7f2ec815e0c0 2024-03-14T12:14:16.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ec559a640 1 -- 172.21.15.82:0/1104060117 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb802cc10 con 0x7f2ec815e0c0 2024-03-14T12:14:16.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ec559a640 1 -- 172.21.15.82:0/1104060117 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80343f0 con 0x7f2ec815e0c0 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 -- 172.21.15.82:0/1104060117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec815e0c0 msgr2=0x7f2ec80cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/1104060117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec815e0c0 0x7f2ec80cd920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009e50 tx=0x7f2eb802bed0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 -- 172.21.15.82:0/1104060117 shutdown_connections 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/1104060117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80d08c0 0x7f2ec80ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/1104060117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec815e0c0 0x7f2ec80cd920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 -- 172.21.15.82:0/1104060117 >> 172.21.15.82:0/1104060117 conn(0x7f2ec80d1bd0 msgr2=0x7f2ec80d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 -- 172.21.15.82:0/1104060117 shutdown_connections 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.518+0000 7f2ecc84b640 1 -- 172.21.15.82:0/1104060117 wait complete. 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.519+0000 7f2ecc84b640 1 Processor -- start 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ecc84b640 1 -- start start 2024-03-14T12:14:16.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ecc84b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80b0bc0 0x7f2ec80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ecc84b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec80d08c0 0x7f2ec80b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ecc84b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ec815e0c0 0x7f2ec81f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ecc84b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815e4a0 con 0x7f2ec80d08c0 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ecc84b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80cd400 con 0x7f2ec815e0c0 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ecc84b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec806bbc0 con 0x7f2ec80b0bc0 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ec5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec80d08c0 0x7f2ec80b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ec5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec80d08c0 0x7f2ec80b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47256/0 (socket says 172.21.15.82:47256) 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ec5d9b640 1 -- 172.21.15.82:0/3972837387 learned_addr learned my addr 172.21.15.82:0/3972837387 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ec659c640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80b0bc0 0x7f2ec80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.520+0000 7f2ec6d9d640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ec815e0c0 0x7f2ec81f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec5d9b640 1 -- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80b0bc0 msgr2=0x7f2ec80b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec5d9b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80b0bc0 0x7f2ec80b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec5d9b640 1 -- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ec815e0c0 msgr2=0x7f2ec81f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec5d9b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ec815e0c0 0x7f2ec81f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec5d9b640 1 -- 172.21.15.82:0/3972837387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec80b4e20 con 0x7f2ec80d08c0 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec6d9d640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ec815e0c0 0x7f2ec81f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec659c640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80b0bc0 0x7f2ec80b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ec5d9b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec80d08c0 0x7f2ec80b7310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c430 tx=0x7f2ebc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2eb6ffd640 1 -- 172.21.15.82:0/3972837387 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011790 con 0x7f2ec80d08c0 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2eb6ffd640 1 -- 172.21.15.82:0/3972837387 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc017070 con 0x7f2ec80d08c0 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec80b5100 con 0x7f2ec80d08c0 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2eb6ffd640 1 -- 172.21.15.82:0/3972837387 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011bb0 con 0x7f2ec80d08c0 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.521+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec80b56c0 con 0x7f2ec80d08c0 2024-03-14T12:14:16.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.522+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec804f050 con 0x7f2ec80d08c0 2024-03-14T12:14:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.522+0000 7f2eb6ffd640 1 -- 172.21.15.82:0/3972837387 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2ebc01c460 con 0x7f2ec80d08c0 2024-03-14T12:14:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.523+0000 7f2eb6ffd640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2ea806bbd0 0x7f2ea806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.523+0000 7f2eb6ffd640 1 -- 172.21.15.82:0/3972837387 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ebc08a8c0 con 0x7f2ec80d08c0 2024-03-14T12:14:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.523+0000 7f2ec659c640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2ea806bbd0 0x7f2ea806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.523+0000 7f2ec659c640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2ea806bbd0 0x7f2ea806e090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8035000 tx=0x7f2eb8030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.527+0000 7f2eb6ffd640 1 -- 172.21.15.82:0/3972837387 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2ebc059440 con 0x7f2ec80d08c0 2024-03-14T12:14:16.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.702+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ec80d2b00 con 0x7f2ec80d08c0 2024-03-14T12:14:16.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.703+0000 7f2eb6ffd640 1 -- 172.21.15.82:0/3972837387 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ebc058d40 con 0x7f2ec80d08c0 2024-03-14T12:14:16.702 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2ea806bbd0 msgr2=0x7f2ea806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2ea806bbd0 0x7f2ea806e090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8035000 tx=0x7f2eb8030000 comp rx=0 tx=0).stop 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec80d08c0 msgr2=0x7f2ec80b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec80d08c0 0x7f2ec80b7310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c430 tx=0x7f2ebc00c8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 shutdown_connections 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ec815e0c0 0x7f2ec81f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ec80d08c0 0x7f2ec80b7310 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2ea806bbd0 0x7f2ea806e090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 --2- 172.21.15.82:0/3972837387 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2ec80b0bc0 0x7f2ec80b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 >> 172.21.15.82:0/3972837387 conn(0x7f2ec80d1bd0 msgr2=0x7f2ec80af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 shutdown_connections 2024-03-14T12:14:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.705+0000 7f2ecc84b640 1 -- 172.21.15.82:0/3972837387 wait complete. 2024-03-14T12:14:16.715 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:16.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.819+0000 7fdc22f0a640 1 Processor -- start 2024-03-14T12:14:16.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.819+0000 7fdc22f0a640 1 -- start start 2024-03-14T12:14:16.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc22f0a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc1c162220 0x7fdc1c0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc22f0a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c06ee00 con 0x7fdc1c161580 2024-03-14T12:14:16.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc22f0a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c06f0e0 con 0x7fdc1c06c900 2024-03-14T12:14:16.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc22f0a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c06be20 con 0x7fdc1c162220 2024-03-14T12:14:16.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc1bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc1c162220 0x7fdc1c0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc20ca6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdc1c161580 0x7fdc1c161b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51682/0 (socket says 172.21.15.82:51682) 2024-03-14T12:14:16.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc20ca6640 1 -- 172.21.15.82:0/2372142908 learned_addr learned my addr 172.21.15.82:0/2372142908 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:16.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc1bfff640 1 -- 172.21.15.82:0/2372142908 >> v1:172.21.15.164:6789/0 conn(0x7fdc1c06c900 legacy=0x7fdc1c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:16.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc1bfff640 1 -- 172.21.15.82:0/2372142908 >> v1:172.21.15.82:6789/0 conn(0x7fdc1c161580 legacy=0x7fdc1c161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc1bfff640 1 -- 172.21.15.82:0/2372142908 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c06c100 con 0x7fdc1c162220 2024-03-14T12:14:16.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.820+0000 7fdc1b7fe640 1 -- 172.21.15.82:0/2372142908 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58735685 0 0) 0x7fdc1c06ee00 con 0x7fdc1c161580 2024-03-14T12:14:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc1bfff640 1 --2- 172.21.15.82:0/2372142908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c162220 0x7fdc1c0b41a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc080097a0 tx=0x7fdc0802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3214fdad37e1d353 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc1b7fe640 1 -- 172.21.15.82:0/2372142908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08034070 con 0x7fdc1c162220 2024-03-14T12:14:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc1b7fe640 1 -- 172.21.15.82:0/2372142908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08030610 con 0x7fdc1c162220 2024-03-14T12:14:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc1b7fe640 1 -- 172.21.15.82:0/2372142908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc080309e0 con 0x7fdc1c162220 2024-03-14T12:14:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc22f0a640 1 -- 172.21.15.82:0/2372142908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c162220 msgr2=0x7fdc1c0b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/2372142908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c162220 0x7fdc1c0b41a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc080097a0 tx=0x7fdc0802b960 comp rx=0 tx=0).stop 2024-03-14T12:14:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc22f0a640 1 -- 172.21.15.82:0/2372142908 shutdown_connections 2024-03-14T12:14:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.821+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/2372142908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c162220 0x7fdc1c0b41a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.822+0000 7fdc22f0a640 1 -- 172.21.15.82:0/2372142908 >> 172.21.15.82:0/2372142908 conn(0x7fdc1c0b3680 msgr2=0x7fdc1c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.822+0000 7fdc22f0a640 1 -- 172.21.15.82:0/2372142908 shutdown_connections 2024-03-14T12:14:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.822+0000 7fdc22f0a640 1 -- 172.21.15.82:0/2372142908 wait complete. 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 Processor -- start 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 -- start start 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c06c900 0x7fdc1c0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc1c161580 0x7fdc1c0c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc1c162220 0x7fdc1c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c06eb60 con 0x7fdc1c06c900 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c06e9e0 con 0x7fdc1c162220 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.823+0000 7fdc22f0a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c162790 con 0x7fdc1c161580 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c06c900 0x7fdc1c0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c06c900 0x7fdc1c0b7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47274/0 (socket says 172.21.15.82:47274) 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 -- 172.21.15.82:0/1676738964 learned_addr learned my addr 172.21.15.82:0/1676738964 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc1bfff640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc1c161580 0x7fdc1c0c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc214a7640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc1c162220 0x7fdc1c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 -- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc1c161580 msgr2=0x7fdc1c0c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc1c161580 0x7fdc1c0c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 -- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc1c162220 msgr2=0x7fdc1c15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc1c162220 0x7fdc1c15c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 -- 172.21.15.82:0/1676738964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0c7b30 con 0x7fdc1c06c900 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc20ca6640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c06c900 0x7fdc1c0b7250 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c4d0 tx=0x7fdc1000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc197fa640 1 -- 172.21.15.82:0/1676738964 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10018070 con 0x7fdc1c06c900 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc197fa640 1 -- 172.21.15.82:0/1676738964 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10007500 con 0x7fdc1c06c900 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c0c7db0 con 0x7fdc1c06c900 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc197fa640 1 -- 172.21.15.82:0/1676738964 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004a20 con 0x7fdc1c06c900 2024-03-14T12:14:16.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.824+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0c8370 con 0x7fdc1c06c900 2024-03-14T12:14:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.825+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c0704c0 con 0x7fdc1c06c900 2024-03-14T12:14:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.826+0000 7fdc197fa640 1 -- 172.21.15.82:0/1676738964 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdc1001b070 con 0x7fdc1c06c900 2024-03-14T12:14:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.826+0000 7fdc197fa640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc0006bbd0 0x7fdc0006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.826+0000 7fdc1bfff640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc0006bbd0 0x7fdc0006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.826+0000 7fdc197fa640 1 -- 172.21.15.82:0/1676738964 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc10089d10 con 0x7fdc1c06c900 2024-03-14T12:14:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.827+0000 7fdc1bfff640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc0006bbd0 0x7fdc0006e090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc08036000 tx=0x7fdc0803b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:16.831+0000 7fdc197fa640 1 -- 172.21.15.82:0/1676738964 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdc1c0704c0 con 0x7fdc1c06c900 2024-03-14T12:14:17.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.001+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc1c04f050 con 0x7fdc1c06c900 2024-03-14T12:14:17.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.001+0000 7fdc197fa640 1 -- 172.21.15.82:0/1676738964 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc1c0c7b30 con 0x7fdc1c06c900 2024-03-14T12:14:17.000 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:17.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc0006bbd0 msgr2=0x7fdc0006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc0006bbd0 0x7fdc0006e090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc08036000 tx=0x7fdc0803b040 comp rx=0 tx=0).stop 2024-03-14T12:14:17.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c06c900 msgr2=0x7fdc1c0b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c06c900 0x7fdc1c0b7250 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c4d0 tx=0x7fdc1000c990 comp rx=0 tx=0).stop 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 shutdown_connections 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc1c162220 0x7fdc1c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc0006bbd0 0x7fdc0006e090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc1c161580 0x7fdc1c0c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 --2- 172.21.15.82:0/1676738964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c06c900 0x7fdc1c0b7250 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 >> 172.21.15.82:0/1676738964 conn(0x7fdc1c0b3680 msgr2=0x7fdc1c0b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 shutdown_connections 2024-03-14T12:14:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.003+0000 7fdc22f0a640 1 -- 172.21.15.82:0/1676738964 wait complete. 2024-03-14T12:14:17.013 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:17.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.114+0000 7f268c8ea640 1 Processor -- start 2024-03-14T12:14:17.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.114+0000 7f268c8ea640 1 -- start start 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f268c8ea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2688161a80 0x7f2688161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f268c8ea640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f268806f330 0x7f268806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f268c8ea640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26880b60f0 con 0x7f26881551d0 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f268c8ea640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26880b6d00 con 0x7f2688161a80 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f268c8ea640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2688162420 con 0x7f268806f330 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f267ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2688161a80 0x7f2688161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f268659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f26881551d0 0x7f26880b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51684/0 (socket says 172.21.15.82:51684) 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f268659c640 1 -- 172.21.15.82:0/3048452406 learned_addr learned my addr 172.21.15.82:0/3048452406 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f2686d9d640 1 --2- 172.21.15.82:0/3048452406 >> v2:172.21.15.164:3300/0 conn(0x7f268806f330 0x7f268806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f267ffff640 1 -- 172.21.15.82:0/3048452406 >> v2:172.21.15.164:3300/0 conn(0x7f268806f330 msgr2=0x7f268806f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:17.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f267ffff640 1 --2- 172.21.15.82:0/3048452406 >> v2:172.21.15.164:3300/0 conn(0x7f268806f330 0x7f268806f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f267ffff640 1 -- 172.21.15.82:0/3048452406 >> v1:172.21.15.82:6789/0 conn(0x7f26881551d0 legacy=0x7f26880b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f2685d9b640 1 -- 172.21.15.82:0/3048452406 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21473484 0 0) 0x7f26880b60f0 con 0x7f26881551d0 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f267ffff640 1 -- 172.21.15.82:0/3048452406 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880b63d0 con 0x7f2688161a80 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.115+0000 7f2686d9d640 1 --2- 172.21.15.82:0/3048452406 >> v2:172.21.15.164:3300/0 conn(0x7f268806f330 0x7f268806f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f267ffff640 1 --2- 172.21.15.82:0/3048452406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f2688161ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2688162b40 tx=0x7f267002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2797c1d71a0fb90f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f2685d9b640 1 -- 172.21.15.82:0/3048452406 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670035020 con 0x7f2688161a80 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f2685d9b640 1 -- 172.21.15.82:0/3048452406 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670031830 con 0x7f2688161a80 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f2685d9b640 1 -- 172.21.15.82:0/3048452406 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670031c00 con 0x7f2688161a80 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f268c8ea640 1 -- 172.21.15.82:0/3048452406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 msgr2=0x7f2688161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f268c8ea640 1 --2- 172.21.15.82:0/3048452406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f2688161ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2688162b40 tx=0x7f267002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f268c8ea640 1 -- 172.21.15.82:0/3048452406 shutdown_connections 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f268c8ea640 1 --2- 172.21.15.82:0/3048452406 >> v2:172.21.15.164:3300/0 conn(0x7f268806f330 0x7f268806f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f268c8ea640 1 --2- 172.21.15.82:0/3048452406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f2688161ee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f268c8ea640 1 -- 172.21.15.82:0/3048452406 >> 172.21.15.82:0/3048452406 conn(0x7f2688072ee0 msgr2=0x7f268809a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:17.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.116+0000 7f268c8ea640 1 -- 172.21.15.82:0/3048452406 shutdown_connections 2024-03-14T12:14:17.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.117+0000 7f268c8ea640 1 -- 172.21.15.82:0/3048452406 wait complete. 2024-03-14T12:14:17.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.118+0000 7f268c8ea640 1 Processor -- start 2024-03-14T12:14:17.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.118+0000 7f268c8ea640 1 -- start start 2024-03-14T12:14:17.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f268806f330 0x7f268813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26881551d0 0x7f26880cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f26880b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880b5d80 con 0x7f2688161a80 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880b6990 con 0x7f26881551d0 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268806fce0 con 0x7f268806f330 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f26880b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f26880b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47288/0 (socket says 172.21.15.82:47288) 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 -- 172.21.15.82:0/1861140037 learned_addr learned my addr 172.21.15.82:0/1861140037 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 -- 172.21.15.82:0/1861140037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f268806f330 msgr2=0x7f268813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f268806f330 0x7f268813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 -- 172.21.15.82:0/1861140037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26881551d0 msgr2=0x7f26880cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26881551d0 0x7f26880cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 -- 172.21.15.82:0/1861140037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880b2010 con 0x7f2688161a80 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f2686d9d640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f26880b3370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f268000cef0 tx=0x7f268000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f267effd640 1 -- 172.21.15.82:0/1861140037 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268001a070 con 0x7f2688161a80 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f267effd640 1 -- 172.21.15.82:0/1861140037 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268000d860 con 0x7f2688161a80 2024-03-14T12:14:17.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26880b22f0 con 0x7f2688161a80 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f267effd640 1 -- 172.21.15.82:0/1861140037 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268000dc50 con 0x7f2688161a80 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.119+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26880b2b90 con 0x7f2688161a80 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.120+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2688070a80 con 0x7f2688161a80 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.121+0000 7f267effd640 1 -- 172.21.15.82:0/1861140037 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f268001d070 con 0x7f2688161a80 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.121+0000 7f267effd640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266806bbd0 0x7f266806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.121+0000 7f267effd640 1 -- 172.21.15.82:0/1861140037 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f268008b320 con 0x7f2688161a80 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.121+0000 7f268659c640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266806bbd0 0x7f266806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.121+0000 7f268659c640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266806bbd0 0x7f266806e090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2678007fd0 tx=0x7f2678007920 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.125+0000 7f267effd640 1 -- 172.21.15.82:0/1861140037 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2688070a80 con 0x7f2688161a80 2024-03-14T12:14:17.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.300+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f268804f050 con 0x7f2688161a80 2024-03-14T12:14:17.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.301+0000 7f267effd640 1 -- 172.21.15.82:0/1861140037 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f26800880f0 con 0x7f2688161a80 2024-03-14T12:14:17.300 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:17.300 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266806bbd0 msgr2=0x7f266806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266806bbd0 0x7f266806e090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2678007fd0 tx=0x7f2678007920 comp rx=0 tx=0).stop 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 msgr2=0x7f26880b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f26880b3370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f268000cef0 tx=0x7f268000a710 comp rx=0 tx=0).stop 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 shutdown_connections 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2688161a80 0x7f26880b3370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26881551d0 0x7f26880cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266806bbd0 0x7f266806e090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 --2- 172.21.15.82:0/1861140037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f268806f330 0x7f268813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 >> 172.21.15.82:0/1861140037 conn(0x7f2688072ee0 msgr2=0x7f26880afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:17.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 shutdown_connections 2024-03-14T12:14:17.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.303+0000 7f268c8ea640 1 -- 172.21.15.82:0/1861140037 wait complete. 2024-03-14T12:14:17.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:17.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:17.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.418+0000 7f44ad59f640 1 Processor -- start 2024-03-14T12:14:17.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.418+0000 7f44ad59f640 1 -- start start 2024-03-14T12:14:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44ad59f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f44a815e120 0x7f44a80d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44ad59f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44a80b20e0 0x7f44a80d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44ad59f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80d1da0 con 0x7f44a80b2b70 2024-03-14T12:14:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44ad59f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a815e500 con 0x7f44a80b20e0 2024-03-14T12:14:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44ad59f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a80d4810 con 0x7f44a815e120 2024-03-14T12:14:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44acd9e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44a80b20e0 0x7f44a80d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44acd9e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44a80b20e0 0x7f44a80d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47298/0 (socket says 172.21.15.82:47298) 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44acd9e640 1 -- 172.21.15.82:0/2948312171 learned_addr learned my addr 172.21.15.82:0/2948312171 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44a7fff640 1 --2- 172.21.15.82:0/2948312171 >> v2:172.21.15.164:3300/0 conn(0x7f44a815e120 0x7f44a80d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44acd9e640 1 -- 172.21.15.82:0/2948312171 >> v2:172.21.15.164:3300/0 conn(0x7f44a815e120 msgr2=0x7f44a80d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44acd9e640 1 --2- 172.21.15.82:0/2948312171 >> v2:172.21.15.164:3300/0 conn(0x7f44a815e120 0x7f44a80d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44acd9e640 1 -- 172.21.15.82:0/2948312171 >> v1:172.21.15.164:6789/0 conn(0x7f44a80b2b70 legacy=0x7f44a80ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44acd9e640 1 -- 172.21.15.82:0/2948312171 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80d4af0 con 0x7f44a80b20e0 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.419+0000 7f44a7fff640 1 --2- 172.21.15.82:0/2948312171 >> v2:172.21.15.164:3300/0 conn(0x7f44a815e120 0x7f44a80d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.420+0000 7f44acd9e640 1 --2- 172.21.15.82:0/2948312171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a80b20e0 0x7f44a80d2770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44a8069b50 tx=0x7f44a002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce2e6ea35c1810a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.420+0000 7f44a6ffd640 1 -- 172.21.15.82:0/2948312171 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a002cb20 con 0x7f44a80b20e0 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.420+0000 7f44a6ffd640 1 -- 172.21.15.82:0/2948312171 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a002ccc0 con 0x7f44a80b20e0 2024-03-14T12:14:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.420+0000 7f44a6ffd640 1 -- 172.21.15.82:0/2948312171 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0034610 con 0x7f44a80b20e0 2024-03-14T12:14:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 -- 172.21.15.82:0/2948312171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a80b20e0 msgr2=0x7f44a80d2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 --2- 172.21.15.82:0/2948312171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a80b20e0 0x7f44a80d2770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44a8069b50 tx=0x7f44a002bf80 comp rx=0 tx=0).stop 2024-03-14T12:14:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 -- 172.21.15.82:0/2948312171 shutdown_connections 2024-03-14T12:14:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 --2- 172.21.15.82:0/2948312171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a80b20e0 0x7f44a80d2770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 --2- 172.21.15.82:0/2948312171 >> v2:172.21.15.164:3300/0 conn(0x7f44a815e120 0x7f44a80d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 -- 172.21.15.82:0/2948312171 >> 172.21.15.82:0/2948312171 conn(0x7f44a80d3c50 msgr2=0x7f44a80d4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 -- 172.21.15.82:0/2948312171 shutdown_connections 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.421+0000 7f44ad59f640 1 -- 172.21.15.82:0/2948312171 wait complete. 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.422+0000 7f44ad59f640 1 Processor -- start 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.422+0000 7f44ad59f640 1 -- start start 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44ad59f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a80b20e0 0x7f44a80c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44ad59f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a80b2b70 0x7f44a80b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44ad59f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a815e120 0x7f44a80b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44ad59f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80d1a30 con 0x7f44a815e120 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44ad59f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80b0bb0 con 0x7f44a80b20e0 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44ad59f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80d0cf0 con 0x7f44a80b2b70 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a815e120 0x7f44a80b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a815e120 0x7f44a80b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47314/0 (socket says 172.21.15.82:47314) 2024-03-14T12:14:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 -- 172.21.15.82:0/1884570685 learned_addr learned my addr 172.21.15.82:0/1884570685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 -- 172.21.15.82:0/1884570685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a80b2b70 msgr2=0x7f44a80b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a80b2b70 0x7f44a80b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 -- 172.21.15.82:0/1884570685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a80b20e0 msgr2=0x7f44a80c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a80b20e0 0x7f44a80c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 -- 172.21.15.82:0/1884570685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80b8d20 con 0x7f44a815e120 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44acd9e640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a815e120 0x7f44a80b6aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009f40 tx=0x7f44a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44a4ff9640 1 -- 172.21.15.82:0/1884570685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0043020 con 0x7f44a815e120 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44a4ff9640 1 -- 172.21.15.82:0/1884570685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a003e450 con 0x7f44a815e120 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a80c8360 con 0x7f44a815e120 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.423+0000 7f44a4ff9640 1 -- 172.21.15.82:0/1884570685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a003e820 con 0x7f44a815e120 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.424+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a80c8920 con 0x7f44a815e120 2024-03-14T12:14:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.424+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a8075210 con 0x7f44a815e120 2024-03-14T12:14:17.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.425+0000 7f44a4ff9640 1 -- 172.21.15.82:0/1884570685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f44a0046020 con 0x7f44a815e120 2024-03-14T12:14:17.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.425+0000 7f44a4ff9640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f449406bbd0 0x7f449406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.425+0000 7f44a4ff9640 1 -- 172.21.15.82:0/1884570685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44a00ad350 con 0x7f44a815e120 2024-03-14T12:14:17.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.425+0000 7f44a7fff640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f449406bbd0 0x7f449406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.426+0000 7f44a7fff640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f449406bbd0 0x7f449406e090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4498006fd0 tx=0x7f4498008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.430+0000 7f44a4ff9640 1 -- 172.21.15.82:0/1884570685 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f44a004d030 con 0x7f44a815e120 2024-03-14T12:14:17.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.601+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44a80b28d0 con 0x7f44a815e120 2024-03-14T12:14:17.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.601+0000 7f44a4ff9640 1 -- 172.21.15.82:0/1884570685 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44a007be40 con 0x7f44a815e120 2024-03-14T12:14:17.600 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f449406bbd0 msgr2=0x7f449406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f449406bbd0 0x7f449406e090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4498006fd0 tx=0x7f4498008040 comp rx=0 tx=0).stop 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a815e120 msgr2=0x7f44a80b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a815e120 0x7f44a80b6aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009f40 tx=0x7f44a0004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 shutdown_connections 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a815e120 0x7f44a80b6aa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f449406bbd0 0x7f449406e090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a80b2b70 0x7f44a80b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 --2- 172.21.15.82:0/1884570685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a80b20e0 0x7f44a80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 >> 172.21.15.82:0/1884570685 conn(0x7f44a80d3c50 msgr2=0x7f44a80b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 shutdown_connections 2024-03-14T12:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.603+0000 7f44ad59f640 1 -- 172.21.15.82:0/1884570685 wait complete. 2024-03-14T12:14:17.613 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:17.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.718+0000 7fee2f329640 1 Processor -- start 2024-03-14T12:14:17.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.718+0000 7fee2f329640 1 -- start start 2024-03-14T12:14:17.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2f329640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee2815bde0 0x7fee2815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2f329640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fee280d08c0 0x7fee280cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2f329640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee280ce380 con 0x7fee280cd250 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2f329640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee280cef90 con 0x7fee2815bde0 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2f329640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee280b5a30 con 0x7fee280d08c0 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2d0c5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee2815bde0 0x7fee2815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2c8c4640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fee280cd250 0x7fee280d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51698/0 (socket says 172.21.15.82:51698) 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.719+0000 7fee2c8c4640 1 -- 172.21.15.82:0/3570869855 learned_addr learned my addr 172.21.15.82:0/3570869855 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee2d8c6640 1 --2- 172.21.15.82:0/3570869855 >> v2:172.21.15.164:3300/0 conn(0x7fee280d08c0 0x7fee280cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee2d0c5640 1 -- 172.21.15.82:0/3570869855 >> v2:172.21.15.164:3300/0 conn(0x7fee280d08c0 msgr2=0x7fee280cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee2d0c5640 1 --2- 172.21.15.82:0/3570869855 >> v2:172.21.15.164:3300/0 conn(0x7fee280d08c0 0x7fee280cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee2d0c5640 1 -- 172.21.15.82:0/3570869855 >> v1:172.21.15.82:6789/0 conn(0x7fee280cd250 legacy=0x7fee280d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee1ffff640 1 -- 172.21.15.82:0/3570869855 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043927277 0 0) 0x7fee280ce380 con 0x7fee280cd250 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee2d0c5640 1 -- 172.21.15.82:0/3570869855 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280ce660 con 0x7fee2815bde0 2024-03-14T12:14:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee2d8c6640 1 --2- 172.21.15.82:0/3570869855 >> v2:172.21.15.164:3300/0 conn(0x7fee280d08c0 0x7fee280cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:17.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee2d0c5640 1 --2- 172.21.15.82:0/3570869855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee2815c1c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee28071d70 tx=0x7fee2402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c8d79c5e6579d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.720+0000 7fee1ffff640 1 -- 172.21.15.82:0/3570869855 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24039070 con 0x7fee2815bde0 2024-03-14T12:14:17.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee1ffff640 1 -- 172.21.15.82:0/3570869855 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2402cd00 con 0x7fee2815bde0 2024-03-14T12:14:17.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee1ffff640 1 -- 172.21.15.82:0/3570869855 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24034660 con 0x7fee2815bde0 2024-03-14T12:14:17.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 -- 172.21.15.82:0/3570869855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 msgr2=0x7fee2815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 --2- 172.21.15.82:0/3570869855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee2815c1c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee28071d70 tx=0x7fee2402bf50 comp rx=0 tx=0).stop 2024-03-14T12:14:17.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 -- 172.21.15.82:0/3570869855 shutdown_connections 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 --2- 172.21.15.82:0/3570869855 >> v2:172.21.15.164:3300/0 conn(0x7fee280d08c0 0x7fee280cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 --2- 172.21.15.82:0/3570869855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee2815c1c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 -- 172.21.15.82:0/3570869855 >> 172.21.15.82:0/3570869855 conn(0x7fee280d1bd0 msgr2=0x7fee280d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 -- 172.21.15.82:0/3570869855 shutdown_connections 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.721+0000 7fee2f329640 1 -- 172.21.15.82:0/3570869855 wait complete. 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.722+0000 7fee2f329640 1 Processor -- start 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.722+0000 7fee2f329640 1 -- start start 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2f329640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee280cd250 0x7fee280c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2f329640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee280d08c0 0x7fee281f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2f329640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee281f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2f329640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee280cf420 con 0x7fee2815bde0 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2f329640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee280cec20 con 0x7fee280d08c0 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2f329640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee280b6c20 con 0x7fee280cd250 2024-03-14T12:14:17.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee281f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee281f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47344/0 (socket says 172.21.15.82:47344) 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 -- 172.21.15.82:0/2751753976 learned_addr learned my addr 172.21.15.82:0/2751753976 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2c8c4640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee280d08c0 0x7fee281f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d0c5640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee280cd250 0x7fee280c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 -- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee280cd250 msgr2=0x7fee280c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee280cd250 0x7fee280c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 -- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee280d08c0 msgr2=0x7fee281f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee280d08c0 0x7fee281f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 -- 172.21.15.82:0/2751753976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280c6810 con 0x7fee2815bde0 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d0c5640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee280cd250 0x7fee280c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.723+0000 7fee2d8c6640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee281f3f60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fee2000c7f0 tx=0x7fee2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.724+0000 7fee1dffb640 1 -- 172.21.15.82:0/2751753976 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20015070 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.724+0000 7fee1dffb640 1 -- 172.21.15.82:0/2751753976 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20004240 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.724+0000 7fee1dffb640 1 -- 172.21.15.82:0/2751753976 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20007620 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.724+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee280c6af0 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.724+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee280c70b0 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.725+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2804f050 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.725+0000 7fee1dffb640 1 -- 172.21.15.82:0/2751753976 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fee200079d0 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.725+0000 7fee1dffb640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee0c06bbd0 0x7fee0c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.726+0000 7fee1dffb640 1 -- 172.21.15.82:0/2751753976 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee2008aab0 con 0x7fee2815bde0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.726+0000 7fee2d0c5640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee0c06bbd0 0x7fee0c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:17.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.726+0000 7fee2d0c5640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee0c06bbd0 0x7fee0c06e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee24036000 tx=0x7fee24030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:17.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.730+0000 7fee1dffb640 1 -- 172.21.15.82:0/2751753976 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fee20058f50 con 0x7fee2815bde0 2024-03-14T12:14:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.900+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee280b7b50 con 0x7fee2815bde0 2024-03-14T12:14:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.900+0000 7fee1dffb640 1 -- 172.21.15.82:0/2751753976 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee2005cff0 con 0x7fee2815bde0 2024-03-14T12:14:17.899 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:17.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee0c06bbd0 msgr2=0x7fee0c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee0c06bbd0 0x7fee0c06e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fee24036000 tx=0x7fee24030000 comp rx=0 tx=0).stop 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 msgr2=0x7fee281f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee281f3f60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fee2000c7f0 tx=0x7fee2000ccb0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 shutdown_connections 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2815bde0 0x7fee281f3f60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee280d08c0 0x7fee281f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee0c06bbd0 0x7fee0c06e090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 --2- 172.21.15.82:0/2751753976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee280cd250 0x7fee280c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 >> 172.21.15.82:0/2751753976 conn(0x7fee280d1bd0 msgr2=0x7fee280cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 shutdown_connections 2024-03-14T12:14:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:17.902+0000 7fee2f329640 1 -- 172.21.15.82:0/2751753976 wait complete. 2024-03-14T12:14:17.912 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:18.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.017+0000 7f2597a61640 1 Processor -- start 2024-03-14T12:14:18.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.017+0000 7f2597a61640 1 -- start start 2024-03-14T12:14:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2597a61640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f25900cf3f0 0x7f25900cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2597a61640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25900b41d0 0x7f25900d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2597a61640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25900b3450 con 0x7f259015e780 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2597a61640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25900cf7d0 con 0x7f25900b41d0 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2597a61640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259015f4b0 con 0x7f25900cf3f0 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2594ffc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25900b41d0 0x7f25900d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2594ffc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25900b41d0 0x7f25900d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47360/0 (socket says 172.21.15.82:47360) 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2594ffc640 1 -- 172.21.15.82:0/2733772090 learned_addr learned my addr 172.21.15.82:0/2733772090 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f25957fd640 1 --2- 172.21.15.82:0/2733772090 >> v2:172.21.15.164:3300/0 conn(0x7f25900cf3f0 0x7f25900cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2594ffc640 1 -- 172.21.15.82:0/2733772090 >> v2:172.21.15.164:3300/0 conn(0x7f25900cf3f0 msgr2=0x7f25900cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2594ffc640 1 --2- 172.21.15.82:0/2733772090 >> v2:172.21.15.164:3300/0 conn(0x7f25900cf3f0 0x7f25900cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2594ffc640 1 -- 172.21.15.82:0/2733772090 >> v1:172.21.15.164:6789/0 conn(0x7f259015e780 legacy=0x7f259015eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.018+0000 7f2594ffc640 1 -- 172.21.15.82:0/2733772090 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900b36f0 con 0x7f25900b41d0 2024-03-14T12:14:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.019+0000 7f2594ffc640 1 --2- 172.21.15.82:0/2733772090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f25900d0100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f259006bcb0 tx=0x7f258002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c8da3409b86879 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.019+0000 7f2587fff640 1 -- 172.21.15.82:0/2733772090 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580035020 con 0x7f25900b41d0 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.019+0000 7f2587fff640 1 -- 172.21.15.82:0/2733772090 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2580031870 con 0x7f25900b41d0 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.019+0000 7f2587fff640 1 -- 172.21.15.82:0/2733772090 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580031c40 con 0x7f25900b41d0 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.019+0000 7f2597a61640 1 -- 172.21.15.82:0/2733772090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 msgr2=0x7f25900d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.019+0000 7f2597a61640 1 --2- 172.21.15.82:0/2733772090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f25900d0100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f259006bcb0 tx=0x7f258002b910 comp rx=0 tx=0).stop 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.019+0000 7f2597a61640 1 -- 172.21.15.82:0/2733772090 shutdown_connections 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.020+0000 7f2597a61640 1 --2- 172.21.15.82:0/2733772090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f25900d0100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.020+0000 7f2597a61640 1 --2- 172.21.15.82:0/2733772090 >> v2:172.21.15.164:3300/0 conn(0x7f25900cf3f0 0x7f25900cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.020+0000 7f2597a61640 1 -- 172.21.15.82:0/2733772090 >> 172.21.15.82:0/2733772090 conn(0x7f259006aef0 msgr2=0x7f259006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.020+0000 7f2597a61640 1 -- 172.21.15.82:0/2733772090 shutdown_connections 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.020+0000 7f2597a61640 1 -- 172.21.15.82:0/2733772090 wait complete. 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 Processor -- start 2024-03-14T12:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 -- start start 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f2590154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25900cf3f0 0x7f25901432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f259015e780 0x7f25900cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259006d890 con 0x7f25900b41d0 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25900b3cb0 con 0x7f259015e780 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.021+0000 7f2597a61640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259006da10 con 0x7f25900cf3f0 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f2590154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f2590154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47362/0 (socket says 172.21.15.82:47362) 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 -- 172.21.15.82:0/3447620516 learned_addr learned my addr 172.21.15.82:0/3447620516 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 -- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25900cf3f0 msgr2=0x7f25901432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25900cf3f0 0x7f25901432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 -- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f259015e780 msgr2=0x7f25900cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f2595ffe640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f259015e780 0x7f25900cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f259015e780 0x7f25900cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 -- 172.21.15.82:0/3447620516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590155440 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f2595ffe640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f259015e780 0x7f25900cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f25957fd640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f2590154f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d800 tx=0x7f258c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f2585ffb640 1 -- 172.21.15.82:0/3447620516 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00b6c0 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f2585ffb640 1 -- 172.21.15.82:0/3447620516 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c019070 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25900b0ad0 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f2585ffb640 1 -- 172.21.15.82:0/3447620516 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00bac0 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.022+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25900b1850 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.023+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259004f050 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.024+0000 7f2585ffb640 1 -- 172.21.15.82:0/3447620516 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f258c00bc60 con 0x7f25900b41d0 2024-03-14T12:14:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.024+0000 7f2585ffb640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f257406bbd0 0x7f257406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.024+0000 7f2585ffb640 1 -- 172.21.15.82:0/3447620516 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f258c08aed0 con 0x7f25900b41d0 2024-03-14T12:14:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.024+0000 7f2594ffc640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f257406bbd0 0x7f257406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.024+0000 7f2594ffc640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f257406bbd0 0x7f257406e090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2580036000 tx=0x7f2580030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.028+0000 7f2585ffb640 1 -- 172.21.15.82:0/3447620516 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2590155440 con 0x7f25900b41d0 2024-03-14T12:14:18.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.202+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f259006d320 con 0x7f25900b41d0 2024-03-14T12:14:18.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.202+0000 7f2585ffb640 1 -- 172.21.15.82:0/3447620516 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f258c059a50 con 0x7f25900b41d0 2024-03-14T12:14:18.202 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.204+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f257406bbd0 msgr2=0x7f257406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.204+0000 7f2597a61640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f257406bbd0 0x7f257406e090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2580036000 tx=0x7f2580030000 comp rx=0 tx=0).stop 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 msgr2=0x7f2590154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f2590154f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d800 tx=0x7f258c00dcc0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 shutdown_connections 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f259015e780 0x7f25900cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f257406bbd0 0x7f257406e090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25900cf3f0 0x7f25901432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 --2- 172.21.15.82:0/3447620516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25900b41d0 0x7f2590154f00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 >> 172.21.15.82:0/3447620516 conn(0x7f259006aef0 msgr2=0x7f25900b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:18.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 shutdown_connections 2024-03-14T12:14:18.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.205+0000 7f2597a61640 1 -- 172.21.15.82:0/3447620516 wait complete. 2024-03-14T12:14:18.215 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:18.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.316+0000 7f07835b9640 1 Processor -- start 2024-03-14T12:14:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.317+0000 7f07835b9640 1 -- start start 2024-03-14T12:14:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.317+0000 7f07835b9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0cf370 0x7f077c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.317+0000 7f07835b9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0cf8c0 con 0x7f077c0b7720 2024-03-14T12:14:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.317+0000 7f07835b9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0cfba0 con 0x7f077c0d0760 2024-03-14T12:14:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.317+0000 7f07835b9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0b6c40 con 0x7f077c0cf370 2024-03-14T12:14:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0781b56640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f077c0b7720 0x7f077c0b7bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51702/0 (socket says 172.21.15.82:51702) 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0781b56640 1 -- 172.21.15.82:0/3960555190 learned_addr learned my addr 172.21.15.82:0/3960555190 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0780b54640 1 --2- 172.21.15.82:0/3960555190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0cf370 0x7f077c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662143294 0 0) 0x7f077c0cf8c0 con 0x7f077c0b7720 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0768003520 con 0x7f077c0b7720 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935757923 0 0) 0x7f077c0cfba0 con 0x7f077c0d0760 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f077c0cf8c0 con 0x7f077c0d0760 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 893017178 0 0) 0x7f0768003520 con 0x7f077c0b7720 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0780b54640 1 -- 172.21.15.82:0/3960555190 >> v1:172.21.15.164:6789/0 conn(0x7f077c0d0760 legacy=0x7f077c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0780b54640 1 -- 172.21.15.82:0/3960555190 >> v1:172.21.15.82:6789/0 conn(0x7f077c0b7720 legacy=0x7f077c0b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.318+0000 7f0780b54640 1 -- 172.21.15.82:0/3960555190 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0b6f20 con 0x7f077c0cf370 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.319+0000 7f0780b54640 1 --2- 172.21.15.82:0/3960555190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0cf370 0x7f077c0d4660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f076c0097d0 tx=0x7f076c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89ccb5839642db4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.319+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774001670 con 0x7f077c0cf370 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.319+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c037070 con 0x7f077c0cf370 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.319+0000 7f0773fff640 1 -- 172.21.15.82:0/3960555190 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c031780 con 0x7f077c0cf370 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.320+0000 7f07835b9640 1 -- 172.21.15.82:0/3960555190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0cf370 msgr2=0x7f077c0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.320+0000 7f07835b9640 1 --2- 172.21.15.82:0/3960555190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0cf370 0x7f077c0d4660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f076c0097d0 tx=0x7f076c02b8a0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.320+0000 7f07835b9640 1 -- 172.21.15.82:0/3960555190 shutdown_connections 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.320+0000 7f07835b9640 1 --2- 172.21.15.82:0/3960555190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0cf370 0x7f077c0d4660 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.320+0000 7f07835b9640 1 -- 172.21.15.82:0/3960555190 >> 172.21.15.82:0/3960555190 conn(0x7f077c09d030 msgr2=0x7f077c0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.320+0000 7f07835b9640 1 -- 172.21.15.82:0/3960555190 shutdown_connections 2024-03-14T12:14:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.320+0000 7f07835b9640 1 -- 172.21.15.82:0/3960555190 wait complete. 2024-03-14T12:14:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.321+0000 7f07835b9640 1 Processor -- start 2024-03-14T12:14:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.321+0000 7f07835b9640 1 -- start start 2024-03-14T12:14:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f07835b9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0b7720 0x7f077c0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f07835b9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0cf370 0x7f077c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f07835b9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077c0d0760 0x7f077c0cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f07835b9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c1617a0 con 0x7f077c0cf370 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f07835b9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0d4ba0 con 0x7f077c0d0760 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f07835b9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c161920 con 0x7f077c0b7720 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0cf370 0x7f077c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0cf370 0x7f077c0cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47366/0 (socket says 172.21.15.82:47366) 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 -- 172.21.15.82:0/834812260 learned_addr learned my addr 172.21.15.82:0/834812260 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0781355640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0b7720 0x7f077c0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0781b56640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077c0d0760 0x7f077c0cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 -- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0b7720 msgr2=0x7f077c0ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0b7720 0x7f077c0ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 -- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077c0d0760 msgr2=0x7f077c0cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077c0d0760 0x7f077c0cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 -- 172.21.15.82:0/834812260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0cc320 con 0x7f077c0cf370 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0781b56640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077c0d0760 0x7f077c0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0780b54640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0cf370 0x7f077c0cb2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f076c02b8e0 tx=0x7f076c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0771ffb640 1 -- 172.21.15.82:0/834812260 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c031780 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0771ffb640 1 -- 172.21.15.82:0/834812260 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c031940 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c0cd220 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.322+0000 7f0771ffb640 1 -- 172.21.15.82:0/834812260 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c031d30 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.323+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c0cd7e0 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.323+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c04f050 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.324+0000 7f0771ffb640 1 -- 172.21.15.82:0/834812260 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f076c032410 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.324+0000 7f0771ffb640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f076006bbd0 0x7f076006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.324+0000 7f0771ffb640 1 -- 172.21.15.82:0/834812260 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f076c0ad020 con 0x7f077c0cf370 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.324+0000 7f0781355640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f076006bbd0 0x7f076006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.325+0000 7f0781355640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f076006bbd0 0x7f076006e090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0778001cb0 tx=0x7f0778009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.329+0000 7f0771ffb640 1 -- 172.21.15.82:0/834812260 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f076c02f050 con 0x7f077c0cf370 2024-03-14T12:14:18.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.503+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f077c0cb0d0 con 0x7f077c0cf370 2024-03-14T12:14:18.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.504+0000 7f0771ffb640 1 -- 172.21.15.82:0/834812260 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f076c07b440 con 0x7f077c0cf370 2024-03-14T12:14:18.503 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:18.503 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:18.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f076006bbd0 msgr2=0x7f076006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f076006bbd0 0x7f076006e090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0778001cb0 tx=0x7f0778009040 comp rx=0 tx=0).stop 2024-03-14T12:14:18.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0cf370 msgr2=0x7f077c0cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0cf370 0x7f077c0cb2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f076c02b8e0 tx=0x7f076c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 shutdown_connections 2024-03-14T12:14:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077c0d0760 0x7f077c0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0cf370 0x7f077c0cb2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f076006bbd0 0x7f076006e090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 --2- 172.21.15.82:0/834812260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f077c0b7720 0x7f077c0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 >> 172.21.15.82:0/834812260 conn(0x7f077c09d030 msgr2=0x7f077c15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 shutdown_connections 2024-03-14T12:14:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.506+0000 7f07835b9640 1 -- 172.21.15.82:0/834812260 wait complete. 2024-03-14T12:14:18.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:18.517 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:18.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.620+0000 7ff42b21f640 1 Processor -- start 2024-03-14T12:14:18.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.620+0000 7ff42b21f640 1 -- start start 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff42b21f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff4240b5e60 0x7ff42406ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff42b21f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42415c0c0 con 0x7ff4240b1200 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff42b21f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4240afea0 con 0x7ff42415bce0 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff42b21f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240afc50 con 0x7ff4240b5e60 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4297bc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff4240b1200 0x7ff42406f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51710/0 (socket says 172.21.15.82:51710) 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4297bc640 1 -- 172.21.15.82:0/668467184 learned_addr learned my addr 172.21.15.82:0/668467184 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206011220 0 0) 0x7ff42415c0c0 con 0x7ff4240b1200 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff423fff640 1 --2- 172.21.15.82:0/668467184 >> v2:172.21.15.164:3300/0 conn(0x7ff4240b5e60 0x7ff42406ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff414003520 con 0x7ff4240b1200 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754554150 0 0) 0x7ff414003520 con 0x7ff4240b1200 2024-03-14T12:14:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 >> v2:172.21.15.164:3300/0 conn(0x7ff4240b5e60 msgr2=0x7ff42406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 --2- 172.21.15.82:0/668467184 >> v2:172.21.15.164:3300/0 conn(0x7ff4240b5e60 0x7ff42406ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 >> v1:172.21.15.164:6789/0 conn(0x7ff42415bce0 legacy=0x7ff4240b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240b0180 con 0x7ff4240b1200 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7ff41c001a10 con 0x7ff4240b1200 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 >> v1:172.21.15.82:6789/0 conn(0x7ff4240b1200 legacy=0x7ff42406f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.621+0000 7ff4237fe640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff4237fe640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff414004800 0x7ff414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff4237fe640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4140051c0 0x7ff41400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240afc50 con 0x7ff4140039b0 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41400c110 con 0x7ff414004800 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41400c310 con 0x7ff4140051c0 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff428fbb640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff423fff640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff414004800 0x7ff414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff4297bc640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4140051c0 0x7ff41400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff428fbb640 1 -- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4140051c0 msgr2=0x7ff41400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff428fbb640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4140051c0 0x7ff41400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff428fbb640 1 -- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff414004800 msgr2=0x7ff414004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff428fbb640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff414004800 0x7ff414004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff428fbb640 1 -- 172.21.15.82:0/668467184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414003520 con 0x7ff4140039b0 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff423fff640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff414004800 0x7ff414004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.622+0000 7ff4297bc640 1 -- 172.21.15.82:0/668467184 reap_dead start 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff428fbb640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff414003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff418001750 tx=0x7ff414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ab28b7c751fb11 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418001370 con 0x7ff4140039b0 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180052d0 con 0x7ff4140039b0 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff4237fe640 1 -- 172.21.15.82:0/668467184 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418003830 con 0x7ff4140039b0 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff42b21f640 1 -- 172.21.15.82:0/668467184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 msgr2=0x7ff414003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff42b21f640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff414003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff418001750 tx=0x7ff414005830 comp rx=0 tx=0).stop 2024-03-14T12:14:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff42b21f640 1 -- 172.21.15.82:0/668467184 shutdown_connections 2024-03-14T12:14:18.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff42b21f640 1 --2- 172.21.15.82:0/668467184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff414003dd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff42b21f640 1 -- 172.21.15.82:0/668467184 >> 172.21.15.82:0/668467184 conn(0x7ff424072ec0 msgr2=0x7ff42409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:18.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff42b21f640 1 -- 172.21.15.82:0/668467184 shutdown_connections 2024-03-14T12:14:18.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.623+0000 7ff42b21f640 1 -- 172.21.15.82:0/668467184 wait complete. 2024-03-14T12:14:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.624+0000 7ff42b21f640 1 Processor -- start 2024-03-14T12:14:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.624+0000 7ff42b21f640 1 -- start start 2024-03-14T12:14:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff42b21f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff4240b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff42b21f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4241527c0 0x7ff424159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff42b21f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff424159c90 0x7ff42414e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff42b21f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240b0a20 con 0x7ff4140039b0 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff42b21f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240b0740 con 0x7ff424159c90 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff42b21f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240b0cb0 con 0x7ff4241527c0 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff428fbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff4240b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff428fbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff4240b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47388/0 (socket says 172.21.15.82:47388) 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.625+0000 7ff428fbb640 1 -- 172.21.15.82:0/258250748 learned_addr learned my addr 172.21.15.82:0/258250748 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff428fbb640 1 -- 172.21.15.82:0/258250748 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4241527c0 msgr2=0x7ff424159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff428fbb640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4241527c0 0x7ff424159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff428fbb640 1 -- 172.21.15.82:0/258250748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff424159c90 msgr2=0x7ff42414e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff428fbb640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff424159c90 0x7ff42414e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff428fbb640 1 -- 172.21.15.82:0/258250748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4241f7100 con 0x7ff4140039b0 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff428fbb640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff4240b2b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff418000c00 tx=0x7ff418001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff4217fa640 1 -- 172.21.15.82:0/258250748 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418041020 con 0x7ff4140039b0 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff4217fa640 1 -- 172.21.15.82:0/258250748 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff418005480 con 0x7ff4140039b0 2024-03-14T12:14:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4241f73e0 con 0x7ff4140039b0 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff4217fa640 1 -- 172.21.15.82:0/258250748 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418007a30 con 0x7ff4140039b0 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.626+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241f79a0 con 0x7ff4140039b0 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.627+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42404f050 con 0x7ff4140039b0 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.627+0000 7ff4217fa640 1 -- 172.21.15.82:0/258250748 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff418044020 con 0x7ff4140039b0 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.628+0000 7ff4217fa640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff40806bbd0 0x7ff40806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.628+0000 7ff4217fa640 1 -- 172.21.15.82:0/258250748 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4180ac4e0 con 0x7ff4140039b0 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.628+0000 7ff423fff640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff40806bbd0 0x7ff40806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.628+0000 7ff423fff640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff40806bbd0 0x7ff40806e090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff410006d40 tx=0x7ff410006bd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.632+0000 7ff4217fa640 1 -- 172.21.15.82:0/258250748 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff41804b030 con 0x7ff4140039b0 2024-03-14T12:14:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.804+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4240b5a70 con 0x7ff4140039b0 2024-03-14T12:14:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.804+0000 7ff4217fa640 1 -- 172.21.15.82:0/258250748 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4180303c0 con 0x7ff4140039b0 2024-03-14T12:14:18.803 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff40806bbd0 msgr2=0x7ff40806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff40806bbd0 0x7ff40806e090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff410006d40 tx=0x7ff410006bd0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 msgr2=0x7ff4240b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff4240b2b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff418000c00 tx=0x7ff418001120 comp rx=0 tx=0).stop 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 shutdown_connections 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff424159c90 0x7ff42414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff40806bbd0 0x7ff40806e090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4241527c0 0x7ff424159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 --2- 172.21.15.82:0/258250748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4140039b0 0x7ff4240b2b40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 >> 172.21.15.82:0/258250748 conn(0x7ff424072ec0 msgr2=0x7ff42406fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.806+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 shutdown_connections 2024-03-14T12:14:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.807+0000 7ff42b21f640 1 -- 172.21.15.82:0/258250748 wait complete. 2024-03-14T12:14:18.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:18.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.920+0000 7f7dea8e8640 1 Processor -- start 2024-03-14T12:14:18.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.920+0000 7f7dea8e8640 1 -- start start 2024-03-14T12:14:18.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7dea8e8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7dea8e8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7de40d1b00 0x7f7de40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7dea8e8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de4070250 con 0x7f7de415e150 2024-03-14T12:14:18.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7dea8e8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de406ea20 con 0x7f7de40d1b00 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7dea8e8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de406f880 con 0x7f7de415dd70 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7de8e85640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7de40d1b00 0x7f7de40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7de37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7de415e150 0x7f7de40b6690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51712/0 (socket says 172.21.15.82:51712) 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7de37fe640 1 -- 172.21.15.82:0/3306505304 learned_addr learned my addr 172.21.15.82:0/3306505304 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.921+0000 7f7de3fff640 1 --2- 172.21.15.82:0/3306505304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de8e85640 1 -- 172.21.15.82:0/3306505304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 msgr2=0x7f7de406fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de8e85640 1 --2- 172.21.15.82:0/3306505304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de406fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de8e85640 1 -- 172.21.15.82:0/3306505304 >> v1:172.21.15.82:6789/0 conn(0x7f7de415e150 legacy=0x7f7de40b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de8e85640 1 -- 172.21.15.82:0/3306505304 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de406ec70 con 0x7f7de40d1b00 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de2ffd640 1 -- 172.21.15.82:0/3306505304 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853489184 0 0) 0x7f7de4070250 con 0x7f7de415e150 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de8e85640 1 --2- 172.21.15.82:0/3306505304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de40d1fb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009d90 tx=0x7f7ddc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3f81cb5fc704ed server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de2ffd640 1 -- 172.21.15.82:0/3306505304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc039070 con 0x7f7de40d1b00 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de2ffd640 1 -- 172.21.15.82:0/3306505304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc02cc50 con 0x7f7de40d1b00 2024-03-14T12:14:18.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.922+0000 7f7de2ffd640 1 -- 172.21.15.82:0/3306505304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc034440 con 0x7f7de40d1b00 2024-03-14T12:14:18.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 -- 172.21.15.82:0/3306505304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 msgr2=0x7f7de40d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/3306505304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de40d1fb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009d90 tx=0x7f7ddc02bea0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 -- 172.21.15.82:0/3306505304 shutdown_connections 2024-03-14T12:14:18.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/3306505304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de40d1fb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/3306505304 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de406fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 -- 172.21.15.82:0/3306505304 >> 172.21.15.82:0/3306505304 conn(0x7f7de4072ec0 msgr2=0x7f7de409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 -- 172.21.15.82:0/3306505304 shutdown_connections 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.923+0000 7f7dea8e8640 1 -- 172.21.15.82:0/3306505304 wait complete. 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.924+0000 7f7dea8e8640 1 Processor -- start 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.924+0000 7f7dea8e8640 1 -- start start 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7dea8e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de41f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7dea8e8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de40caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7dea8e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de415e150 0x7f7de406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7dea8e8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de406f510 con 0x7f7de40d1b00 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7dea8e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de406f230 con 0x7f7de415e150 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7dea8e8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40b6da0 con 0x7f7de415dd70 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de41f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de41f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47404/0 (socket says 172.21.15.82:47404) 2024-03-14T12:14:18.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 -- 172.21.15.82:0/2222375761 learned_addr learned my addr 172.21.15.82:0/2222375761 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de37fe640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de40caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 -- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 msgr2=0x7f7de40caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de8e85640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de415e150 0x7f7de406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de40caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 -- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de415e150 msgr2=0x7f7de406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de415e150 0x7f7de406d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de3fff640 1 -- 172.21.15.82:0/2222375761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de40cb490 con 0x7f7de40d1b00 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.925+0000 7f7de37fe640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de40caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.926+0000 7f7de3fff640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de41f7240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400d830 tx=0x7f7dd400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.926+0000 7f7de0ff9640 1 -- 172.21.15.82:0/2222375761 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400b6c0 con 0x7f7de40d1b00 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.926+0000 7f7de0ff9640 1 -- 172.21.15.82:0/2222375761 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4019070 con 0x7f7de40d1b00 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.926+0000 7f7de0ff9640 1 -- 172.21.15.82:0/2222375761 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400ba80 con 0x7f7de40d1b00 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.926+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de41f78d0 con 0x7f7de40d1b00 2024-03-14T12:14:18.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.926+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de41f7e90 con 0x7f7de40d1b00 2024-03-14T12:14:18.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.927+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de404f050 con 0x7f7de40d1b00 2024-03-14T12:14:18.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.927+0000 7f7de0ff9640 1 -- 172.21.15.82:0/2222375761 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7dd401e460 con 0x7f7de40d1b00 2024-03-14T12:14:18.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.928+0000 7f7de0ff9640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd006bbd0 0x7f7dd006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:18.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.928+0000 7f7de37fe640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd006bbd0 0x7f7dd006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:18.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.928+0000 7f7de0ff9640 1 -- 172.21.15.82:0/2222375761 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dd408b0c0 con 0x7f7de40d1b00 2024-03-14T12:14:18.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.928+0000 7f7de37fe640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd006bbd0 0x7f7dd006e090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000f60 tx=0x7f7dd8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:18.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:18.932+0000 7f7de0ff9640 1 -- 172.21.15.82:0/2222375761 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7dd4059c40 con 0x7f7de40d1b00 2024-03-14T12:14:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.101+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7de40b5910 con 0x7f7de40d1b00 2024-03-14T12:14:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.102+0000 7f7de0ff9640 1 -- 172.21.15.82:0/2222375761 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7dd4059540 con 0x7f7de40d1b00 2024-03-14T12:14:19.101 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd006bbd0 msgr2=0x7f7dd006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd006bbd0 0x7f7dd006e090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000f60 tx=0x7f7dd8009040 comp rx=0 tx=0).stop 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 msgr2=0x7f7de41f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de41f7240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400d830 tx=0x7f7dd400dcf0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 shutdown_connections 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de415e150 0x7f7de406d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd006bbd0 0x7f7dd006e090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7de415dd70 0x7f7de40caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 --2- 172.21.15.82:0/2222375761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de40d1b00 0x7f7de41f7240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.103+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 >> 172.21.15.82:0/2222375761 conn(0x7f7de4072ec0 msgr2=0x7f7de40b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:19.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.104+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 shutdown_connections 2024-03-14T12:14:19.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.104+0000 7f7dea8e8640 1 -- 172.21.15.82:0/2222375761 wait complete. 2024-03-14T12:14:19.113 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:19.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.216+0000 7f4aabf3b640 1 Processor -- start 2024-03-14T12:14:19.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.216+0000 7f4aabf3b640 1 -- start start 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aabf3b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4aa40d02d0 0x7f4aa415e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aabf3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aabf3b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa40b3450 con 0x7f4aa40cf3f0 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aabf3b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa40cf7d0 con 0x7f4aa40d02d0 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aabf3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa415f200 con 0x7f4aa40b41d0 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aa94d6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4aa40d02d0 0x7f4aa415e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aa9cd7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4aa40cf3f0 0x7f4aa40cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51720/0 (socket says 172.21.15.82:51720) 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aa9cd7640 1 -- 172.21.15.82:0/2487126273 learned_addr learned my addr 172.21.15.82:0/2487126273 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.217+0000 7f4aaa4d8640 1 --2- 172.21.15.82:0/2487126273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa94d6640 1 -- 172.21.15.82:0/2487126273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 msgr2=0x7f4aa415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa94d6640 1 --2- 172.21.15.82:0/2487126273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa94d6640 1 -- 172.21.15.82:0/2487126273 >> v1:172.21.15.82:6789/0 conn(0x7f4aa40cf3f0 legacy=0x7f4aa40cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa94d6640 1 -- 172.21.15.82:0/2487126273 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa415f4b0 con 0x7f4aa40d02d0 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa94d6640 1 --2- 172.21.15.82:0/2487126273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415e780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40cf950 tx=0x7f4a9402de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4465571520dc37d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa8cd5640 1 -- 172.21.15.82:0/2487126273 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9403b070 con 0x7f4aa40d02d0 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa8cd5640 1 -- 172.21.15.82:0/2487126273 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9402eb50 con 0x7f4aa40d02d0 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.218+0000 7f4aa8cd5640 1 -- 172.21.15.82:0/2487126273 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94036440 con 0x7f4aa40d02d0 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2487126273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 msgr2=0x7f4aa415e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2487126273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415e780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40cf950 tx=0x7f4a9402de20 comp rx=0 tx=0).stop 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2487126273 shutdown_connections 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2487126273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2487126273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415e780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2487126273 >> 172.21.15.82:0/2487126273 conn(0x7f4aa406aef0 msgr2=0x7f4aa406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2487126273 shutdown_connections 2024-03-14T12:14:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.219+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2487126273 wait complete. 2024-03-14T12:14:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.220+0000 7f4aabf3b640 1 Processor -- start 2024-03-14T12:14:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.220+0000 7f4aabf3b640 1 -- start start 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aabf3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa4154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aabf3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aa40cf3f0 0x7f4aa414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aabf3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aabf3b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40b3cb0 con 0x7f4aa40d02d0 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aabf3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40b39d0 con 0x7f4aa40cf3f0 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aabf3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa406d890 con 0x7f4aa40b41d0 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47422/0 (socket says 172.21.15.82:47422) 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 -- 172.21.15.82:0/2826658084 learned_addr learned my addr 172.21.15.82:0/2826658084 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aa9cd7640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa4154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aa94d6640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aa40cf3f0 0x7f4aa414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 -- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 msgr2=0x7f4aa4154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa4154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 -- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aa40cf3f0 msgr2=0x7f4aa414f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aa40cf3f0 0x7f4aa414f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aaa4d8640 1 -- 172.21.15.82:0/2826658084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4155060 con 0x7f4aa40d02d0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.221+0000 7f4aa94d6640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aa40cf3f0 0x7f4aa414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.222+0000 7f4aaa4d8640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415c250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00d800 tx=0x7f4a9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.222+0000 7f4a9a7fc640 1 -- 172.21.15.82:0/2826658084 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00b6c0 con 0x7f4aa40d02d0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.222+0000 7f4a9a7fc640 1 -- 172.21.15.82:0/2826658084 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c019070 con 0x7f4aa40d02d0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.222+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa414fec0 con 0x7f4aa40d02d0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.222+0000 7f4a9a7fc640 1 -- 172.21.15.82:0/2826658084 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00bac0 con 0x7f4aa40d02d0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.222+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa4150480 con 0x7f4aa40d02d0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.223+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa404f050 con 0x7f4aa40d02d0 2024-03-14T12:14:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.223+0000 7f4a9a7fc640 1 -- 172.21.15.82:0/2826658084 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4a9c00bc60 con 0x7f4aa40d02d0 2024-03-14T12:14:19.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.223+0000 7f4a9a7fc640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a8806bbd0 0x7f4a8806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.224+0000 7f4a9a7fc640 1 -- 172.21.15.82:0/2826658084 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a9c08ae40 con 0x7f4aa40d02d0 2024-03-14T12:14:19.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.224+0000 7f4aa9cd7640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a8806bbd0 0x7f4a8806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.224+0000 7f4aa9cd7640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a8806bbd0 0x7f4a8806e090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00012e0 tx=0x7f4aa0001840 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.228+0000 7f4a9a7fc640 1 -- 172.21.15.82:0/2826658084 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4a9c059930 con 0x7f4aa40d02d0 2024-03-14T12:14:19.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.399+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4aa406d320 con 0x7f4aa40d02d0 2024-03-14T12:14:19.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.399+0000 7f4a9a7fc640 1 -- 172.21.15.82:0/2826658084 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a9c059230 con 0x7f4aa40d02d0 2024-03-14T12:14:19.399 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a8806bbd0 msgr2=0x7f4a8806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a8806bbd0 0x7f4a8806e090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00012e0 tx=0x7f4aa0001840 comp rx=0 tx=0).stop 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 msgr2=0x7f4aa415c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415c250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00d800 tx=0x7f4a9c00dcc0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 shutdown_connections 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4aa40d02d0 0x7f4aa415c250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aa40cf3f0 0x7f4aa414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a8806bbd0 0x7f4a8806e090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 --2- 172.21.15.82:0/2826658084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4aa40b41d0 0x7f4aa4154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 >> 172.21.15.82:0/2826658084 conn(0x7f4aa406aef0 msgr2=0x7f4aa40b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 shutdown_connections 2024-03-14T12:14:19.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.401+0000 7f4aabf3b640 1 -- 172.21.15.82:0/2826658084 wait complete. 2024-03-14T12:14:19.441 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters_io (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled perf counters depict the clients performing IO. 2024-03-14T12:14:19.442 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters_io (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-14T12:14:19.442 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:19.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 Processor -- start 2024-03-14T12:14:19.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 -- start start 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c15df30 0x7fed6c06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fed6c06f7f0 0x7fed6c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed6c0b39c0 0x7fed6c071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c0d1590 con 0x7fed6c0b39c0 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c161b70 con 0x7fed6c06f7f0 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.544+0000 7fed71522640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c161e50 con 0x7fed6c15df30 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed6c0b39c0 0x7fed6c071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed6c0b39c0 0x7fed6c071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47426/0 (socket says 172.21.15.82:47426) 2024-03-14T12:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 -- 172.21.15.82:0/941978425 learned_addr learned my addr 172.21.15.82:0/941978425 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:19.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 -- 172.21.15.82:0/941978425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c15df30 msgr2=0x7fed6c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:19.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 --2- 172.21.15.82:0/941978425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c15df30 0x7fed6c06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 -- 172.21.15.82:0/941978425 >> v2:172.21.15.164:3300/0 conn(0x7fed6c06f7f0 msgr2=0x7fed6c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:19.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 --2- 172.21.15.82:0/941978425 >> v2:172.21.15.164:3300/0 conn(0x7fed6c06f7f0 0x7fed6c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 -- 172.21.15.82:0/941978425 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c162130 con 0x7fed6c0b39c0 2024-03-14T12:14:19.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.545+0000 7fed6b7fe640 1 --2- 172.21.15.82:0/941978425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c071740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed6c067a50 tx=0x7fed6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129389655df4fc8c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed69ffb640 1 -- 172.21.15.82:0/941978425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6402ca70 con 0x7fed6c0b39c0 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed69ffb640 1 -- 172.21.15.82:0/941978425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6402cc10 con 0x7fed6c0b39c0 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed69ffb640 1 -- 172.21.15.82:0/941978425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed640343f0 con 0x7fed6c0b39c0 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 -- 172.21.15.82:0/941978425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 msgr2=0x7fed6c071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 --2- 172.21.15.82:0/941978425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c071740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed6c067a50 tx=0x7fed6402bed0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 -- 172.21.15.82:0/941978425 shutdown_connections 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 --2- 172.21.15.82:0/941978425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c071740 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 --2- 172.21.15.82:0/941978425 >> v2:172.21.15.164:3300/0 conn(0x7fed6c06f7f0 0x7fed6c06b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 --2- 172.21.15.82:0/941978425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c15df30 0x7fed6c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 -- 172.21.15.82:0/941978425 >> 172.21.15.82:0/941978425 conn(0x7fed6c0d09d0 msgr2=0x7fed6c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 -- 172.21.15.82:0/941978425 shutdown_connections 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.546+0000 7fed71522640 1 -- 172.21.15.82:0/941978425 wait complete. 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.547+0000 7fed71522640 1 Processor -- start 2024-03-14T12:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed71522640 1 -- start start 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed71522640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c06f7f0 0x7fed6c1f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed71522640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c1f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed71522640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed6c15df30 0x7fed6c145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed71522640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c06fe40 con 0x7fed6c0b39c0 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed71522640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c162860 con 0x7fed6c15df30 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed71522640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c06ffc0 con 0x7fed6c06f7f0 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed6a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c1f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed6a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c1f6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47436/0 (socket says 172.21.15.82:47436) 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.548+0000 7fed6a7fc640 1 -- 172.21.15.82:0/1646780056 learned_addr learned my addr 172.21.15.82:0/1646780056 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed6a7fc640 1 -- 172.21.15.82:0/1646780056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c06f7f0 msgr2=0x7fed6c1f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed6a7fc640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c06f7f0 0x7fed6c1f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed6a7fc640 1 -- 172.21.15.82:0/1646780056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed6c15df30 msgr2=0x7fed6c145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed6a7fc640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed6c15df30 0x7fed6c145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed6a7fc640 1 -- 172.21.15.82:0/1646780056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c0ce8d0 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed6a7fc640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c1f6eb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fed6000b440 tx=0x7fed6000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed537fe640 1 -- 172.21.15.82:0/1646780056 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60015020 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed537fe640 1 -- 172.21.15.82:0/1646780056 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed60011890 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c148800 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed537fe640 1 -- 172.21.15.82:0/1646780056 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60011c80 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.549+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c148dc0 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.550+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c06fbd0 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.550+0000 7fed537fe640 1 -- 172.21.15.82:0/1646780056 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fed60018020 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.551+0000 7fed537fe640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed5806bbd0 0x7fed5806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.551+0000 7fed537fe640 1 -- 172.21.15.82:0/1646780056 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed6008aba0 con 0x7fed6c0b39c0 2024-03-14T12:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.551+0000 7fed6affd640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed5806bbd0 0x7fed5806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.551+0000 7fed6affd640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed5806bbd0 0x7fed5806e090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009710 tx=0x7fed5c006c90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.555+0000 7fed537fe640 1 -- 172.21.15.82:0/1646780056 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fed6001f020 con 0x7fed6c0b39c0 2024-03-14T12:14:19.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.730+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed6c04f050 con 0x7fed6c0b39c0 2024-03-14T12:14:19.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.731+0000 7fed537fe640 1 -- 172.21.15.82:0/1646780056 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fed6000d040 con 0x7fed6c0b39c0 2024-03-14T12:14:19.730 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:19.730 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:19.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed5806bbd0 msgr2=0x7fed5806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed5806bbd0 0x7fed5806e090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009710 tx=0x7fed5c006c90 comp rx=0 tx=0).stop 2024-03-14T12:14:19.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 msgr2=0x7fed6c1f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c1f6eb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fed6000b440 tx=0x7fed6000b900 comp rx=0 tx=0).stop 2024-03-14T12:14:19.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 shutdown_connections 2024-03-14T12:14:19.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed6c15df30 0x7fed6c145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed6c0b39c0 0x7fed6c1f6eb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed5806bbd0 0x7fed5806e090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 --2- 172.21.15.82:0/1646780056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed6c06f7f0 0x7fed6c1f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 >> 172.21.15.82:0/1646780056 conn(0x7fed6c0d09d0 msgr2=0x7fed6c0b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 shutdown_connections 2024-03-14T12:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.733+0000 7fed71522640 1 -- 172.21.15.82:0/1646780056 wait complete. 2024-03-14T12:14:19.743 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:19.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:19.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.847+0000 7fdf169d4640 1 Processor -- start 2024-03-14T12:14:19.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.847+0000 7fdf169d4640 1 -- start start 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf169d4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf169d4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf10161b10 0x7fdf10161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf169d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdf100cfcc0 0x7fdf100d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf169d4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf10162490 con 0x7fdf100cfcc0 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf169d4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1006ee00 con 0x7fdf10161b10 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf169d4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1006f0e0 con 0x7fdf10161580 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdf100cfcc0 0x7fdf100d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdf100cfcc0 0x7fdf100d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47452/0 (socket says 172.21.15.82:47452) 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 -- 172.21.15.82:0/3570500876 learned_addr learned my addr 172.21.15.82:0/3570500876 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf0f7fe640 1 --2- 172.21.15.82:0/3570500876 >> v2:172.21.15.164:3300/0 conn(0x7fdf10161b10 0x7fdf10161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf0ffff640 1 --2- 172.21.15.82:0/3570500876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 -- 172.21.15.82:0/3570500876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 msgr2=0x7fdf100b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 --2- 172.21.15.82:0/3570500876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 -- 172.21.15.82:0/3570500876 >> v2:172.21.15.164:3300/0 conn(0x7fdf10161b10 msgr2=0x7fdf10161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 --2- 172.21.15.82:0/3570500876 >> v2:172.21.15.164:3300/0 conn(0x7fdf10161b10 0x7fdf10161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf14f71640 1 -- 172.21.15.82:0/3570500876 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1006be20 con 0x7fdf100cfcc0 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.848+0000 7fdf0ffff640 1 --2- 172.21.15.82:0/3570500876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.849+0000 7fdf14f71640 1 --2- 172.21.15.82:0/3570500876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf100d0110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdf10072070 tx=0x7fdf0802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf6505ae054994c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.849+0000 7fdf0effd640 1 -- 172.21.15.82:0/3570500876 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0802cb00 con 0x7fdf100cfcc0 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.849+0000 7fdf0effd640 1 -- 172.21.15.82:0/3570500876 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0802cca0 con 0x7fdf100cfcc0 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.849+0000 7fdf0effd640 1 -- 172.21.15.82:0/3570500876 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08034610 con 0x7fdf100cfcc0 2024-03-14T12:14:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.849+0000 7fdf169d4640 1 -- 172.21.15.82:0/3570500876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 msgr2=0x7fdf100d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.849+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3570500876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf100d0110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdf10072070 tx=0x7fdf0802bf60 comp rx=0 tx=0).stop 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.850+0000 7fdf169d4640 1 -- 172.21.15.82:0/3570500876 shutdown_connections 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.850+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3570500876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf100d0110 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.850+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3570500876 >> v2:172.21.15.164:3300/0 conn(0x7fdf10161b10 0x7fdf10161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.850+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3570500876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.850+0000 7fdf169d4640 1 -- 172.21.15.82:0/3570500876 >> 172.21.15.82:0/3570500876 conn(0x7fdf100b3680 msgr2=0x7fdf100b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.850+0000 7fdf169d4640 1 -- 172.21.15.82:0/3570500876 shutdown_connections 2024-03-14T12:14:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.850+0000 7fdf169d4640 1 -- 172.21.15.82:0/3570500876 wait complete. 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.851+0000 7fdf169d4640 1 Processor -- start 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.851+0000 7fdf169d4640 1 -- start start 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf101e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf10161b10 0x7fdf100b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1006e9e0 con 0x7fdf100cfcc0 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1006c3e0 con 0x7fdf10161b10 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1006eb60 con 0x7fdf10161580 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf101e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf101e9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47464/0 (socket says 172.21.15.82:47464) 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 -- 172.21.15.82:0/3308869470 learned_addr learned my addr 172.21.15.82:0/3308869470 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf14f71640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf10161b10 0x7fdf100b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0f7fe640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 -- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 msgr2=0x7fdf100b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 -- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf10161b10 msgr2=0x7fdf100b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf10161b10 0x7fdf100b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 -- 172.21.15.82:0/3308869470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100b5ef0 con 0x7fdf100cfcc0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0ffff640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf101e9bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c7f0 tx=0x7fdf0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0cff9640 1 -- 172.21.15.82:0/3308869470 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04015070 con 0x7fdf100cfcc0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0cff9640 1 -- 172.21.15.82:0/3308869470 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf04007a30 con 0x7fdf100cfcc0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf100b61d0 con 0x7fdf100cfcc0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf0cff9640 1 -- 172.21.15.82:0/3308869470 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04007e00 con 0x7fdf100cfcc0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.852+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf100c6360 con 0x7fdf100cfcc0 2024-03-14T12:14:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.853+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1004f050 con 0x7fdf100cfcc0 2024-03-14T12:14:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.854+0000 7fdf0cff9640 1 -- 172.21.15.82:0/3308869470 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdf04018070 con 0x7fdf100cfcc0 2024-03-14T12:14:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.854+0000 7fdf0cff9640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdefc06bc00 0x7fdefc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.854+0000 7fdf0f7fe640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdefc06bc00 0x7fdefc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.854+0000 7fdf0cff9640 1 -- 172.21.15.82:0/3308869470 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf0408aa40 con 0x7fdf100cfcc0 2024-03-14T12:14:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.855+0000 7fdf0f7fe640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdefc06bc00 0x7fdefc06e0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf00006fd0 tx=0x7fdf00008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:19.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:19.859+0000 7fdf0cff9640 1 -- 172.21.15.82:0/3308869470 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdf04059530 con 0x7fdf100cfcc0 2024-03-14T12:14:20.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.029+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf100c6530 con 0x7fdf100cfcc0 2024-03-14T12:14:20.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.029+0000 7fdf0cff9640 1 -- 172.21.15.82:0/3308869470 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf04058e30 con 0x7fdf100cfcc0 2024-03-14T12:14:20.028 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdefc06bc00 msgr2=0x7fdefc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdefc06bc00 0x7fdefc06e0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf00006fd0 tx=0x7fdf00008040 comp rx=0 tx=0).stop 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 msgr2=0x7fdf101e9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf101e9bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c7f0 tx=0x7fdf0400cbf0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 shutdown_connections 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf10161b10 0x7fdf100b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdefc06bc00 0x7fdefc06e0c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf10161580 0x7fdf100b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 --2- 172.21.15.82:0/3308869470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf100cfcc0 0x7fdf101e9bf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 >> 172.21.15.82:0/3308869470 conn(0x7fdf100b3680 msgr2=0x7fdf1015d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 shutdown_connections 2024-03-14T12:14:20.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.031+0000 7fdf169d4640 1 -- 172.21.15.82:0/3308869470 wait complete. 2024-03-14T12:14:20.041 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.145+0000 7f6bbc137640 1 Processor -- start 2024-03-14T12:14:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.145+0000 7f6bbc137640 1 -- start start 2024-03-14T12:14:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.146+0000 7f6bbc137640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6bb415dd70 0x7f6bb406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.146+0000 7f6bbc137640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.146+0000 7f6bbc137640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb4070250 con 0x7f6bb415e150 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.146+0000 7f6bbc137640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb406ea20 con 0x7f6bb415dd70 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.146+0000 7f6bbc137640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb406f880 con 0x7f6bb40d1b00 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bba6d4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bb9ed3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6bb415dd70 0x7f6bb406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bb96d2640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6bb415e150 0x7f6bb40b6690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:49208/0 (socket says 172.21.15.82:49208) 2024-03-14T12:14:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bb96d2640 1 -- 172.21.15.82:0/1184006944 learned_addr learned my addr 172.21.15.82:0/1184006944 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bb9ed3640 1 -- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 msgr2=0x7f6bb40d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bb9ed3640 1 --2- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bb9ed3640 1 -- 172.21.15.82:0/1184006944 >> v1:172.21.15.164:6789/0 conn(0x7f6bb415e150 legacy=0x7f6bb40b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bb9ed3640 1 -- 172.21.15.82:0/1184006944 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb406ec70 con 0x7f6bb415dd70 2024-03-14T12:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.147+0000 7f6bba6d4640 1 --2- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.148+0000 7f6bb9ed3640 1 --2- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 0x7f6bb406fd10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00097a0 tx=0x7f6bb002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33a2185edfb0b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.148+0000 7f6bb8ed1640 1 -- 172.21.15.82:0/1184006944 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0035020 con 0x7f6bb415dd70 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.148+0000 7f6bb8ed1640 1 -- 172.21.15.82:0/1184006944 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb0031830 con 0x7f6bb415dd70 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.148+0000 7f6bb8ed1640 1 -- 172.21.15.82:0/1184006944 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0031c00 con 0x7f6bb415dd70 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.148+0000 7f6bbc137640 1 -- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 msgr2=0x7f6bb406fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.148+0000 7f6bbc137640 1 --2- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 0x7f6bb406fd10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00097a0 tx=0x7f6bb002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.149+0000 7f6bbc137640 1 -- 172.21.15.82:0/1184006944 shutdown_connections 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.149+0000 7f6bbc137640 1 --2- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.149+0000 7f6bbc137640 1 --2- 172.21.15.82:0/1184006944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 0x7f6bb406fd10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.149+0000 7f6bbc137640 1 -- 172.21.15.82:0/1184006944 >> 172.21.15.82:0/1184006944 conn(0x7f6bb4072ec0 msgr2=0x7f6bb409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.149+0000 7f6bbc137640 1 -- 172.21.15.82:0/1184006944 shutdown_connections 2024-03-14T12:14:20.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.149+0000 7f6bbc137640 1 -- 172.21.15.82:0/1184006944 wait complete. 2024-03-14T12:14:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 Processor -- start 2024-03-14T12:14:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 -- start start 2024-03-14T12:14:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 0x7f6bb41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb415e150 0x7f6bb40b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb406f510 con 0x7f6bb415e150 2024-03-14T12:14:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb406f230 con 0x7f6bb415dd70 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.150+0000 7f6bbc137640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40b6da0 con 0x7f6bb40d1b00 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb415e150 0x7f6bb40b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb415e150 0x7f6bb40b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47474/0 (socket says 172.21.15.82:47474) 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 -- 172.21.15.82:0/652904994 learned_addr learned my addr 172.21.15.82:0/652904994 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bb9ed3640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bb96d2640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 0x7f6bb41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 -- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 msgr2=0x7f6bb40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 -- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 msgr2=0x7f6bb41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 0x7f6bb41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 -- 172.21.15.82:0/652904994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4155630 con 0x7f6bb415e150 2024-03-14T12:14:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bba6d4640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb415e150 0x7f6bb40b2cd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00efc0 tx=0x7f6bac009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6baa7fc640 1 -- 172.21.15.82:0/652904994 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac018030 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6baa7fc640 1 -- 172.21.15.82:0/652904994 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac00cde0 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40b1f10 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6baa7fc640 1 -- 172.21.15.82:0/652904994 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac003bb0 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.151+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40cb330 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.152+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb404f050 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.153+0000 7f6baa7fc640 1 -- 172.21.15.82:0/652904994 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6bac005d10 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.153+0000 7f6baa7fc640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b9806bbd0 0x7f6b9806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.153+0000 7f6baa7fc640 1 -- 172.21.15.82:0/652904994 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bac08b420 con 0x7f6bb415e150 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.153+0000 7f6bb9ed3640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b9806bbd0 0x7f6b9806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.154+0000 7f6bb9ed3640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b9806bbd0 0x7f6b9806e090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.157+0000 7f6baa7fc640 1 -- 172.21.15.82:0/652904994 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6bac022070 con 0x7f6bb415e150 2024-03-14T12:14:20.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.329+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6bb40d29e0 con 0x7f6bb415e150 2024-03-14T12:14:20.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.329+0000 7f6baa7fc640 1 -- 172.21.15.82:0/652904994 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6bac010040 con 0x7f6bb415e150 2024-03-14T12:14:20.328 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:20.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b9806bbd0 msgr2=0x7f6b9806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b9806bbd0 0x7f6b9806e090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb0008030 comp rx=0 tx=0).stop 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb415e150 msgr2=0x7f6bb40b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb415e150 0x7f6bb40b2cd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00efc0 tx=0x7f6bac009f90 comp rx=0 tx=0).stop 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 shutdown_connections 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb415e150 0x7f6bb40b2cd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bb415dd70 0x7f6bb41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b9806bbd0 0x7f6b9806e090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 --2- 172.21.15.82:0/652904994 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bb40d1b00 0x7f6bb40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 >> 172.21.15.82:0/652904994 conn(0x7f6bb4072ec0 msgr2=0x7f6bb40afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 shutdown_connections 2024-03-14T12:14:20.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.331+0000 7f6bbc137640 1 -- 172.21.15.82:0/652904994 wait complete. 2024-03-14T12:14:20.341 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.446+0000 7f1aad390640 1 Processor -- start 2024-03-14T12:14:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.446+0000 7f1aad390640 1 -- start start 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.446+0000 7f1aad390640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1aa80d0e10 0x7f1aa8168c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.446+0000 7f1aad390640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa80d25a0 con 0x7f1aa80d12a0 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.446+0000 7f1aad390640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa80d2880 con 0x7f1aa815dff0 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.446+0000 7f1aad390640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa80d2350 con 0x7f1aa80d0e10 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1aa80d0e10 0x7f1aa8168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa67fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1aa80d12a0 0x7f1aa80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51726/0 (socket says 172.21.15.82:51726) 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1aa80d0e10 0x7f1aa8168c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47490/0 (socket says 172.21.15.82:47490) 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa67fc640 1 -- 172.21.15.82:0/2688472107 learned_addr learned my addr 172.21.15.82:0/2688472107 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa77fe640 1 -- 172.21.15.82:0/2688472107 >> v1:172.21.15.164:6789/0 conn(0x7f1aa815dff0 legacy=0x7f1aa80d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa77fe640 1 -- 172.21.15.82:0/2688472107 >> v1:172.21.15.82:6789/0 conn(0x7f1aa80d12a0 legacy=0x7f1aa80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa77fe640 1 -- 172.21.15.82:0/2688472107 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80d2b60 con 0x7f1aa80d0e10 2024-03-14T12:14:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.447+0000 7f1aa5ffb640 1 -- 172.21.15.82:0/2688472107 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377836371 0 0) 0x7f1aa80d25a0 con 0x7f1aa80d12a0 2024-03-14T12:14:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aa77fe640 1 --2- 172.21.15.82:0/2688472107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa80d0e10 0x7f1aa8168c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80d24d0 tx=0x7f1aa002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0b1a3b93a5bc92 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aa5ffb640 1 -- 172.21.15.82:0/2688472107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0039070 con 0x7f1aa80d0e10 2024-03-14T12:14:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aa5ffb640 1 -- 172.21.15.82:0/2688472107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa002cc50 con 0x7f1aa80d0e10 2024-03-14T12:14:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aa5ffb640 1 -- 172.21.15.82:0/2688472107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0034440 con 0x7f1aa80d0e10 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aad390640 1 -- 172.21.15.82:0/2688472107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa80d0e10 msgr2=0x7f1aa8168c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aad390640 1 --2- 172.21.15.82:0/2688472107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa80d0e10 0x7f1aa8168c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80d24d0 tx=0x7f1aa002bea0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aad390640 1 -- 172.21.15.82:0/2688472107 shutdown_connections 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aad390640 1 --2- 172.21.15.82:0/2688472107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa80d0e10 0x7f1aa8168c90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aad390640 1 -- 172.21.15.82:0/2688472107 >> 172.21.15.82:0/2688472107 conn(0x7f1aa809d0d0 msgr2=0x7f1aa80d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aad390640 1 -- 172.21.15.82:0/2688472107 shutdown_connections 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.448+0000 7f1aad390640 1 -- 172.21.15.82:0/2688472107 wait complete. 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 Processor -- start 2024-03-14T12:14:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 -- start start 2024-03-14T12:14:20.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aa80d0e10 0x7f1aa80ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aa80d12a0 0x7f1aa81563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa815dff0 0x7f1aa80b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80cc4b0 con 0x7f1aa815dff0 2024-03-14T12:14:20.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80cc330 con 0x7f1aa80d0e10 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aad390640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80cc630 con 0x7f1aa80d12a0 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aa77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa815dff0 0x7f1aa80b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aa77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa815dff0 0x7f1aa80b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47500/0 (socket says 172.21.15.82:47500) 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.450+0000 7f1aa77fe640 1 -- 172.21.15.82:0/618507531 learned_addr learned my addr 172.21.15.82:0/618507531 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa67fc640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aa80d12a0 0x7f1aa81563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa77fe640 1 -- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aa80d12a0 msgr2=0x7f1aa81563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa77fe640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aa80d12a0 0x7f1aa81563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa6ffd640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aa80d0e10 0x7f1aa80ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa77fe640 1 -- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aa80d0e10 msgr2=0x7f1aa80ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa77fe640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aa80d0e10 0x7f1aa80ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa77fe640 1 -- 172.21.15.82:0/618507531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80b75f0 con 0x7f1aa815dff0 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aa77fe640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa815dff0 0x7f1aa80b8c60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1aa002c4b0 tx=0x7f1aa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1a8f7fe640 1 -- 172.21.15.82:0/618507531 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0044020 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1a8f7fe640 1 -- 172.21.15.82:0/618507531 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa002ca50 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa80b78d0 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1a8f7fe640 1 -- 172.21.15.82:0/618507531 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0039040 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.451+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa81573e0 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.452+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa804f050 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.453+0000 7f1a8f7fe640 1 -- 172.21.15.82:0/618507531 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1aa0047020 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.453+0000 7f1a8f7fe640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1a9406bc00 0x7f1a9406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.453+0000 7f1a8f7fe640 1 -- 172.21.15.82:0/618507531 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aa00ad220 con 0x7f1aa815dff0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.453+0000 7f1aa6ffd640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1a9406bc00 0x7f1a9406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.453+0000 7f1aa6ffd640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1a9406bc00 0x7f1a9406e0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800a730 tx=0x7f1a98008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.457+0000 7f1a8f7fe640 1 -- 172.21.15.82:0/618507531 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1aa004e070 con 0x7f1aa815dff0 2024-03-14T12:14:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.629+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1aa80cfd10 con 0x7f1aa815dff0 2024-03-14T12:14:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.630+0000 7f1a8f7fe640 1 -- 172.21.15.82:0/618507531 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1aa002cc20 con 0x7f1aa815dff0 2024-03-14T12:14:20.629 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:20.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1a9406bc00 msgr2=0x7f1a9406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1a9406bc00 0x7f1a9406e0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800a730 tx=0x7f1a98008040 comp rx=0 tx=0).stop 2024-03-14T12:14:20.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa815dff0 msgr2=0x7f1aa80b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa815dff0 0x7f1aa80b8c60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1aa002c4b0 tx=0x7f1aa0004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:20.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 shutdown_connections 2024-03-14T12:14:20.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa815dff0 0x7f1aa80b8c60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1a9406bc00 0x7f1a9406e0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aa80d12a0 0x7f1aa81563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 --2- 172.21.15.82:0/618507531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aa80d0e10 0x7f1aa80ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 >> 172.21.15.82:0/618507531 conn(0x7f1aa809d0d0 msgr2=0x7f1aa806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:20.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 shutdown_connections 2024-03-14T12:14:20.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.632+0000 7f1aad390640 1 -- 172.21.15.82:0/618507531 wait complete. 2024-03-14T12:14:20.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io' 2024-03-14T12:14:20.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.742+0000 7f9983e0b640 1 Processor -- start 2024-03-14T12:14:20.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.742+0000 7f9983e0b640 1 -- start start 2024-03-14T12:14:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f9983e0b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f9983e0b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f997c15e670 0x7f997c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f9983e0b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c072060 con 0x7f997c143f60 2024-03-14T12:14:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f9983e0b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997c15ebe0 con 0x7f997c15e670 2024-03-14T12:14:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f9983e0b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c15eec0 con 0x7f997c071740 2024-03-14T12:14:20.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f99813a6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f997c15e670 0x7f997c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f9981ba7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f99813a6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f997c15e670 0x7f997c0b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:44164/0 (socket says 172.21.15.82:44164) 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f99823a8640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f997c143f60 0x7f997c15b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:49220/0 (socket says 172.21.15.82:49220) 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f9981ba7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:41704/0 (socket says 172.21.15.82:41704) 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.743+0000 7f99813a6640 1 -- 172.21.15.82:0/2608486185 learned_addr learned my addr 172.21.15.82:0/2608486185 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.744+0000 7f9981ba7640 1 -- 172.21.15.82:0/2608486185 >> v2:172.21.15.164:3300/0 conn(0x7f997c15e670 msgr2=0x7f997c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.744+0000 7f9981ba7640 1 --2- 172.21.15.82:0/2608486185 >> v2:172.21.15.164:3300/0 conn(0x7f997c15e670 0x7f997c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.744+0000 7f9981ba7640 1 -- 172.21.15.82:0/2608486185 >> v1:172.21.15.164:6789/0 conn(0x7f997c143f60 legacy=0x7f997c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.744+0000 7f9981ba7640 1 -- 172.21.15.82:0/2608486185 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c15f1a0 con 0x7f997c071740 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.744+0000 7f99813a6640 1 --2- 172.21.15.82:0/2608486185 >> v2:172.21.15.164:3300/0 conn(0x7f997c15e670 0x7f997c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:20.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9981ba7640 1 --2- 172.21.15.82:0/2608486185 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c071b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9978009d90 tx=0x7f997802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a995eab13c71254e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9980ba5640 1 -- 172.21.15.82:0/2608486185 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978039070 con 0x7f997c071740 2024-03-14T12:14:20.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9980ba5640 1 -- 172.21.15.82:0/2608486185 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997802cc50 con 0x7f997c071740 2024-03-14T12:14:20.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9980ba5640 1 -- 172.21.15.82:0/2608486185 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978034440 con 0x7f997c071740 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 -- 172.21.15.82:0/2608486185 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 msgr2=0x7f997c071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 --2- 172.21.15.82:0/2608486185 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c071b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9978009d90 tx=0x7f997802bea0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 -- 172.21.15.82:0/2608486185 shutdown_connections 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 --2- 172.21.15.82:0/2608486185 >> v2:172.21.15.164:3300/0 conn(0x7f997c15e670 0x7f997c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 --2- 172.21.15.82:0/2608486185 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c071b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 -- 172.21.15.82:0/2608486185 >> 172.21.15.82:0/2608486185 conn(0x7f997c0b1c70 msgr2=0x7f997c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 -- 172.21.15.82:0/2608486185 shutdown_connections 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.745+0000 7f9983e0b640 1 -- 172.21.15.82:0/2608486185 wait complete. 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.746+0000 7f9983e0b640 1 Processor -- start 2024-03-14T12:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.746+0000 7f9983e0b640 1 -- start start 2024-03-14T12:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f9983e0b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f9983e0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f997c143f60 0x7f997c13d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f9983e0b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997c15e670 0x7f997c1f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f9983e0b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c15b080 con 0x7f997c143f60 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f9983e0b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c143ce0 con 0x7f997c15e670 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f9983e0b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c144670 con 0x7f997c071740 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f99813a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f997c143f60 0x7f997c13d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f99813a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f997c143f60 0x7f997c13d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47508/0 (socket says 172.21.15.82:47508) 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f99813a6640 1 -- 172.21.15.82:0/3751669847 learned_addr learned my addr 172.21.15.82:0/3751669847 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f9981ba7640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.747+0000 7f99823a8640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997c15e670 0x7f997c1f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99813a6640 1 -- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 msgr2=0x7f997c1f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99813a6640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c1f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99813a6640 1 -- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997c15e670 msgr2=0x7f997c1f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99813a6640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997c15e670 0x7f997c1f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99813a6640 1 -- 172.21.15.82:0/3751669847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c13ded0 con 0x7f997c143f60 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99813a6640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f997c143f60 0x7f997c13d6a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f996c00ea00 tx=0x7f996c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99727fc640 1 -- 172.21.15.82:0/3751669847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c017070 con 0x7f997c143f60 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99727fc640 1 -- 172.21.15.82:0/3751669847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00ca60 con 0x7f997c143f60 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f99727fc640 1 -- 172.21.15.82:0/3751669847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00ce10 con 0x7f997c143f60 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c13e1b0 con 0x7f997c143f60 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.748+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c13dbe0 con 0x7f997c143f60 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.749+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c04f050 con 0x7f997c143f60 2024-03-14T12:14:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.749+0000 7f99727fc640 1 -- 172.21.15.82:0/3751669847 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f996c01a070 con 0x7f997c143f60 2024-03-14T12:14:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.750+0000 7f99727fc640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f996006bbd0 0x7f996006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.750+0000 7f9981ba7640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f996006bbd0 0x7f996006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.750+0000 7f99727fc640 1 -- 172.21.15.82:0/3751669847 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f996c08ac50 con 0x7f997c143f60 2024-03-14T12:14:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.750+0000 7f9981ba7640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f996006bbd0 0x7f996006e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9978036000 tx=0x7f9978030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.754+0000 7f99727fc640 1 -- 172.21.15.82:0/3751669847 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f996c090020 con 0x7f997c143f60 2024-03-14T12:14:20.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:20.989+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io"]} v 0) v1 -- 0x7f997c0d07b0 con 0x7f997c143f60 2024-03-14T12:14:21.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:14:21.524 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:14:21.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.672+0000 7f99727fc640 1 -- 172.21.15.82:0/3751669847 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io"]}]=0 v51) v1 ==== 170+0+0 (secure 0 0 0) 0x7f996c059180 con 0x7f997c143f60 2024-03-14T12:14:21.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f996006bbd0 msgr2=0x7f996006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f996006bbd0 0x7f996006e090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9978036000 tx=0x7f9978030000 comp rx=0 tx=0).stop 2024-03-14T12:14:21.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f997c143f60 msgr2=0x7f997c13d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f997c143f60 0x7f997c13d6a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f996c00ea00 tx=0x7f996c00eec0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 shutdown_connections 2024-03-14T12:14:21.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997c15e670 0x7f997c1f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f997c143f60 0x7f997c13d6a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f996006bbd0 0x7f996006e090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 --2- 172.21.15.82:0/3751669847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f997c071740 0x7f997c1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 >> 172.21.15.82:0/3751669847 conn(0x7f997c0b1c70 msgr2=0x7f997c138d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:21.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 shutdown_connections 2024-03-14T12:14:21.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.674+0000 7f9983e0b640 1 -- 172.21.15.82:0/3751669847 wait complete. 2024-03-14T12:14:21.684 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-14T12:14:21.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.788+0000 7f3629206640 1 Processor -- start 2024-03-14T12:14:21.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f3629206640 1 -- start start 2024-03-14T12:14:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f3629206640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f362415dff0 0x7f36240d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f3629206640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d0e10 0x7f36240cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f3629206640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36240d25a0 con 0x7f36240d12a0 2024-03-14T12:14:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f3629206640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36240d2350 con 0x7f362415dff0 2024-03-14T12:14:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f3629206640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240d2880 con 0x7f36240d0e10 2024-03-14T12:14:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f3622d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f362415dff0 0x7f36240d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f362259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f36240d12a0 0x7f36240d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50920/0 (socket says 172.21.15.82:50920) 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.789+0000 7f362259c640 1 -- 172.21.15.82:0/1851786924 learned_addr learned my addr 172.21.15.82:0/1851786924 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f362359e640 1 --2- 172.21.15.82:0/1851786924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d0e10 0x7f36240cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f3622d9d640 1 -- 172.21.15.82:0/1851786924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d0e10 msgr2=0x7f36240cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f3622d9d640 1 --2- 172.21.15.82:0/1851786924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d0e10 0x7f36240cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f3622d9d640 1 -- 172.21.15.82:0/1851786924 >> v1:172.21.15.82:6789/0 conn(0x7f36240d12a0 legacy=0x7f36240d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f3621d9b640 1 -- 172.21.15.82:0/1851786924 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813962352 0 0) 0x7f36240d25a0 con 0x7f36240d12a0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f3622d9d640 1 -- 172.21.15.82:0/1851786924 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240d2b60 con 0x7f362415dff0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f362359e640 1 --2- 172.21.15.82:0/1851786924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d0e10 0x7f36240cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.790+0000 7f3622d9d640 1 --2- 172.21.15.82:0/1851786924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f36240d4480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f36180097a0 tx=0x7f361802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c2f7f6454134e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3621d9b640 1 -- 172.21.15.82:0/1851786924 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618035020 con 0x7f362415dff0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3621d9b640 1 -- 172.21.15.82:0/1851786924 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618031830 con 0x7f362415dff0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3621d9b640 1 -- 172.21.15.82:0/1851786924 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618031c00 con 0x7f362415dff0 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 -- 172.21.15.82:0/1851786924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 msgr2=0x7f36240d4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 --2- 172.21.15.82:0/1851786924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f36240d4480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f36180097a0 tx=0x7f361802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 -- 172.21.15.82:0/1851786924 shutdown_connections 2024-03-14T12:14:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 --2- 172.21.15.82:0/1851786924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d0e10 0x7f36240cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 --2- 172.21.15.82:0/1851786924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f36240d4480 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 -- 172.21.15.82:0/1851786924 >> 172.21.15.82:0/1851786924 conn(0x7f362409d0d0 msgr2=0x7f36240d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 -- 172.21.15.82:0/1851786924 shutdown_connections 2024-03-14T12:14:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.791+0000 7f3629206640 1 -- 172.21.15.82:0/1851786924 wait complete. 2024-03-14T12:14:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 Processor -- start 2024-03-14T12:14:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 -- start start 2024-03-14T12:14:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240d0e10 0x7f362415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d12a0 0x7f36240cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f3624151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362415e550 con 0x7f362415dff0 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362415e3d0 con 0x7f36240d0e10 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f3629206640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240cb860 con 0x7f36240d12a0 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f362359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f3624151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f362359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f3624151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39280/0 (socket says 172.21.15.82:39280) 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.793+0000 7f362359e640 1 -- 172.21.15.82:0/2220306890 learned_addr learned my addr 172.21.15.82:0/2220306890 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362259c640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d12a0 0x7f36240cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f3622d9d640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240d0e10 0x7f362415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362359e640 1 -- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d12a0 msgr2=0x7f36240cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362359e640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d12a0 0x7f36240cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362359e640 1 -- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240d0e10 msgr2=0x7f362415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362359e640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240d0e10 0x7f362415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362359e640 1 -- 172.21.15.82:0/2220306890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240c85e0 con 0x7f362415dff0 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362259c640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d12a0 0x7f36240cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f3622d9d640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240d0e10 0x7f362415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f362359e640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f3624151190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f361c00cf20 tx=0x7f361c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f360b7fe640 1 -- 172.21.15.82:0/2220306890 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c01a070 con 0x7f362415dff0 2024-03-14T12:14:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f360b7fe640 1 -- 172.21.15.82:0/2220306890 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00d840 con 0x7f362415dff0 2024-03-14T12:14:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3624151e10 con 0x7f362415dff0 2024-03-14T12:14:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f360b7fe640 1 -- 172.21.15.82:0/2220306890 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00dc30 con 0x7f362415dff0 2024-03-14T12:14:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.794+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362415aac0 con 0x7f362415dff0 2024-03-14T12:14:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.795+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362404f050 con 0x7f362415dff0 2024-03-14T12:14:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.795+0000 7f360b7fe640 1 -- 172.21.15.82:0/2220306890 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f361c01d070 con 0x7f362415dff0 2024-03-14T12:14:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.796+0000 7f360b7fe640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361006bbd0 0x7f361006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:21.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.796+0000 7f360b7fe640 1 -- 172.21.15.82:0/2220306890 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f361c08b2a0 con 0x7f362415dff0 2024-03-14T12:14:21.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.796+0000 7f3622d9d640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361006bbd0 0x7f361006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:21.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.796+0000 7f3622d9d640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361006bbd0 0x7f361006e090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3618036000 tx=0x7f3618008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:21.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.800+0000 7f360b7fe640 1 -- 172.21.15.82:0/2220306890 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f361c059e20 con 0x7f362415dff0 2024-03-14T12:14:21.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.971+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f36240cbf00 con 0x7f362415dff0 2024-03-14T12:14:21.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.974+0000 7f360b7fe640 1 -- 172.21.15.82:0/2220306890 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f361c08b0c0 con 0x7f362415dff0 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361006bbd0 msgr2=0x7f361006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361006bbd0 0x7f361006e090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3618036000 tx=0x7f3618008030 comp rx=0 tx=0).stop 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 msgr2=0x7f3624151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f3624151190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f361c00cf20 tx=0x7f361c00a7d0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 shutdown_connections 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362415dff0 0x7f3624151190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361006bbd0 0x7f361006e090 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36240d12a0 0x7f36240cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 --2- 172.21.15.82:0/2220306890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36240d0e10 0x7f362415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 >> 172.21.15.82:0/2220306890 conn(0x7f362409d0d0 msgr2=0x7f36240d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:21.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 shutdown_connections 2024-03-14T12:14:21.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:21.976+0000 7f3629206640 1 -- 172.21.15.82:0/2220306890 wait complete. 2024-03-14T12:14:21.987 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:14:21.987 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:14:22.025 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:14:22.075 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:14:22.075 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:22.075 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:22.091 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:22.091 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:22.146 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:22.146 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:22.201 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:22.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.342+0000 7f028964a640 1 Processor -- start 2024-03-14T12:14:22.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.342+0000 7f028964a640 1 -- start start 2024-03-14T12:14:22.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f028964a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f028415df30 0x7f028406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f028964a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0284072560 con 0x7f02840b39c0 2024-03-14T12:14:22.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f028964a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02840d1590 con 0x7f028406f7f0 2024-03-14T12:14:22.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f028964a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0284161b70 con 0x7f028415df30 2024-03-14T12:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f0282ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f028415df30 0x7f028406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f0282ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f028415df30 0x7f028406ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39296/0 (socket says 172.21.15.82:39296) 2024-03-14T12:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f0282ffd640 1 -- 172.21.15.82:0/3830290818 learned_addr learned my addr 172.21.15.82:0/3830290818 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f0282ffd640 1 -- 172.21.15.82:0/3830290818 >> v1:172.21.15.164:6789/0 conn(0x7f028406f7f0 legacy=0x7f0284071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f0282ffd640 1 -- 172.21.15.82:0/3830290818 >> v1:172.21.15.82:6789/0 conn(0x7f02840b39c0 legacy=0x7f0284071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f0282ffd640 1 -- 172.21.15.82:0/3830290818 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284161e50 con 0x7f028415df30 2024-03-14T12:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.343+0000 7f02827fc640 1 -- 172.21.15.82:0/3830290818 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143765257 0 0) 0x7f0284072560 con 0x7f02840b39c0 2024-03-14T12:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.344+0000 7f0282ffd640 1 --2- 172.21.15.82:0/3830290818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028415df30 0x7f028406ad10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0274009870 tx=0x7f027402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d2e1a6c070cd23 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.344+0000 7f02827fc640 1 -- 172.21.15.82:0/3830290818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027402f840 con 0x7f028415df30 2024-03-14T12:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.344+0000 7f02827fc640 1 -- 172.21.15.82:0/3830290818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027402f9e0 con 0x7f028415df30 2024-03-14T12:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.344+0000 7f02827fc640 1 -- 172.21.15.82:0/3830290818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027402fdb0 con 0x7f028415df30 2024-03-14T12:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.345+0000 7f028964a640 1 -- 172.21.15.82:0/3830290818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028415df30 msgr2=0x7f028406ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.345+0000 7f028964a640 1 --2- 172.21.15.82:0/3830290818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028415df30 0x7f028406ad10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0274009870 tx=0x7f027402ba20 comp rx=0 tx=0).stop 2024-03-14T12:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.345+0000 7f028964a640 1 -- 172.21.15.82:0/3830290818 shutdown_connections 2024-03-14T12:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.345+0000 7f028964a640 1 --2- 172.21.15.82:0/3830290818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028415df30 0x7f028406ad10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.345+0000 7f028964a640 1 -- 172.21.15.82:0/3830290818 >> 172.21.15.82:0/3830290818 conn(0x7f02840d09d0 msgr2=0x7f02840d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.345+0000 7f028964a640 1 -- 172.21.15.82:0/3830290818 shutdown_connections 2024-03-14T12:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.345+0000 7f028964a640 1 -- 172.21.15.82:0/3830290818 wait complete. 2024-03-14T12:14:22.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.346+0000 7f028964a640 1 Processor -- start 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.346+0000 7f028964a640 1 -- start start 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f028964a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f028406f7f0 0x7f02841f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f028964a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02840b39c0 0x7f0284145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f028964a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f028415df30 0x7f02840ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f028964a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02841627d0 con 0x7f02840b39c0 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f028964a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0284162650 con 0x7f028406f7f0 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f028964a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0284162950 con 0x7f028415df30 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02840b39c0 0x7f0284145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02840b39c0 0x7f0284145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39312/0 (socket says 172.21.15.82:39312) 2024-03-14T12:14:22.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 -- 172.21.15.82:0/3319756800 learned_addr learned my addr 172.21.15.82:0/3319756800 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 -- 172.21.15.82:0/3319756800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f028415df30 msgr2=0x7f02840ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f0282ffd640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f028406f7f0 0x7f02841f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f028415df30 0x7f02840ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 -- 172.21.15.82:0/3319756800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f028406f7f0 msgr2=0x7f02841f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f028406f7f0 0x7f02841f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 -- 172.21.15.82:0/3319756800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284146260 con 0x7f02840b39c0 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027bfff640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02840b39c0 0x7f0284145880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c450 tx=0x7f026c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027b7fe640 1 -- 172.21.15.82:0/3319756800 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c017070 con 0x7f02840b39c0 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.347+0000 7f027b7fe640 1 -- 172.21.15.82:0/3319756800 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c0052d0 con 0x7f02840b39c0 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.348+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0284146540 con 0x7f02840b39c0 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.348+0000 7f027b7fe640 1 -- 172.21.15.82:0/3319756800 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c003830 con 0x7f02840b39c0 2024-03-14T12:14:22.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.348+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02841f4420 con 0x7f02840b39c0 2024-03-14T12:14:22.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.348+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028415ee40 con 0x7f02840b39c0 2024-03-14T12:14:22.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.349+0000 7f027b7fe640 1 -- 172.21.15.82:0/3319756800 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f026c01a070 con 0x7f02840b39c0 2024-03-14T12:14:22.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.349+0000 7f027b7fe640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026406bbd0 0x7f026406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.349+0000 7f027b7fe640 1 -- 172.21.15.82:0/3319756800 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f026c089e10 con 0x7f02840b39c0 2024-03-14T12:14:22.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.349+0000 7f0282ffd640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026406bbd0 0x7f026406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.350+0000 7f0282ffd640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026406bbd0 0x7f026406e090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0274036000 tx=0x7f0274039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.354+0000 7f027b7fe640 1 -- 172.21.15.82:0/3319756800 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f026c058900 con 0x7f02840b39c0 2024-03-14T12:14:22.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.528+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02841f3f30 con 0x7f02840b39c0 2024-03-14T12:14:22.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.529+0000 7f027b7fe640 1 -- 172.21.15.82:0/3319756800 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f026c058200 con 0x7f02840b39c0 2024-03-14T12:14:22.528 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:22.528 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:22.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026406bbd0 msgr2=0x7f026406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026406bbd0 0x7f026406e090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0274036000 tx=0x7f0274039040 comp rx=0 tx=0).stop 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02840b39c0 msgr2=0x7f0284145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02840b39c0 0x7f0284145880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c450 tx=0x7f026c00c910 comp rx=0 tx=0).stop 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 shutdown_connections 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f026406bbd0 0x7f026406e090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f028415df30 0x7f02840ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02840b39c0 0x7f0284145880 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 --2- 172.21.15.82:0/3319756800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f028406f7f0 0x7f02841f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 >> 172.21.15.82:0/3319756800 conn(0x7f02840d09d0 msgr2=0x7f02840b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 shutdown_connections 2024-03-14T12:14:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.531+0000 7f028964a640 1 -- 172.21.15.82:0/3319756800 wait complete. 2024-03-14T12:14:22.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:22.541 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:22.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.645+0000 7f8a32a32640 1 Processor -- start 2024-03-14T12:14:22.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.645+0000 7f8a32a32640 1 -- start start 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a32a32640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8a2c15c2f0 0x7f8a2c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a32a32640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c0670c0 con 0x7f8a2c15bf10 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a32a32640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c06c130 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a32a32640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2c0b62e0 con 0x7f8a2c15c2f0 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a2b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8a2c15c2f0 0x7f8a2c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a2bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8a2c15bf10 0x7f8a2c0af5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50928/0 (socket says 172.21.15.82:50928) 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a2bfff640 1 -- 172.21.15.82:0/1509302333 learned_addr learned my addr 172.21.15.82:0/1509302333 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a2b7fe640 1 -- 172.21.15.82:0/1509302333 >> v1:172.21.15.164:6789/0 conn(0x7f8a2c0b1e00 legacy=0x7f8a2c142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a2b7fe640 1 -- 172.21.15.82:0/1509302333 >> v1:172.21.15.82:6789/0 conn(0x7f8a2c15bf10 legacy=0x7f8a2c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.646+0000 7f8a2b7fe640 1 -- 172.21.15.82:0/1509302333 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c066de0 con 0x7f8a2c15c2f0 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a2b7fe640 1 --2- 172.21.15.82:0/1509302333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c0b5da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097a0 tx=0x7f8a1c02b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e4ddf2d1bc8de3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a2affd640 1 -- 172.21.15.82:0/1509302333 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c037070 con 0x7f8a2c15c2f0 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a2affd640 1 -- 172.21.15.82:0/1509302333 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c030710 con 0x7f8a2c15c2f0 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a2affd640 1 -- 172.21.15.82:0/1509302333 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c030ae0 con 0x7f8a2c15c2f0 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a32a32640 1 -- 172.21.15.82:0/1509302333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c15c2f0 msgr2=0x7f8a2c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a32a32640 1 --2- 172.21.15.82:0/1509302333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c0b5da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097a0 tx=0x7f8a1c02b680 comp rx=0 tx=0).stop 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a32a32640 1 -- 172.21.15.82:0/1509302333 shutdown_connections 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.647+0000 7f8a32a32640 1 --2- 172.21.15.82:0/1509302333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c0b5da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.648+0000 7f8a32a32640 1 -- 172.21.15.82:0/1509302333 >> 172.21.15.82:0/1509302333 conn(0x7f8a2c0b4fd0 msgr2=0x7f8a2c0b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.648+0000 7f8a32a32640 1 -- 172.21.15.82:0/1509302333 shutdown_connections 2024-03-14T12:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.648+0000 7f8a32a32640 1 -- 172.21.15.82:0/1509302333 wait complete. 2024-03-14T12:14:22.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 Processor -- start 2024-03-14T12:14:22.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 -- start start 2024-03-14T12:14:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c0b1e00 0x7f8a2c099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a2c15bf10 0x7f8a2c153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c09d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c145b60 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c06c740 con 0x7f8a2c15c2f0 2024-03-14T12:14:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.649+0000 7f8a32a32640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c066a20 con 0x7f8a2c15bf10 2024-03-14T12:14:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c0b1e00 0x7f8a2c099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c0b1e00 0x7f8a2c099ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39330/0 (socket says 172.21.15.82:39330) 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 -- 172.21.15.82:0/2801439218 learned_addr learned my addr 172.21.15.82:0/2801439218 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2b7fe640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a2c15bf10 0x7f8a2c153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a30fcf640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c09d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 -- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a2c15bf10 msgr2=0x7f8a2c153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a2c15bf10 0x7f8a2c153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 -- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2c15c2f0 msgr2=0x7f8a2c09d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c09d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 -- 172.21.15.82:0/2801439218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c153f80 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2b7fe640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a2c15bf10 0x7f8a2c153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a30fcf640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c09d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a2bfff640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c0b1e00 0x7f8a2c099ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c780 tx=0x7f8a2000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a28ff9640 1 -- 172.21.15.82:0/2801439218 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20018070 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a28ff9640 1 -- 172.21.15.82:0/2801439218 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a200072d0 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c154260 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a28ff9640 1 -- 172.21.15.82:0/2801439218 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200047e0 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.650+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c154820 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.651+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c04f050 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.652+0000 7f8a28ff9640 1 -- 172.21.15.82:0/2801439218 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8a20004980 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.652+0000 7f8a28ff9640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8a1806bbd0 0x7f8a1806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.652+0000 7f8a2b7fe640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8a1806bbd0 0x7f8a1806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.652+0000 7f8a28ff9640 1 -- 172.21.15.82:0/2801439218 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a2008a020 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.652+0000 7f8a2b7fe640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8a1806bbd0 0x7f8a1806e090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c034000 tx=0x7f8a1c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.656+0000 7f8a28ff9640 1 -- 172.21.15.82:0/2801439218 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8a20004c70 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.828+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a2c15b160 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.829+0000 7f8a28ff9640 1 -- 172.21.15.82:0/2801439218 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a20058a90 con 0x7f8a2c0b1e00 2024-03-14T12:14:22.828 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8a1806bbd0 msgr2=0x7f8a1806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8a1806bbd0 0x7f8a1806e090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c034000 tx=0x7f8a1c008030 comp rx=0 tx=0).stop 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c0b1e00 msgr2=0x7f8a2c099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c0b1e00 0x7f8a2c099ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c780 tx=0x7f8a2000cc40 comp rx=0 tx=0).stop 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 shutdown_connections 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a2c15c2f0 0x7f8a2c09d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8a1806bbd0 0x7f8a1806e090 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a2c15bf10 0x7f8a2c153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 --2- 172.21.15.82:0/2801439218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a2c0b1e00 0x7f8a2c099ac0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 >> 172.21.15.82:0/2801439218 conn(0x7f8a2c0b4fd0 msgr2=0x7f8a2c158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:22.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 shutdown_connections 2024-03-14T12:14:22.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.831+0000 7f8a32a32640 1 -- 172.21.15.82:0/2801439218 wait complete. 2024-03-14T12:14:22.841 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:22.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.944+0000 7f31bd729640 1 Processor -- start 2024-03-14T12:14:22.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.944+0000 7f31bd729640 1 -- start start 2024-03-14T12:14:22.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31bd729640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31b81566e0 0x7f31b815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31bd729640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f31b806b720 0x7f31b806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31bd729640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b806c070 con 0x7f31b80af3d0 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31bd729640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b8067040 con 0x7f31b806b720 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31bd729640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b8066ac0 con 0x7f31b81566e0 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31b77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f31b806b720 0x7f31b806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31b77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f31b806b720 0x7f31b806bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39332/0 (socket says 172.21.15.82:39332) 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31b77fe640 1 -- 172.21.15.82:0/4115061137 learned_addr learned my addr 172.21.15.82:0/4115061137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.945+0000 7f31b6ffd640 1 --2- 172.21.15.82:0/4115061137 >> v2:172.21.15.164:3300/0 conn(0x7f31b81566e0 0x7f31b815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b77fe640 1 -- 172.21.15.82:0/4115061137 >> v2:172.21.15.164:3300/0 conn(0x7f31b81566e0 msgr2=0x7f31b815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b77fe640 1 --2- 172.21.15.82:0/4115061137 >> v2:172.21.15.164:3300/0 conn(0x7f31b81566e0 0x7f31b815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b77fe640 1 -- 172.21.15.82:0/4115061137 >> v1:172.21.15.164:6789/0 conn(0x7f31b80af3d0 legacy=0x7f31b80b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b77fe640 1 -- 172.21.15.82:0/4115061137 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b80b5f20 con 0x7f31b806b720 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b77fe640 1 --2- 172.21.15.82:0/4115061137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b806bb30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f31b8071ff0 tx=0x7f31b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e351616b5a63a53c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b5ffb640 1 -- 172.21.15.82:0/4115061137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b002e070 con 0x7f31b806b720 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b5ffb640 1 -- 172.21.15.82:0/4115061137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b0037070 con 0x7f31b806b720 2024-03-14T12:14:22.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.946+0000 7f31b5ffb640 1 -- 172.21.15.82:0/4115061137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0031a00 con 0x7f31b806b720 2024-03-14T12:14:22.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 -- 172.21.15.82:0/4115061137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 msgr2=0x7f31b806bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 --2- 172.21.15.82:0/4115061137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b806bb30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f31b8071ff0 tx=0x7f31b002b880 comp rx=0 tx=0).stop 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 -- 172.21.15.82:0/4115061137 shutdown_connections 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 --2- 172.21.15.82:0/4115061137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b806bb30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 --2- 172.21.15.82:0/4115061137 >> v2:172.21.15.164:3300/0 conn(0x7f31b81566e0 0x7f31b815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 -- 172.21.15.82:0/4115061137 >> 172.21.15.82:0/4115061137 conn(0x7f31b80b4fd0 msgr2=0x7f31b80b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 -- 172.21.15.82:0/4115061137 shutdown_connections 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.947+0000 7f31bd729640 1 -- 172.21.15.82:0/4115061137 wait complete. 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.948+0000 7f31bd729640 1 Processor -- start 2024-03-14T12:14:22.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.948+0000 7f31bd729640 1 -- start start 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31bd729640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b809dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31bd729640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b80af3d0 0x7f31b815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31bd729640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b81566e0 0x7f31b815b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31bd729640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80d04a0 con 0x7f31b806b720 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31bd729640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80b6200 con 0x7f31b80af3d0 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31bd729640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80d0620 con 0x7f31b81566e0 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b809dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b809dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39338/0 (socket says 172.21.15.82:39338) 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 -- 172.21.15.82:0/2165369614 learned_addr learned my addr 172.21.15.82:0/2165369614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b67fc640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b80af3d0 0x7f31b815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b77fe640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b81566e0 0x7f31b815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 -- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b81566e0 msgr2=0x7f31b815b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b81566e0 0x7f31b815b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 -- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b80af3d0 msgr2=0x7f31b815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b80af3d0 0x7f31b815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b6ffd640 1 -- 172.21.15.82:0/2165369614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b81e6f40 con 0x7f31b806b720 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.949+0000 7f31b67fc640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b80af3d0 0x7f31b815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.950+0000 7f31b6ffd640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b809dcf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31a8007e40 tx=0x7f31a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.950+0000 7f319f7fe640 1 -- 172.21.15.82:0/2165369614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8014620 con 0x7f31b806b720 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.950+0000 7f319f7fe640 1 -- 172.21.15.82:0/2165369614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8019070 con 0x7f31b806b720 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.950+0000 7f319f7fe640 1 -- 172.21.15.82:0/2165369614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8014a00 con 0x7f31b806b720 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.950+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b81e7220 con 0x7f31b806b720 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.950+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b81e77e0 con 0x7f31b806b720 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.951+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b809e2f0 con 0x7f31b806b720 2024-03-14T12:14:22.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.951+0000 7f319f7fe640 1 -- 172.21.15.82:0/2165369614 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f31a8014ba0 con 0x7f31b806b720 2024-03-14T12:14:22.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.952+0000 7f319f7fe640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f31a406bbd0 0x7f31a406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:22.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.952+0000 7f319f7fe640 1 -- 172.21.15.82:0/2165369614 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31a808b280 con 0x7f31b806b720 2024-03-14T12:14:22.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.952+0000 7f31b67fc640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f31a406bbd0 0x7f31a406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:22.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.952+0000 7f31b67fc640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f31a406bbd0 0x7f31a406e090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f31b81f75b0 tx=0x7f31ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:22.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:22.956+0000 7f319f7fe640 1 -- 172.21.15.82:0/2165369614 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f31b809e2f0 con 0x7f31b806b720 2024-03-14T12:14:23.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.125+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31b804f050 con 0x7f31b806b720 2024-03-14T12:14:23.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.126+0000 7f319f7fe640 1 -- 172.21.15.82:0/2165369614 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31a8059e00 con 0x7f31b806b720 2024-03-14T12:14:23.125 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f31a406bbd0 msgr2=0x7f31a406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f31a406bbd0 0x7f31a406e090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f31b81f75b0 tx=0x7f31ac009040 comp rx=0 tx=0).stop 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 msgr2=0x7f31b809dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b809dcf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31a8007e40 tx=0x7f31a800e4b0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 shutdown_connections 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f31a406bbd0 0x7f31a406e090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b81566e0 0x7f31b815b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.127+0000 7f31bd729640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b80af3d0 0x7f31b815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.128+0000 7f31bd729640 1 --2- 172.21.15.82:0/2165369614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b806b720 0x7f31b809dcf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.128+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 >> 172.21.15.82:0/2165369614 conn(0x7f31b80b4fd0 msgr2=0x7f31b81532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:23.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.128+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 shutdown_connections 2024-03-14T12:14:23.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.128+0000 7f31bd729640 1 -- 172.21.15.82:0/2165369614 wait complete. 2024-03-14T12:14:23.138 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:23.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.241+0000 7fade8722640 1 Processor -- start 2024-03-14T12:14:23.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.241+0000 7fade8722640 1 -- start start 2024-03-14T12:14:23.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade8722640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fade015dd70 0x7fade006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade8722640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fade015e150 0x7fade00b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade8722640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade00d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade8722640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade006ea20 con 0x7fade015dd70 2024-03-14T12:14:23.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade8722640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade006f880 con 0x7fade015e150 2024-03-14T12:14:23.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade8722640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade0070250 con 0x7fade00b6bd0 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fade015dd70 0x7fade006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fade015dd70 0x7fade006fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39354/0 (socket says 172.21.15.82:39354) 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 -- 172.21.15.82:0/4291842942 learned_addr learned my addr 172.21.15.82:0/4291842942 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade6cbf640 1 --2- 172.21.15.82:0/4291842942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade00d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade5cbd640 1 --2- 172.21.15.82:0/4291842942 >> v2:172.21.15.164:3300/0 conn(0x7fade015e150 0x7fade00b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 -- 172.21.15.82:0/4291842942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 msgr2=0x7fade00d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 --2- 172.21.15.82:0/4291842942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade00d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 -- 172.21.15.82:0/4291842942 >> v2:172.21.15.164:3300/0 conn(0x7fade015e150 msgr2=0x7fade00b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 --2- 172.21.15.82:0/4291842942 >> v2:172.21.15.164:3300/0 conn(0x7fade015e150 0x7fade00b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.242+0000 7fade64be640 1 -- 172.21.15.82:0/4291842942 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade006ec70 con 0x7fade015dd70 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.243+0000 7fade64be640 1 --2- 172.21.15.82:0/4291842942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015dd70 0x7fade006fd10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fade006e420 tx=0x7faddc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f021a4eaa1279b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.243+0000 7fade54bc640 1 -- 172.21.15.82:0/4291842942 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc035020 con 0x7fade015dd70 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.243+0000 7fade54bc640 1 -- 172.21.15.82:0/4291842942 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc031870 con 0x7fade015dd70 2024-03-14T12:14:23.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.243+0000 7fade54bc640 1 -- 172.21.15.82:0/4291842942 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc031c40 con 0x7fade015dd70 2024-03-14T12:14:23.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 -- 172.21.15.82:0/4291842942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015dd70 msgr2=0x7fade006fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 --2- 172.21.15.82:0/4291842942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015dd70 0x7fade006fd10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fade006e420 tx=0x7faddc02b910 comp rx=0 tx=0).stop 2024-03-14T12:14:23.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 -- 172.21.15.82:0/4291842942 shutdown_connections 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 --2- 172.21.15.82:0/4291842942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade00d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 --2- 172.21.15.82:0/4291842942 >> v2:172.21.15.164:3300/0 conn(0x7fade015e150 0x7fade00b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 --2- 172.21.15.82:0/4291842942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015dd70 0x7fade006fd10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 -- 172.21.15.82:0/4291842942 >> 172.21.15.82:0/4291842942 conn(0x7fade0072ec0 msgr2=0x7fade009a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 -- 172.21.15.82:0/4291842942 shutdown_connections 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.244+0000 7fade8722640 1 -- 172.21.15.82:0/4291842942 wait complete. 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.245+0000 7fade8722640 1 Processor -- start 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.245+0000 7fade8722640 1 -- start start 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade8722640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade8722640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade015dd70 0x7fade00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade8722640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015e150 0x7fade00cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade8722640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade006f510 con 0x7fade015e150 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade8722640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade006f230 con 0x7fade015dd70 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade8722640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00d2040 con 0x7fade00b6bd0 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015e150 0x7fade00cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015e150 0x7fade00cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39362/0 (socket says 172.21.15.82:39362) 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 -- 172.21.15.82:0/38558301 learned_addr learned my addr 172.21.15.82:0/38558301 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade64be640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade5cbd640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade015dd70 0x7fade00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 -- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 msgr2=0x7fade014f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade014f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 -- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade015dd70 msgr2=0x7fade00b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade015dd70 0x7fade00b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 -- 172.21.15.82:0/38558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00ce910 con 0x7fade015e150 2024-03-14T12:14:23.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade5cbd640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade015dd70 0x7fade00b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade6cbf640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015e150 0x7fade00cf9b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fadd8007e60 tx=0x7fadd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fadd6ffd640 1 -- 172.21.15.82:0/38558301 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8014620 con 0x7fade015e150 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fadd6ffd640 1 -- 172.21.15.82:0/38558301 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd8019070 con 0x7fade015e150 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade00cebf0 con 0x7fade015e150 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.246+0000 7fadd6ffd640 1 -- 172.21.15.82:0/38558301 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8014a20 con 0x7fade015e150 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.247+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade00cf1b0 con 0x7fade015e150 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.248+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade004f050 con 0x7fade015e150 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.248+0000 7fadd6ffd640 1 -- 172.21.15.82:0/38558301 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fadd8014bc0 con 0x7fade015e150 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.248+0000 7fadd6ffd640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fadc406bbd0 0x7fadc406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.249+0000 7fade64be640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fadc406bbd0 0x7fadc406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.249+0000 7fadd6ffd640 1 -- 172.21.15.82:0/38558301 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadd808b230 con 0x7fade015e150 2024-03-14T12:14:23.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.249+0000 7fade64be640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fadc406bbd0 0x7fadc406e090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faddc036000 tx=0x7faddc03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:23.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.253+0000 7fadd6ffd640 1 -- 172.21.15.82:0/38558301 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fadd8059d20 con 0x7fade015e150 2024-03-14T12:14:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.423+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fade00afc90 con 0x7fade015e150 2024-03-14T12:14:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.424+0000 7fadd6ffd640 1 -- 172.21.15.82:0/38558301 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadd8059620 con 0x7fade015e150 2024-03-14T12:14:23.423 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fadc406bbd0 msgr2=0x7fadc406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fadc406bbd0 0x7fadc406e090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faddc036000 tx=0x7faddc03c040 comp rx=0 tx=0).stop 2024-03-14T12:14:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015e150 msgr2=0x7fade00cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015e150 0x7fade00cf9b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fadd8007e60 tx=0x7fadd800e4b0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 shutdown_connections 2024-03-14T12:14:23.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fade015e150 0x7fade00cf9b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fade015dd70 0x7fade00b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fadc406bbd0 0x7fadc406e090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 --2- 172.21.15.82:0/38558301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fade00b6bd0 0x7fade014f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 >> 172.21.15.82:0/38558301 conn(0x7fade0072ec0 msgr2=0x7fade00b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:23.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 shutdown_connections 2024-03-14T12:14:23.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.426+0000 7fade8722640 1 -- 172.21.15.82:0/38558301 wait complete. 2024-03-14T12:14:23.436 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:14:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.539+0000 7f519dfc4640 1 Processor -- start 2024-03-14T12:14:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.539+0000 7f519dfc4640 1 -- start start 2024-03-14T12:14:23.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f519dfc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980d0610 0x7f519806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f519dfc4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51980b4130 0x7f519806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f519dfc4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51980cd430 con 0x7f51980cfb20 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f519dfc4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980cd9f0 con 0x7f51980b4130 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f519dfc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980cd710 con 0x7f51980d0610 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f51977fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f51980cfb20 0x7f51980cff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:51240/0 (socket says 172.21.15.82:51240) 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980d0610 0x7f519806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f51977fe640 1 -- 172.21.15.82:0/1537495197 learned_addr learned my addr 172.21.15.82:0/1537495197 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.540+0000 7f5197fff640 1 --2- 172.21.15.82:0/1537495197 >> v2:172.21.15.164:3300/0 conn(0x7f51980b4130 0x7f519806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.541+0000 7f5196ffd640 1 -- 172.21.15.82:0/1537495197 >> v2:172.21.15.164:3300/0 conn(0x7f51980b4130 msgr2=0x7f519806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.541+0000 7f5196ffd640 1 --2- 172.21.15.82:0/1537495197 >> v2:172.21.15.164:3300/0 conn(0x7f51980b4130 0x7f519806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.541+0000 7f5196ffd640 1 -- 172.21.15.82:0/1537495197 >> v1:172.21.15.164:6789/0 conn(0x7f51980cfb20 legacy=0x7f51980cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.541+0000 7f5196ffd640 1 -- 172.21.15.82:0/1537495197 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519806eb60 con 0x7f51980d0610 2024-03-14T12:14:23.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.541+0000 7f5197fff640 1 --2- 172.21.15.82:0/1537495197 >> v2:172.21.15.164:3300/0 conn(0x7f51980b4130 0x7f519806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:23.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.541+0000 7f5196ffd640 1 --2- 172.21.15.82:0/1537495197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980d0610 0x7f519806f360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f518c0097a0 tx=0x7f518c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249ade18d46b656d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:23.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f51967fc640 1 -- 172.21.15.82:0/1537495197 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c035020 con 0x7f51980d0610 2024-03-14T12:14:23.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f51967fc640 1 -- 172.21.15.82:0/1537495197 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c031830 con 0x7f51980d0610 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f51967fc640 1 -- 172.21.15.82:0/1537495197 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c031c00 con 0x7f51980d0610 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 -- 172.21.15.82:0/1537495197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980d0610 msgr2=0x7f519806f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1537495197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980d0610 0x7f519806f360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f518c0097a0 tx=0x7f518c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 -- 172.21.15.82:0/1537495197 shutdown_connections 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1537495197 >> v2:172.21.15.164:3300/0 conn(0x7f51980b4130 0x7f519806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1537495197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980d0610 0x7f519806f360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 -- 172.21.15.82:0/1537495197 >> 172.21.15.82:0/1537495197 conn(0x7f51980b3610 msgr2=0x7f51980b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 -- 172.21.15.82:0/1537495197 shutdown_connections 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.542+0000 7f519dfc4640 1 -- 172.21.15.82:0/1537495197 wait complete. 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.543+0000 7f519dfc4640 1 Processor -- start 2024-03-14T12:14:23.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.543+0000 7f519dfc4640 1 -- start start 2024-03-14T12:14:23.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f519dfc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980b4130 0x7f51980b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f519dfc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51980cfb20 0x7f51981f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f519dfc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51980d0610 0x7f51980c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f519dfc4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519815c070 con 0x7f51980cfb20 2024-03-14T12:14:23.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f519dfc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980cd0c0 con 0x7f51980d0610 2024-03-14T12:14:23.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f519dfc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519815c1f0 con 0x7f51980b4130 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51980cfb20 0x7f51981f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51980cfb20 0x7f51981f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39366/0 (socket says 172.21.15.82:39366) 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f5196ffd640 1 -- 172.21.15.82:0/1985293827 learned_addr learned my addr 172.21.15.82:0/1985293827 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f5197fff640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51980d0610 0x7f51980c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.544+0000 7f51977fe640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980b4130 0x7f51980b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f5196ffd640 1 -- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980b4130 msgr2=0x7f51980b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f5196ffd640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980b4130 0x7f51980b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f5196ffd640 1 -- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51980d0610 msgr2=0x7f51980c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f5196ffd640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51980d0610 0x7f51980c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f5196ffd640 1 -- 172.21.15.82:0/1985293827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980b63e0 con 0x7f51980cfb20 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f5196ffd640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51980cfb20 0x7f51981f4e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f518c02bee0 tx=0x7f518c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f517ffff640 1 -- 172.21.15.82:0/1985293827 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c041070 con 0x7f51980cfb20 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f517ffff640 1 -- 172.21.15.82:0/1985293827 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c03f070 con 0x7f51980cfb20 2024-03-14T12:14:23.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51980b66c0 con 0x7f51980cfb20 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f517ffff640 1 -- 172.21.15.82:0/1985293827 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c035040 con 0x7f51980cfb20 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.545+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51980b6c80 con 0x7f51980cfb20 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.546+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51980b6ec0 con 0x7f51980cfb20 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.546+0000 7f517ffff640 1 -- 172.21.15.82:0/1985293827 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f518c038020 con 0x7f51980cfb20 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.547+0000 7f517ffff640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f518406bc00 0x7f518406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.547+0000 7f517ffff640 1 -- 172.21.15.82:0/1985293827 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f518c0ac1c0 con 0x7f51980cfb20 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.547+0000 7f51977fe640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f518406bc00 0x7f518406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:23.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.547+0000 7f51977fe640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f518406bc00 0x7f518406e0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f519815b6c0 tx=0x7f5188007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:23.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.551+0000 7f517ffff640 1 -- 172.21.15.82:0/1985293827 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f518c031800 con 0x7f51980cfb20 2024-03-14T12:14:23.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.739+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f519804f050 con 0x7f51980cfb20 2024-03-14T12:14:23.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.740+0000 7f517ffff640 1 -- 172.21.15.82:0/1985293827 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f518c07ad40 con 0x7f51980cfb20 2024-03-14T12:14:23.739 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:14:23.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f518406bc00 msgr2=0x7f518406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f518406bc00 0x7f518406e0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f519815b6c0 tx=0x7f5188007e80 comp rx=0 tx=0).stop 2024-03-14T12:14:23.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51980cfb20 msgr2=0x7f51981f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51980cfb20 0x7f51981f4e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f518c02bee0 tx=0x7f518c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 shutdown_connections 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51980d0610 0x7f51980c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51980cfb20 0x7f51981f4e80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f518406bc00 0x7f518406e0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 --2- 172.21.15.82:0/1985293827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51980b4130 0x7f51980b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 >> 172.21.15.82:0/1985293827 conn(0x7f51980b3610 msgr2=0x7f51980cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 shutdown_connections 2024-03-14T12:14:23.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:23.742+0000 7f519dfc4640 1 -- 172.21.15.82:0/1985293827 wait complete. 2024-03-14T12:14:23.752 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T12:14:23.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:23.752 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-14T12:14:23.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:14:23.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:23.780 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T12:14:23.840 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:23.841 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:23.859 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:23.860 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:23.918 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:23.918 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:23.976 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:24.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.077+0000 7f784a412640 1 Processor -- start 2024-03-14T12:14:24.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.077+0000 7f784a412640 1 -- start start 2024-03-14T12:14:24.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f784a412640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f78440d0760 0x7f78440d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f784a412640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78440cf370 0x7f78440d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f784a412640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f784a412640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78440d1080 con 0x7f78440cf370 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f784a412640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78440cf8c0 con 0x7f78440d0760 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f784a412640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78440cfba0 con 0x7f78440d1da0 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f78437fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78440cf370 0x7f78440d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f78437fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78440cf370 0x7f78440d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39380/0 (socket says 172.21.15.82:39380) 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.078+0000 7f78437fe640 1 -- 172.21.15.82:0/890217353 learned_addr learned my addr 172.21.15.82:0/890217353 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f78489af640 1 --2- 172.21.15.82:0/890217353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f7843fff640 1 --2- 172.21.15.82:0/890217353 >> v2:172.21.15.164:3300/0 conn(0x7f78440d0760 0x7f78440d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f78437fe640 1 -- 172.21.15.82:0/890217353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 msgr2=0x7f78440d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f78437fe640 1 --2- 172.21.15.82:0/890217353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f78437fe640 1 -- 172.21.15.82:0/890217353 >> v2:172.21.15.164:3300/0 conn(0x7f78440d0760 msgr2=0x7f78440d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f78437fe640 1 --2- 172.21.15.82:0/890217353 >> v2:172.21.15.164:3300/0 conn(0x7f78440d0760 0x7f78440d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f78437fe640 1 -- 172.21.15.82:0/890217353 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78440b6c40 con 0x7f78440cf370 2024-03-14T12:14:24.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f78437fe640 1 --2- 172.21.15.82:0/890217353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440cf370 0x7f78440d4660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f784408ca90 tx=0x7f783802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6aca78847a66197 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.079+0000 7f7842ffd640 1 -- 172.21.15.82:0/890217353 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838034070 con 0x7f78440cf370 2024-03-14T12:14:24.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f7842ffd640 1 -- 172.21.15.82:0/890217353 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7838030860 con 0x7f78440cf370 2024-03-14T12:14:24.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f7842ffd640 1 -- 172.21.15.82:0/890217353 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838030c30 con 0x7f78440cf370 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 -- 172.21.15.82:0/890217353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440cf370 msgr2=0x7f78440d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 --2- 172.21.15.82:0/890217353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440cf370 0x7f78440d4660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f784408ca90 tx=0x7f783802b9a0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 -- 172.21.15.82:0/890217353 shutdown_connections 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 --2- 172.21.15.82:0/890217353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 --2- 172.21.15.82:0/890217353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440cf370 0x7f78440d4660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 --2- 172.21.15.82:0/890217353 >> v2:172.21.15.164:3300/0 conn(0x7f78440d0760 0x7f78440d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 -- 172.21.15.82:0/890217353 >> 172.21.15.82:0/890217353 conn(0x7f784409d030 msgr2=0x7f78440d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 -- 172.21.15.82:0/890217353 shutdown_connections 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.080+0000 7f784a412640 1 -- 172.21.15.82:0/890217353 wait complete. 2024-03-14T12:14:24.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.081+0000 7f784a412640 1 Processor -- start 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.081+0000 7f784a412640 1 -- start start 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f784a412640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78440cf370 0x7f7844154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f784a412640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440d0760 0x7f78440cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f784a412640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f784a412640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78440d4ba0 con 0x7f78440d0760 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f784a412640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78440b7200 con 0x7f78440cf370 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f784a412640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78441617a0 con 0x7f78440d1da0 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440d0760 0x7f78440cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440d0760 0x7f78440cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39382/0 (socket says 172.21.15.82:39382) 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 -- 172.21.15.82:0/78870056 learned_addr learned my addr 172.21.15.82:0/78870056 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f7843fff640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78440cf370 0x7f7844154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78489af640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 -- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 msgr2=0x7f78440cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 -- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78440cf370 msgr2=0x7f7844154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78440cf370 0x7f7844154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.082+0000 7f78437fe640 1 -- 172.21.15.82:0/78870056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78440cb870 con 0x7f78440d0760 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.083+0000 7f78489af640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.083+0000 7f78437fe640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440d0760 0x7f78440cc910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7838000c00 tx=0x7f7838004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.083+0000 7f7840ff9640 1 -- 172.21.15.82:0/78870056 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838048070 con 0x7f78440d0760 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.083+0000 7f7840ff9640 1 -- 172.21.15.82:0/78870056 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7838030660 con 0x7f78440d0760 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.083+0000 7f7840ff9640 1 -- 172.21.15.82:0/78870056 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838034040 con 0x7f78440d0760 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.083+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78440cbb50 con 0x7f78440d0760 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.083+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78440cc110 con 0x7f78440d0760 2024-03-14T12:14:24.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.084+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784404f050 con 0x7f78440d0760 2024-03-14T12:14:24.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.084+0000 7f7840ff9640 1 -- 172.21.15.82:0/78870056 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f783804b070 con 0x7f78440d0760 2024-03-14T12:14:24.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.084+0000 7f7840ff9640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f783006bbd0 0x7f783006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.085+0000 7f7840ff9640 1 -- 172.21.15.82:0/78870056 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78380acba0 con 0x7f78440d0760 2024-03-14T12:14:24.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.085+0000 7f7843fff640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f783006bbd0 0x7f783006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.085+0000 7f7843fff640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f783006bbd0 0x7f783006e090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7834006fd0 tx=0x7f7834008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.089+0000 7f7840ff9640 1 -- 172.21.15.82:0/78870056 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f783807b7b0 con 0x7f78440d0760 2024-03-14T12:14:24.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.263+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78440d0540 con 0x7f78440d0760 2024-03-14T12:14:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.264+0000 7f7840ff9640 1 -- 172.21.15.82:0/78870056 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f783807b0b0 con 0x7f78440d0760 2024-03-14T12:14:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:24.263 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f783006bbd0 msgr2=0x7f783006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f783006bbd0 0x7f783006e090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7834006fd0 tx=0x7f7834008040 comp rx=0 tx=0).stop 2024-03-14T12:14:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440d0760 msgr2=0x7f78440cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440d0760 0x7f78440cc910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7838000c00 tx=0x7f7838004730 comp rx=0 tx=0).stop 2024-03-14T12:14:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 shutdown_connections 2024-03-14T12:14:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f783006bbd0 0x7f783006e090 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78440d1da0 0x7f78440cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78440d0760 0x7f78440cc910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 --2- 172.21.15.82:0/78870056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78440cf370 0x7f7844154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 >> 172.21.15.82:0/78870056 conn(0x7f784409d030 msgr2=0x7f7844160a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 shutdown_connections 2024-03-14T12:14:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.266+0000 7f784a412640 1 -- 172.21.15.82:0/78870056 wait complete. 2024-03-14T12:14:24.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:24.276 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:24.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.380+0000 7f26b8d1b640 1 Processor -- start 2024-03-14T12:14:24.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.380+0000 7f26b8d1b640 1 -- start start 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b8d1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 0x7f26b406f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b8d1b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b415c0c0 con 0x7f26b415bce0 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b8d1b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40afea0 con 0x7f26b40b5e60 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b8d1b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40afc50 con 0x7f26b40b1200 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f26b415bce0 0x7f26b40b5750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50930/0 (socket says 172.21.15.82:50930) 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b259c640 1 -- 172.21.15.82:0/2882495202 learned_addr learned my addr 172.21.15.82:0/2882495202 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b2d9d640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 0x7f26b406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711778665 0 0) 0x7f26b415c0c0 con 0x7f26b415bce0 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26a0003520 con 0x7f26b415bce0 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953383647 0 0) 0x7f26b40afea0 con 0x7f26b40b5e60 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b415c0c0 con 0x7f26b40b5e60 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3561716659 0 0) 0x7f26a0003520 con 0x7f26b415bce0 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 msgr2=0x7f26b406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 0x7f26b406f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 >> v1:172.21.15.164:6789/0 conn(0x7f26b40b5e60 legacy=0x7f26b406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40b0180 con 0x7f26b415bce0 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f26a4001670 con 0x7f26b415bce0 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 >> v1:172.21.15.82:6789/0 conn(0x7f26b415bce0 legacy=0x7f26b40b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26a0003b70 0x7f26a0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 0x7f26a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.381+0000 7f26b1d9b640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26a0004d40 0x7f26a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40afc50 con 0x7f26a0004d40 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a000ba70 con 0x7f26a0003b70 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a000bc70 con 0x7f26b40b1200 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b2d9d640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26a0004d40 0x7f26a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b259c640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26a0003b70 0x7f26a0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26a9d9b640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 0x7f26a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b2d9d640 1 -- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 msgr2=0x7f26a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b2d9d640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26b40b1200 0x7f26a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b2d9d640 1 -- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26a0003b70 msgr2=0x7f26a0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b2d9d640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26a0003b70 0x7f26a0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.382+0000 7f26b2d9d640 1 -- 172.21.15.82:0/2882495202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0003520 con 0x7f26a0004d40 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b2d9d640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26a0004d40 0x7f26a00051c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009fc0 tx=0x7f26a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2797f2d3332ed5e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac039070 con 0x7f26a0004d40 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac02cc20 con 0x7f26a0004d40 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b1d9b640 1 -- 172.21.15.82:0/2882495202 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac009c10 con 0x7f26a0004d40 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b8d1b640 1 -- 172.21.15.82:0/2882495202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26a0004d40 msgr2=0x7f26a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b8d1b640 1 --2- 172.21.15.82:0/2882495202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26a0004d40 0x7f26a00051c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009fc0 tx=0x7f26a0005830 comp rx=0 tx=0).stop 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b2d9d640 1 -- 172.21.15.82:0/2882495202 reap_dead start 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b8d1b640 1 -- 172.21.15.82:0/2882495202 shutdown_connections 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b8d1b640 1 -- 172.21.15.82:0/2882495202 >> 172.21.15.82:0/2882495202 conn(0x7f26b4072ec0 msgr2=0x7f26b409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b8d1b640 1 -- 172.21.15.82:0/2882495202 shutdown_connections 2024-03-14T12:14:24.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.383+0000 7f26b8d1b640 1 -- 172.21.15.82:0/2882495202 wait complete. 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.385+0000 7f26b8d1b640 1 Processor -- start 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.385+0000 7f26b8d1b640 1 -- start start 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.385+0000 7f26b8d1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26a0004d40 0x7f26b414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.385+0000 7f26b8d1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b409cd80 0x7f26b409d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.385+0000 7f26b8d1b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b409bb70 0x7f26b409bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.385+0000 7f26b8d1b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40b0a20 con 0x7f26b409cd80 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26b8d1b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40b0740 con 0x7f26b409bb70 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26b8d1b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40b0cb0 con 0x7f26a0004d40 2024-03-14T12:14:24.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b409cd80 0x7f26b409d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b409cd80 0x7f26b409d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39410/0 (socket says 172.21.15.82:39410) 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 -- 172.21.15.82:0/1215543229 learned_addr learned my addr 172.21.15.82:0/1215543229 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26b2d9d640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b409bb70 0x7f26b409bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 -- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26a0004d40 msgr2=0x7f26b414a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26a0004d40 0x7f26b414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 -- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b409bb70 msgr2=0x7f26b409bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b409bb70 0x7f26b409bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 -- 172.21.15.82:0/1215543229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b414ab50 con 0x7f26b409cd80 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26b2d9d640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b409bb70 0x7f26b409bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26a9d9b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b409cd80 0x7f26b409d200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f269c00f840 tx=0x7f269c00fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26ab7fe640 1 -- 172.21.15.82:0/1215543229 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0052e0 con 0x7f26b409cd80 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26ab7fe640 1 -- 172.21.15.82:0/1215543229 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c018070 con 0x7f26b409cd80 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b406cca0 con 0x7f26b409cd80 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26ab7fe640 1 -- 172.21.15.82:0/1215543229 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00d910 con 0x7f26b409cd80 2024-03-14T12:14:24.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.386+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b406d260 con 0x7f26b409cd80 2024-03-14T12:14:24.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.387+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b404f050 con 0x7f26b409cd80 2024-03-14T12:14:24.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.388+0000 7f26ab7fe640 1 -- 172.21.15.82:0/1215543229 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f269c00dab0 con 0x7f26b409cd80 2024-03-14T12:14:24.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.388+0000 7f26ab7fe640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f269406bbd0 0x7f269406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.388+0000 7f26ab7fe640 1 -- 172.21.15.82:0/1215543229 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f269c08aa30 con 0x7f26b409cd80 2024-03-14T12:14:24.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.388+0000 7f26b259c640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f269406bbd0 0x7f269406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.389+0000 7f26b259c640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f269406bbd0 0x7f269406e090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26a4001bc0 tx=0x7f26a4001ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.392+0000 7f26ab7fe640 1 -- 172.21.15.82:0/1215543229 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f26b414ab50 con 0x7f26b409cd80 2024-03-14T12:14:24.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.562+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26b40d29e0 con 0x7f26b409cd80 2024-03-14T12:14:24.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.563+0000 7f26ab7fe640 1 -- 172.21.15.82:0/1215543229 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f269c059520 con 0x7f26b409cd80 2024-03-14T12:14:24.562 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f269406bbd0 msgr2=0x7f269406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f269406bbd0 0x7f269406e090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26a4001bc0 tx=0x7f26a4001ab0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b409cd80 msgr2=0x7f26b409d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b409cd80 0x7f26b409d200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f269c00f840 tx=0x7f269c00fd00 comp rx=0 tx=0).stop 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 shutdown_connections 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26b409bb70 0x7f26b409bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b409cd80 0x7f26b409d200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f269406bbd0 0x7f269406e090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 --2- 172.21.15.82:0/1215543229 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f26a0004d40 0x7f26b414a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 >> 172.21.15.82:0/1215543229 conn(0x7f26b4072ec0 msgr2=0x7f26b415e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 shutdown_connections 2024-03-14T12:14:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.565+0000 7f26b8d1b640 1 -- 172.21.15.82:0/1215543229 wait complete. 2024-03-14T12:14:24.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:24.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.677+0000 7f665f97e640 1 Processor -- start 2024-03-14T12:14:24.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.678+0000 7f665f97e640 1 -- start start 2024-03-14T12:14:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.678+0000 7f665f97e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6658143c60 0x7f66580b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.678+0000 7f665f97e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66580b41c0 con 0x7f6658140e70 2024-03-14T12:14:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.678+0000 7f665f97e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6658161c10 con 0x7f66581626f0 2024-03-14T12:14:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.678+0000 7f665f97e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6658161ef0 con 0x7f6658143c60 2024-03-14T12:14:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.678+0000 7f665d71a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6658140e70 0x7f6658143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50940/0 (socket says 172.21.15.82:50940) 2024-03-14T12:14:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f665d71a640 1 -- 172.21.15.82:0/2524126850 learned_addr learned my addr 172.21.15.82:0/2524126850 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f665cf19640 1 --2- 172.21.15.82:0/2524126850 >> v2:172.21.15.164:3300/0 conn(0x7f6658143c60 0x7f66580b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f664ffff640 1 -- 172.21.15.82:0/2524126850 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864259485 0 0) 0x7f66580b41c0 con 0x7f6658140e70 2024-03-14T12:14:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f664ffff640 1 -- 172.21.15.82:0/2524126850 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6644003520 con 0x7f6658140e70 2024-03-14T12:14:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f664ffff640 1 -- 172.21.15.82:0/2524126850 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1440179631 0 0) 0x7f6644003520 con 0x7f6658140e70 2024-03-14T12:14:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f665cf19640 1 -- 172.21.15.82:0/2524126850 >> v1:172.21.15.164:6789/0 conn(0x7f66581626f0 legacy=0x7f66580af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f665cf19640 1 -- 172.21.15.82:0/2524126850 >> v1:172.21.15.82:6789/0 conn(0x7f6658140e70 legacy=0x7f6658143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.679+0000 7f665cf19640 1 -- 172.21.15.82:0/2524126850 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66581621d0 con 0x7f6658143c60 2024-03-14T12:14:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.680+0000 7f665cf19640 1 --2- 172.21.15.82:0/2524126850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 0x7f66580b3b30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f66540097a0 tx=0x7f665402b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdfd11bf6b3ff685 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.680+0000 7f664ffff640 1 -- 172.21.15.82:0/2524126850 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654037070 con 0x7f6658143c60 2024-03-14T12:14:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.680+0000 7f664ffff640 1 -- 172.21.15.82:0/2524126850 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654030510 con 0x7f6658143c60 2024-03-14T12:14:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.680+0000 7f664ffff640 1 -- 172.21.15.82:0/2524126850 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648001670 con 0x7f6658143c60 2024-03-14T12:14:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.681+0000 7f665f97e640 1 -- 172.21.15.82:0/2524126850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 msgr2=0x7f66580b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.681+0000 7f665f97e640 1 --2- 172.21.15.82:0/2524126850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 0x7f66580b3b30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f66540097a0 tx=0x7f665402b680 comp rx=0 tx=0).stop 2024-03-14T12:14:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.681+0000 7f665f97e640 1 -- 172.21.15.82:0/2524126850 shutdown_connections 2024-03-14T12:14:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.681+0000 7f665f97e640 1 --2- 172.21.15.82:0/2524126850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 0x7f66580b3b30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.681+0000 7f665f97e640 1 -- 172.21.15.82:0/2524126850 >> 172.21.15.82:0/2524126850 conn(0x7f66580b2830 msgr2=0x7f66580b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.681+0000 7f665f97e640 1 -- 172.21.15.82:0/2524126850 shutdown_connections 2024-03-14T12:14:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.681+0000 7f665f97e640 1 -- 172.21.15.82:0/2524126850 wait complete. 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.682+0000 7f665f97e640 1 Processor -- start 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.682+0000 7f665f97e640 1 -- start start 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.682+0000 7f665f97e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6658140e70 0x7f665813fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.682+0000 7f665f97e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 0x7f66580b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665f97e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66581626f0 0x7f66580b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665f97e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665815f300 con 0x7f6658140e70 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665f97e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665815f180 con 0x7f6658143c60 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665f97e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66580aff80 con 0x7f66581626f0 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6658140e70 0x7f665813fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6658140e70 0x7f665813fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39418/0 (socket says 172.21.15.82:39418) 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 -- 172.21.15.82:0/2671842605 learned_addr learned my addr 172.21.15.82:0/2671842605 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665cf19640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 0x7f66580b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665df1b640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66581626f0 0x7f66580b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 -- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66581626f0 msgr2=0x7f66580b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66581626f0 0x7f66580b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 -- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 msgr2=0x7f66580b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 0x7f66580b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 -- 172.21.15.82:0/2671842605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6658146a60 con 0x7f6658140e70 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665d71a640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6658140e70 0x7f665813fcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f664800c780 tx=0x7f664800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f664dffb640 1 -- 172.21.15.82:0/2671842605 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648018070 con 0x7f6658140e70 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f664dffb640 1 -- 172.21.15.82:0/2671842605 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66480072d0 con 0x7f6658140e70 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6658144430 con 0x7f6658140e70 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f664dffb640 1 -- 172.21.15.82:0/2671842605 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480047e0 con 0x7f6658140e70 2024-03-14T12:14:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.683+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66581449f0 con 0x7f6658140e70 2024-03-14T12:14:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.685+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665804f050 con 0x7f6658140e70 2024-03-14T12:14:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.685+0000 7f664dffb640 1 -- 172.21.15.82:0/2671842605 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f664801b070 con 0x7f6658140e70 2024-03-14T12:14:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.685+0000 7f664dffb640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f663c06bbd0 0x7f663c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.685+0000 7f664dffb640 1 -- 172.21.15.82:0/2671842605 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6648089ec0 con 0x7f6658140e70 2024-03-14T12:14:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.685+0000 7f665cf19640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f663c06bbd0 0x7f663c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.686+0000 7f665cf19640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f663c06bbd0 0x7f663c06e090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6654034000 tx=0x7f6654008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.689+0000 7f664dffb640 1 -- 172.21.15.82:0/2671842605 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f66480589b0 con 0x7f6658140e70 2024-03-14T12:14:24.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.862+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66580b3700 con 0x7f6658140e70 2024-03-14T12:14:24.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.862+0000 7f664dffb640 1 -- 172.21.15.82:0/2671842605 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66480582b0 con 0x7f6658140e70 2024-03-14T12:14:24.861 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f663c06bbd0 msgr2=0x7f663c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f663c06bbd0 0x7f663c06e090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6654034000 tx=0x7f6654008030 comp rx=0 tx=0).stop 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6658140e70 msgr2=0x7f665813fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6658140e70 0x7f665813fcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f664800c780 tx=0x7f664800cc40 comp rx=0 tx=0).stop 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 shutdown_connections 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f663c06bbd0 0x7f663c06e090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66581626f0 0x7f66580b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6658143c60 0x7f66580b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 --2- 172.21.15.82:0/2671842605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6658140e70 0x7f665813fcd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 >> 172.21.15.82:0/2671842605 conn(0x7f66580b2830 msgr2=0x7f66580b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:24.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 shutdown_connections 2024-03-14T12:14:24.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.864+0000 7f665f97e640 1 -- 172.21.15.82:0/2671842605 wait complete. 2024-03-14T12:14:24.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:24.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.977+0000 7fce7d7c4640 1 Processor -- start 2024-03-14T12:14:24.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.977+0000 7fce7d7c4640 1 -- start start 2024-03-14T12:14:24.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.978+0000 7fce7d7c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 0x7fce7815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.978+0000 7fce7d7c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fce780d01b0 0x7fce780d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.978+0000 7fce7d7c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780ce380 con 0x7fce7815bde0 2024-03-14T12:14:24.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.978+0000 7fce7d7c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce780cef90 con 0x7fce780d01b0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.978+0000 7fce7d7c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780b5a30 con 0x7fce780cd960 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.978+0000 7fce76ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fce7815bde0 0x7fce780cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50944/0 (socket says 172.21.15.82:50944) 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.978+0000 7fce76ffd640 1 -- 172.21.15.82:0/3092859843 learned_addr learned my addr 172.21.15.82:0/3092859843 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce767fc640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 0x7fce7815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce777fe640 1 --2- 172.21.15.82:0/3092859843 >> v2:172.21.15.164:3300/0 conn(0x7fce780d01b0 0x7fce780d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966664456 0 0) 0x7fce780ce380 con 0x7fce7815bde0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce60003520 con 0x7fce7815bde0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2807650008 0 0) 0x7fce60003520 con 0x7fce7815bde0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 msgr2=0x7fce7815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 0x7fce7815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 >> v2:172.21.15.164:3300/0 conn(0x7fce780d01b0 msgr2=0x7fce780d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 --2- 172.21.15.82:0/3092859843 >> v2:172.21.15.164:3300/0 conn(0x7fce780d01b0 0x7fce780d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780ce660 con 0x7fce7815bde0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fce680019f0 con 0x7fce7815bde0 2024-03-14T12:14:24.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 >> v1:172.21.15.82:6789/0 conn(0x7fce7815bde0 legacy=0x7fce780cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce600039b0 0x7fce60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 0x7fce60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce60004d40 0x7fce600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780cef90 con 0x7fce60004d40 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780b5a30 con 0x7fce600039b0 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6000b9c0 con 0x7fce780cd960 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce777fe640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce60004d40 0x7fce600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce76ffd640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce600039b0 0x7fce60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce767fc640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 0x7fce60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce777fe640 1 -- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 msgr2=0x7fce60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce777fe640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780cd960 0x7fce60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce777fe640 1 -- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce600039b0 msgr2=0x7fce60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce777fe640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce600039b0 0x7fce60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.979+0000 7fce777fe640 1 -- 172.21.15.82:0/3092859843 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60003520 con 0x7fce60004d40 2024-03-14T12:14:24.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.980+0000 7fce777fe640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce60004d40 0x7fce600051c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce70009ac0 tx=0x7fce60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a21004eb0b4495b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.980+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70039070 con 0x7fce60004d40 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.980+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7002c710 con 0x7fce60004d40 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.980+0000 7fce75ffb640 1 -- 172.21.15.82:0/3092859843 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70009690 con 0x7fce60004d40 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.981+0000 7fce7d7c4640 1 -- 172.21.15.82:0/3092859843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce60004d40 msgr2=0x7fce600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.981+0000 7fce7d7c4640 1 --2- 172.21.15.82:0/3092859843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce60004d40 0x7fce600051c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce70009ac0 tx=0x7fce60005830 comp rx=0 tx=0).stop 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.981+0000 7fce777fe640 1 -- 172.21.15.82:0/3092859843 reap_dead start 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.981+0000 7fce7d7c4640 1 -- 172.21.15.82:0/3092859843 shutdown_connections 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.981+0000 7fce7d7c4640 1 -- 172.21.15.82:0/3092859843 >> 172.21.15.82:0/3092859843 conn(0x7fce780d1bd0 msgr2=0x7fce780d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.981+0000 7fce7d7c4640 1 -- 172.21.15.82:0/3092859843 shutdown_connections 2024-03-14T12:14:24.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.981+0000 7fce7d7c4640 1 -- 172.21.15.82:0/3092859843 wait complete. 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.982+0000 7fce7d7c4640 1 Processor -- start 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.982+0000 7fce7d7c4640 1 -- start start 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce7d7c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7806fdd0 0x7fce780c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce7d7c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780701b0 0x7fce780c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce7d7c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7814fc40 0x7fce781500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce7d7c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780cf420 con 0x7fce7806fdd0 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce7d7c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780cec20 con 0x7fce7814fc40 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce7d7c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7815c1c0 con 0x7fce780701b0 2024-03-14T12:14:24.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7806fdd0 0x7fce780c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7806fdd0 0x7fce780c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39440/0 (socket says 172.21.15.82:39440) 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 -- 172.21.15.82:0/1423533071 learned_addr learned my addr 172.21.15.82:0/1423533071 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce777fe640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7814fc40 0x7fce781500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce767fc640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780701b0 0x7fce780c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 -- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780701b0 msgr2=0x7fce780c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780701b0 0x7fce780c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 -- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7814fc40 msgr2=0x7fce781500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7814fc40 0x7fce781500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 -- 172.21.15.82:0/1423533071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78150600 con 0x7fce7806fdd0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce777fe640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7814fc40 0x7fce781500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce76ffd640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7806fdd0 0x7fce780c64f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce6800a400 tx=0x7fce6800a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.983+0000 7fce5f7fe640 1 -- 172.21.15.82:0/1423533071 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68017020 con 0x7fce7806fdd0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.984+0000 7fce5f7fe640 1 -- 172.21.15.82:0/1423533071 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68013600 con 0x7fce7806fdd0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.984+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce780c70b0 con 0x7fce7806fdd0 2024-03-14T12:14:24.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.984+0000 7fce5f7fe640 1 -- 172.21.15.82:0/1423533071 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce680139f0 con 0x7fce7806fdd0 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.984+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780c7670 con 0x7fce7806fdd0 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.985+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7804f050 con 0x7fce7806fdd0 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.985+0000 7fce5f7fe640 1 -- 172.21.15.82:0/1423533071 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fce6801a020 con 0x7fce7806fdd0 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.985+0000 7fce5f7fe640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fce6406bc00 0x7fce6406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.985+0000 7fce5f7fe640 1 -- 172.21.15.82:0/1423533071 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce6808ba40 con 0x7fce7806fdd0 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.985+0000 7fce767fc640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fce6406bc00 0x7fce6406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.986+0000 7fce767fc640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fce6406bc00 0x7fce6406e0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce780b5f80 tx=0x7fce6c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:24.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:24.990+0000 7fce5f7fe640 1 -- 172.21.15.82:0/1423533071 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fce68021020 con 0x7fce7806fdd0 2024-03-14T12:14:25.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.162+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce780b7b50 con 0x7fce7806fdd0 2024-03-14T12:14:25.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.163+0000 7fce5f7fe640 1 -- 172.21.15.82:0/1423533071 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce6808be20 con 0x7fce7806fdd0 2024-03-14T12:14:25.162 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fce6406bc00 msgr2=0x7fce6406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fce6406bc00 0x7fce6406e0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce780b5f80 tx=0x7fce6c008040 comp rx=0 tx=0).stop 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7806fdd0 msgr2=0x7fce780c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7806fdd0 0x7fce780c64f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce6800a400 tx=0x7fce6800a8c0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 shutdown_connections 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce7814fc40 0x7fce781500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fce6406bc00 0x7fce6406e0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce780701b0 0x7fce780c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 --2- 172.21.15.82:0/1423533071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7806fdd0 0x7fce780c64f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 >> 172.21.15.82:0/1423533071 conn(0x7fce780d1bd0 msgr2=0x7fce780b69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:25.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 shutdown_connections 2024-03-14T12:14:25.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.165+0000 7fce7d7c4640 1 -- 172.21.15.82:0/1423533071 wait complete. 2024-03-14T12:14:25.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:14:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.276+0000 7f1e1d9df640 1 Processor -- start 2024-03-14T12:14:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.276+0000 7f1e1d9df640 1 -- start start 2024-03-14T12:14:25.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e1d9df640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e1d9df640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e180d0c40 0x7f1e180ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e1d9df640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1806ed00 con 0x7f1e180ce480 2024-03-14T12:14:25.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e1d9df640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e180cee20 con 0x7f1e180d0c40 2024-03-14T12:14:25.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e1d9df640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180cc0f0 con 0x7f1e1815e190 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e177fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e180d0c40 0x7f1e180ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e177fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e180d0c40 0x7f1e180ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39454/0 (socket says 172.21.15.82:39454) 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e177fe640 1 -- 172.21.15.82:0/1136883881 learned_addr learned my addr 172.21.15.82:0/1136883881 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e16ffd640 1 --2- 172.21.15.82:0/1136883881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e177fe640 1 -- 172.21.15.82:0/1136883881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 msgr2=0x7f1e180b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e177fe640 1 --2- 172.21.15.82:0/1136883881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e177fe640 1 -- 172.21.15.82:0/1136883881 >> v1:172.21.15.164:6789/0 conn(0x7f1e180ce480 legacy=0x7f1e180ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e177fe640 1 -- 172.21.15.82:0/1136883881 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1806efe0 con 0x7f1e180d0c40 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.277+0000 7f1e16ffd640 1 --2- 172.21.15.82:0/1136883881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.278+0000 7f1e177fe640 1 --2- 172.21.15.82:0/1136883881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e180ce8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009870 tx=0x7f1e1002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d2fd14b28043a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.278+0000 7f1e15ffb640 1 -- 172.21.15.82:0/1136883881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e100317c0 con 0x7f1e180d0c40 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.278+0000 7f1e15ffb640 1 -- 172.21.15.82:0/1136883881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10037070 con 0x7f1e180d0c40 2024-03-14T12:14:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.278+0000 7f1e15ffb640 1 -- 172.21.15.82:0/1136883881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10031b80 con 0x7f1e180d0c40 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 -- 172.21.15.82:0/1136883881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 msgr2=0x7f1e180ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/1136883881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e180ce8e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009870 tx=0x7f1e1002b860 comp rx=0 tx=0).stop 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 -- 172.21.15.82:0/1136883881 shutdown_connections 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/1136883881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e180ce8e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/1136883881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 -- 172.21.15.82:0/1136883881 >> 172.21.15.82:0/1136883881 conn(0x7f1e180b3610 msgr2=0x7f1e180b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 -- 172.21.15.82:0/1136883881 shutdown_connections 2024-03-14T12:14:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.279+0000 7f1e1d9df640 1 -- 172.21.15.82:0/1136883881 wait complete. 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.280+0000 7f1e1d9df640 1 Processor -- start 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.280+0000 7f1e1d9df640 1 -- start start 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e1d9df640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e180ce480 0x7f1e1815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e1d9df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e181575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e1d9df640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e1d9df640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180d10a0 con 0x7f1e180d0c40 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e1d9df640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180cc6b0 con 0x7f1e180ce480 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e1d9df640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180cd250 con 0x7f1e1815e190 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e181575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e181575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39462/0 (socket says 172.21.15.82:39462) 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 -- 172.21.15.82:0/3744975706 learned_addr learned my addr 172.21.15.82:0/3744975706 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e16ffd640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e180ce480 0x7f1e1815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e177fe640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 -- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 msgr2=0x7f1e180b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 -- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e180ce480 msgr2=0x7f1e1815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e180ce480 0x7f1e1815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 -- 172.21.15.82:0/3744975706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e181584a0 con 0x7f1e180d0c40 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.281+0000 7f1e167fc640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e181575e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c400 tx=0x7f1e0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.282+0000 7f1dfffff640 1 -- 172.21.15.82:0/3744975706 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0052d0 con 0x7f1e180d0c40 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.282+0000 7f1dfffff640 1 -- 172.21.15.82:0/3744975706 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c005490 con 0x7f1e180d0c40 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.282+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1815ec00 con 0x7f1e180d0c40 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.282+0000 7f1dfffff640 1 -- 172.21.15.82:0/3744975706 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c007260 con 0x7f1e180d0c40 2024-03-14T12:14:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.282+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1815f1c0 con 0x7f1e180d0c40 2024-03-14T12:14:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.283+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e180ce8e0 con 0x7f1e180d0c40 2024-03-14T12:14:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.283+0000 7f1dfffff640 1 -- 172.21.15.82:0/3744975706 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1e0c007400 con 0x7f1e180d0c40 2024-03-14T12:14:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.284+0000 7f1dfffff640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bc00 0x7f1e0406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.284+0000 7f1e16ffd640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bc00 0x7f1e0406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.284+0000 7f1dfffff640 1 -- 172.21.15.82:0/3744975706 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e0c08a020 con 0x7f1e180d0c40 2024-03-14T12:14:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.284+0000 7f1e16ffd640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bc00 0x7f1e0406e0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1e08003fd0 tx=0x7f1e080039f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:25.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.288+0000 7f1dfffff640 1 -- 172.21.15.82:0/3744975706 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1e180ce8e0 con 0x7f1e180d0c40 2024-03-14T12:14:25.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.470+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1e1804f050 con 0x7f1e180d0c40 2024-03-14T12:14:25.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.470+0000 7f1dfffff640 1 -- 172.21.15.82:0/3744975706 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1e0c058a80 con 0x7f1e180d0c40 2024-03-14T12:14:25.469 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:14:25.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bc00 msgr2=0x7f1e0406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bc00 0x7f1e0406e0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1e08003fd0 tx=0x7f1e080039f0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 msgr2=0x7f1e181575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e181575e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c400 tx=0x7f1e0c00c8c0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 shutdown_connections 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bc00 0x7f1e0406e0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1815e190 0x7f1e180b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d0c40 0x7f1e181575e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 --2- 172.21.15.82:0/3744975706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e180ce480 0x7f1e1815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 >> 172.21.15.82:0/3744975706 conn(0x7f1e180b3610 msgr2=0x7f1e180ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 shutdown_connections 2024-03-14T12:14:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.472+0000 7f1e1d9df640 1 -- 172.21.15.82:0/3744975706 wait complete. 2024-03-14T12:14:25.482 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T12:14:25.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:25.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-14T12:14:25.516 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:14:25.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:25.516 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T12:14:25.579 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:25.580 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:25.595 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:25.595 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:25.651 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:25.651 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:25.706 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.847+0000 7f980daa4640 1 Processor -- start 2024-03-14T12:14:25.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.847+0000 7f980daa4640 1 -- start start 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f980daa4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98080b0f60 0x7f98080cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f980daa4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f980806b420 0x7f98080d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f980daa4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98080b1340 con 0x7f98080afff0 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f980daa4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98080d0ee0 con 0x7f98080b0f60 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f980daa4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98080b4ee0 con 0x7f980806b420 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f98077fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98080b0f60 0x7f98080cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f9806ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f98080afff0 0x7f980806ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50960/0 (socket says 172.21.15.82:50960) 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f9806ffd640 1 -- 172.21.15.82:0/3633604780 learned_addr learned my addr 172.21.15.82:0/3633604780 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f98077fe640 1 -- 172.21.15.82:0/3633604780 >> v2:172.21.15.164:3300/0 conn(0x7f980806b420 msgr2=0x7f98080d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:25.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f98077fe640 1 --2- 172.21.15.82:0/3633604780 >> v2:172.21.15.164:3300/0 conn(0x7f980806b420 0x7f98080d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f98077fe640 1 -- 172.21.15.82:0/3633604780 >> v1:172.21.15.82:6789/0 conn(0x7f98080afff0 legacy=0x7f980806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.848+0000 7f98077fe640 1 -- 172.21.15.82:0/3633604780 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080cd030 con 0x7f98080b0f60 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.849+0000 7f98077fe640 1 --2- 172.21.15.82:0/3633604780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080cc9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f98080d1160 tx=0x7f97f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a791672cd7d6516f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.849+0000 7f98067fc640 1 -- 172.21.15.82:0/3633604780 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8035020 con 0x7f98080b0f60 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.849+0000 7f98067fc640 1 -- 172.21.15.82:0/3633604780 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f8031830 con 0x7f98080b0f60 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.849+0000 7f98067fc640 1 -- 172.21.15.82:0/3633604780 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8031c00 con 0x7f98080b0f60 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 -- 172.21.15.82:0/3633604780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 msgr2=0x7f98080cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 --2- 172.21.15.82:0/3633604780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080cc9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f98080d1160 tx=0x7f97f802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 -- 172.21.15.82:0/3633604780 shutdown_connections 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 --2- 172.21.15.82:0/3633604780 >> v2:172.21.15.164:3300/0 conn(0x7f980806b420 0x7f98080d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 --2- 172.21.15.82:0/3633604780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080cc9a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 -- 172.21.15.82:0/3633604780 >> 172.21.15.82:0/3633604780 conn(0x7f9808072310 msgr2=0x7f9808072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 -- 172.21.15.82:0/3633604780 shutdown_connections 2024-03-14T12:14:25.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.850+0000 7f980daa4640 1 -- 172.21.15.82:0/3633604780 wait complete. 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.851+0000 7f980daa4640 1 Processor -- start 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.851+0000 7f980daa4640 1 -- start start 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f980daa4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f980806b420 0x7f9808151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f980daa4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98080afff0 0x7f980815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f980daa4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f980daa4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080cd8d0 con 0x7f98080b0f60 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f980daa4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080cd5f0 con 0x7f980806b420 2024-03-14T12:14:25.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f980daa4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98080b0450 con 0x7f98080afff0 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39480/0 (socket says 172.21.15.82:39480) 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 -- 172.21.15.82:0/2645862153 learned_addr learned my addr 172.21.15.82:0/2645862153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 -- 172.21.15.82:0/2645862153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98080afff0 msgr2=0x7f980815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98080afff0 0x7f980815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 -- 172.21.15.82:0/2645862153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f980806b420 msgr2=0x7f9808151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f98077fe640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f980806b420 0x7f9808151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f980806b420 0x7f9808151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 -- 172.21.15.82:0/2645862153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080c8090 con 0x7f98080b0f60 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f9807fff640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080ae020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f980000ba10 tx=0x7f980000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f97effff640 1 -- 172.21.15.82:0/2645862153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800019070 con 0x7f98080b0f60 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.852+0000 7f97effff640 1 -- 172.21.15.82:0/2645862153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980000cb40 con 0x7f98080b0f60 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.853+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98080c8370 con 0x7f98080b0f60 2024-03-14T12:14:25.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.853+0000 7f97effff640 1 -- 172.21.15.82:0/2645862153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800014440 con 0x7f98080b0f60 2024-03-14T12:14:25.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.853+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98080c8930 con 0x7f98080b0f60 2024-03-14T12:14:25.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.853+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980804f050 con 0x7f98080b0f60 2024-03-14T12:14:25.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.854+0000 7f97effff640 1 -- 172.21.15.82:0/2645862153 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f980001c070 con 0x7f98080b0f60 2024-03-14T12:14:25.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.854+0000 7f97effff640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f97f406bbd0 0x7f97f406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:25.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.854+0000 7f97effff640 1 -- 172.21.15.82:0/2645862153 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f980008a750 con 0x7f98080b0f60 2024-03-14T12:14:25.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.854+0000 7f98077fe640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f97f406bbd0 0x7f97f406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:25.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.855+0000 7f98077fe640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f97f406bbd0 0x7f97f406e090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97f8036000 tx=0x7f97f803c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:25.859+0000 7f97effff640 1 -- 172.21.15.82:0/2645862153 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9800059240 con 0x7f98080b0f60 2024-03-14T12:14:26.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.031+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9808071200 con 0x7f98080b0f60 2024-03-14T12:14:26.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.032+0000 7f97effff640 1 -- 172.21.15.82:0/2645862153 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f9800058b40 con 0x7f98080b0f60 2024-03-14T12:14:26.032 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:26.032 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.034+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f97f406bbd0 msgr2=0x7f97f406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.034+0000 7f980daa4640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f97f406bbd0 0x7f97f406e090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97f8036000 tx=0x7f97f803c040 comp rx=0 tx=0).stop 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.034+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 msgr2=0x7f98080ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.034+0000 7f980daa4640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080ae020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f980000ba10 tx=0x7f980000bed0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 shutdown_connections 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98080b0f60 0x7f98080ae020 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f97f406bbd0 0x7f97f406e090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98080afff0 0x7f980815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 --2- 172.21.15.82:0/2645862153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f980806b420 0x7f9808151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 >> 172.21.15.82:0/2645862153 conn(0x7f9808072310 msgr2=0x7f98080b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:26.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 shutdown_connections 2024-03-14T12:14:26.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.035+0000 7f980daa4640 1 -- 172.21.15.82:0/2645862153 wait complete. 2024-03-14T12:14:26.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:26.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:26.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.151+0000 7f2679420640 1 Processor -- start 2024-03-14T12:14:26.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.151+0000 7f2679420640 1 -- start start 2024-03-14T12:14:26.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f2679420640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2674145790 0x7f26740b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f2679420640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f26740d02d0 0x7f26740b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f2679420640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26740b2a50 con 0x7f267415c2a0 2024-03-14T12:14:26.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f2679420640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26740ce8e0 con 0x7f26740d02d0 2024-03-14T12:14:26.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f2679420640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26740ceb30 con 0x7f2674145790 2024-03-14T12:14:26.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f26737fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f26740d02d0 0x7f26740b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f26737fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f26740d02d0 0x7f26740b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39490/0 (socket says 172.21.15.82:39490) 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f26737fe640 1 -- 172.21.15.82:0/133780638 learned_addr learned my addr 172.21.15.82:0/133780638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f26737fe640 1 -- 172.21.15.82:0/133780638 >> v2:172.21.15.164:3300/0 conn(0x7f2674145790 msgr2=0x7f26740b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f2672ffd640 1 --2- 172.21.15.82:0/133780638 >> v2:172.21.15.164:3300/0 conn(0x7f2674145790 0x7f26740b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f26737fe640 1 --2- 172.21.15.82:0/133780638 >> v2:172.21.15.164:3300/0 conn(0x7f2674145790 0x7f26740b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f26737fe640 1 -- 172.21.15.82:0/133780638 >> v1:172.21.15.164:6789/0 conn(0x7f267415c2a0 legacy=0x7f26740cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.152+0000 7f26737fe640 1 -- 172.21.15.82:0/133780638 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26740cee10 con 0x7f26740d02d0 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.153+0000 7f26737fe640 1 --2- 172.21.15.82:0/133780638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26740d02d0 0x7f26740b31e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26740719f0 tx=0x7f266c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a194efa9ebbb264 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.153+0000 7f2671ffb640 1 -- 172.21.15.82:0/133780638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c02c9f0 con 0x7f26740d02d0 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.153+0000 7f2671ffb640 1 -- 172.21.15.82:0/133780638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c02cb90 con 0x7f26740d02d0 2024-03-14T12:14:26.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.153+0000 7f2671ffb640 1 -- 172.21.15.82:0/133780638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c0343f0 con 0x7f26740d02d0 2024-03-14T12:14:26.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 -- 172.21.15.82:0/133780638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26740d02d0 msgr2=0x7f26740b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 --2- 172.21.15.82:0/133780638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26740d02d0 0x7f26740b31e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26740719f0 tx=0x7f266c02be50 comp rx=0 tx=0).stop 2024-03-14T12:14:26.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 -- 172.21.15.82:0/133780638 shutdown_connections 2024-03-14T12:14:26.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 --2- 172.21.15.82:0/133780638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26740d02d0 0x7f26740b31e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 --2- 172.21.15.82:0/133780638 >> v2:172.21.15.164:3300/0 conn(0x7f2674145790 0x7f26740b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 -- 172.21.15.82:0/133780638 >> 172.21.15.82:0/133780638 conn(0x7f267406aef0 msgr2=0x7f267406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:26.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 -- 172.21.15.82:0/133780638 shutdown_connections 2024-03-14T12:14:26.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.154+0000 7f2679420640 1 -- 172.21.15.82:0/133780638 wait complete. 2024-03-14T12:14:26.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.155+0000 7f2679420640 1 Processor -- start 2024-03-14T12:14:26.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.155+0000 7f2679420640 1 -- start start 2024-03-14T12:14:26.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f2679420640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26740d02d0 0x7f26740ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f2679420640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2674145790 0x7f26740ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f2679420640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f267415c2a0 0x7f267414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f2679420640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26740cf0f0 con 0x7f2674145790 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f2679420640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26740cc600 con 0x7f26740d02d0 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f2679420640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26740b4d60 con 0x7f267415c2a0 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2674145790 0x7f26740ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2674145790 0x7f26740ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39504/0 (socket says 172.21.15.82:39504) 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 -- 172.21.15.82:0/3367762162 learned_addr learned my addr 172.21.15.82:0/3367762162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f2672ffd640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26740d02d0 0x7f26740ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26737fe640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f267415c2a0 0x7f267414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 -- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f267415c2a0 msgr2=0x7f267414f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f267415c2a0 0x7f267414f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 -- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26740d02d0 msgr2=0x7f26740ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26740d02d0 0x7f26740ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.156+0000 7f26727fc640 1 -- 172.21.15.82:0/3367762162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26740aab40 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.157+0000 7f26727fc640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2674145790 0x7f26740ae810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f266400c410 tx=0x7f266400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.157+0000 7f265b7fe640 1 -- 172.21.15.82:0/3367762162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664017070 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.157+0000 7f265b7fe640 1 -- 172.21.15.82:0/3367762162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26640035f0 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.157+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267414d820 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.157+0000 7f265b7fe640 1 -- 172.21.15.82:0/3367762162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664003a20 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.157+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267414dde0 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.158+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267404f050 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.158+0000 7f265b7fe640 1 -- 172.21.15.82:0/3367762162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f266401a070 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.159+0000 7f265b7fe640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266006bbd0 0x7f266006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.159+0000 7f265b7fe640 1 -- 172.21.15.82:0/3367762162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2664089db0 con 0x7f2674145790 2024-03-14T12:14:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.159+0000 7f2672ffd640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266006bbd0 0x7f266006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.159+0000 7f2672ffd640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266006bbd0 0x7f266006e090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f266800aac0 tx=0x7f2668008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.163+0000 7f265b7fe640 1 -- 172.21.15.82:0/3367762162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f26640010c0 con 0x7f2674145790 2024-03-14T12:14:26.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.337+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2674071ab0 con 0x7f2674145790 2024-03-14T12:14:26.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.338+0000 7f265b7fe640 1 -- 172.21.15.82:0/3367762162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26640582e0 con 0x7f2674145790 2024-03-14T12:14:26.337 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:26.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266006bbd0 msgr2=0x7f266006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266006bbd0 0x7f266006e090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f266800aac0 tx=0x7f2668008040 comp rx=0 tx=0).stop 2024-03-14T12:14:26.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2674145790 msgr2=0x7f26740ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2674145790 0x7f26740ae810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f266400c410 tx=0x7f266400c8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 shutdown_connections 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f266006bbd0 0x7f266006e090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f267415c2a0 0x7f267414f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2674145790 0x7f26740ae810 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 --2- 172.21.15.82:0/3367762162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26740d02d0 0x7f26740ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 >> 172.21.15.82:0/3367762162 conn(0x7f267406aef0 msgr2=0x7f2674141020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 shutdown_connections 2024-03-14T12:14:26.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.340+0000 7f2679420640 1 -- 172.21.15.82:0/3367762162 wait complete. 2024-03-14T12:14:26.350 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:26.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.458+0000 7f0de3a81640 1 Processor -- start 2024-03-14T12:14:26.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.458+0000 7f0de3a81640 1 -- start start 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de3a81640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ddc1447c0 0x7f0ddc06f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de3a81640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0ddc0b4bc0 0x7f0ddc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de3a81640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc06f6d0 con 0x7f0ddc0b3b30 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de3a81640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc06e0f0 con 0x7f0ddc1447c0 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de3a81640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc06e3d0 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de181d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ddc1447c0 0x7f0ddc06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de181d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ddc1447c0 0x7f0ddc06f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39506/0 (socket says 172.21.15.82:39506) 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de181d640 1 -- 172.21.15.82:0/2333743571 learned_addr learned my addr 172.21.15.82:0/2333743571 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de201e640 1 --2- 172.21.15.82:0/2333743571 >> v2:172.21.15.164:3300/0 conn(0x7f0ddc0b4bc0 0x7f0ddc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de181d640 1 -- 172.21.15.82:0/2333743571 >> v2:172.21.15.164:3300/0 conn(0x7f0ddc0b4bc0 msgr2=0x7f0ddc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de181d640 1 --2- 172.21.15.82:0/2333743571 >> v2:172.21.15.164:3300/0 conn(0x7f0ddc0b4bc0 0x7f0ddc0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de181d640 1 -- 172.21.15.82:0/2333743571 >> v1:172.21.15.164:6789/0 conn(0x7f0ddc0b3b30 legacy=0x7f0ddc06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.459+0000 7f0de181d640 1 -- 172.21.15.82:0/2333743571 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc06e6b0 con 0x7f0ddc1447c0 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.460+0000 7f0de181d640 1 --2- 172.21.15.82:0/2333743571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc1447c0 0x7f0ddc06f190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc072860 tx=0x7f0dd802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4e336390e86f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.460+0000 7f0de081b640 1 -- 172.21.15.82:0/2333743571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd803b070 con 0x7f0ddc1447c0 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.460+0000 7f0de081b640 1 -- 172.21.15.82:0/2333743571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd8036440 con 0x7f0ddc1447c0 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.460+0000 7f0de081b640 1 -- 172.21.15.82:0/2333743571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8036810 con 0x7f0ddc1447c0 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 -- 172.21.15.82:0/2333743571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc1447c0 msgr2=0x7f0ddc06f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 --2- 172.21.15.82:0/2333743571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc1447c0 0x7f0ddc06f190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc072860 tx=0x7f0dd802e0e0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 -- 172.21.15.82:0/2333743571 shutdown_connections 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 --2- 172.21.15.82:0/2333743571 >> v2:172.21.15.164:3300/0 conn(0x7f0ddc0b4bc0 0x7f0ddc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 --2- 172.21.15.82:0/2333743571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc1447c0 0x7f0ddc06f190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 -- 172.21.15.82:0/2333743571 >> 172.21.15.82:0/2333743571 conn(0x7f0ddc0d0b40 msgr2=0x7f0ddc0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 -- 172.21.15.82:0/2333743571 shutdown_connections 2024-03-14T12:14:26.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.461+0000 7f0de3a81640 1 -- 172.21.15.82:0/2333743571 wait complete. 2024-03-14T12:14:26.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.463+0000 7f0de3a81640 1 Processor -- start 2024-03-14T12:14:26.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.463+0000 7f0de3a81640 1 -- start start 2024-03-14T12:14:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de3a81640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc0b3b30 0x7f0ddc1f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de3a81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc0b4bc0 0x7f0ddc1493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de3a81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ddc1447c0 0x7f0ddc140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de3a81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0ce0d0 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de3a81640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc06ec70 con 0x7f0ddc0b3b30 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de3a81640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0ce250 con 0x7f0ddc1447c0 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de101c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc0b4bc0 0x7f0ddc1493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de101c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc0b4bc0 0x7f0ddc1493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39522/0 (socket says 172.21.15.82:39522) 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de101c640 1 -- 172.21.15.82:0/756795729 learned_addr learned my addr 172.21.15.82:0/756795729 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de181d640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc0b3b30 0x7f0ddc1f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.464+0000 7f0de201e640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ddc1447c0 0x7f0ddc140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de101c640 1 -- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ddc1447c0 msgr2=0x7f0ddc140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de101c640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ddc1447c0 0x7f0ddc140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de101c640 1 -- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc0b3b30 msgr2=0x7f0ddc1f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de101c640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc0b3b30 0x7f0ddc1f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de101c640 1 -- 172.21.15.82:0/756795729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc141090 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de101c640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc0b4bc0 0x7f0ddc1493f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00c410 tx=0x7f0dcc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0dd27fc640 1 -- 172.21.15.82:0/756795729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc017070 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0dd27fc640 1 -- 172.21.15.82:0/756795729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0035f0 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc1e11b0 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0dd27fc640 1 -- 172.21.15.82:0/756795729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0039e0 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.465+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc1e1770 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.466+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc1e1970 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.466+0000 7f0dd27fc640 1 -- 172.21.15.82:0/756795729 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0dcc01a070 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.467+0000 7f0dd27fc640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0dc006bbd0 0x7f0dc006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.467+0000 7f0dd27fc640 1 -- 172.21.15.82:0/756795729 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dcc089e60 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.467+0000 7f0de181d640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0dc006bbd0 0x7f0dc006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.467+0000 7f0de181d640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0dc006bbd0 0x7f0dc006e090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8038000 tx=0x7f0dd8032000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.471+0000 7f0dd27fc640 1 -- 172.21.15.82:0/756795729 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0ddc1e1970 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.649+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ddc1e0fa0 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.649+0000 7f0dd27fc640 1 -- 172.21.15.82:0/756795729 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0dcc0010c0 con 0x7f0ddc0b4bc0 2024-03-14T12:14:26.649 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0dc006bbd0 msgr2=0x7f0dc006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0dc006bbd0 0x7f0dc006e090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8038000 tx=0x7f0dd8032000 comp rx=0 tx=0).stop 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc0b4bc0 msgr2=0x7f0ddc1493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc0b4bc0 0x7f0ddc1493f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00c410 tx=0x7f0dcc00c8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 shutdown_connections 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0dc006bbd0 0x7f0dc006e090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ddc1447c0 0x7f0ddc140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ddc0b4bc0 0x7f0ddc1493f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 --2- 172.21.15.82:0/756795729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc0b3b30 0x7f0ddc1f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.651+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 >> 172.21.15.82:0/756795729 conn(0x7f0ddc0d0b40 msgr2=0x7f0ddc0ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.652+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 shutdown_connections 2024-03-14T12:14:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.652+0000 7f0de3a81640 1 -- 172.21.15.82:0/756795729 wait complete. 2024-03-14T12:14:26.661 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:26.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.769+0000 7ff87e529640 1 Processor -- start 2024-03-14T12:14:26.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.769+0000 7ff87e529640 1 -- start start 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff87e529640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 0x7ff878168c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff87e529640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780d25a0 con 0x7ff8780d12a0 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff87e529640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780d2880 con 0x7ff87815dff0 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff87e529640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780d2350 con 0x7ff8780d0e10 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff8777fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff8780d12a0 0x7ff8780d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50962/0 (socket says 172.21.15.82:50962) 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff8777fe640 1 -- 172.21.15.82:0/1146360735 learned_addr learned my addr 172.21.15.82:0/1146360735 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff87cac6640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 0x7ff878168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548922461 0 0) 0x7ff8780d25a0 con 0x7ff8780d12a0 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff860003520 con 0x7ff8780d12a0 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663279048 0 0) 0x7ff8780d2880 con 0x7ff87815dff0 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8780d25a0 con 0x7ff87815dff0 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2862797058 0 0) 0x7ff860003520 con 0x7ff8780d12a0 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 msgr2=0x7ff878168c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 0x7ff878168c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 >> v1:172.21.15.164:6789/0 conn(0x7ff87815dff0 legacy=0x7ff8780d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780d2b60 con 0x7ff8780d12a0 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7ff868001670 con 0x7ff8780d12a0 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.770+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 >> v1:172.21.15.82:6789/0 conn(0x7ff8780d12a0 legacy=0x7ff8780d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff876ffd640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 0x7ff860003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff876ffd640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8600040d0 0x7ff860004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff876ffd640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780d2350 con 0x7ff860004d40 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000baa0 con 0x7ff8600040d0 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000bca0 con 0x7ff8780d0e10 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff87cac6640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff8777fe640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8600040d0 0x7ff860004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff877fff640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 0x7ff860003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff87cac6640 1 -- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 msgr2=0x7ff860003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff87cac6640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8780d0e10 0x7ff860003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff87cac6640 1 -- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8600040d0 msgr2=0x7ff860004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff87cac6640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8600040d0 0x7ff860004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.771+0000 7ff87cac6640 1 -- 172.21.15.82:0/1146360735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff860003520 con 0x7ff860004d40 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.772+0000 7ff87cac6640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff870009fc0 tx=0x7ff860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c135894bbebea8ff server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.772+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870039070 con 0x7ff860004d40 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.772+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87002cc20 con 0x7ff860004d40 2024-03-14T12:14:26.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.772+0000 7ff876ffd640 1 -- 172.21.15.82:0/1146360735 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870009c10 con 0x7ff860004d40 2024-03-14T12:14:26.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.773+0000 7ff87e529640 1 -- 172.21.15.82:0/1146360735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff860004d40 msgr2=0x7ff8600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.773+0000 7ff87e529640 1 --2- 172.21.15.82:0/1146360735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff870009fc0 tx=0x7ff860005830 comp rx=0 tx=0).stop 2024-03-14T12:14:26.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.773+0000 7ff87cac6640 1 -- 172.21.15.82:0/1146360735 reap_dead start 2024-03-14T12:14:26.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.773+0000 7ff87e529640 1 -- 172.21.15.82:0/1146360735 shutdown_connections 2024-03-14T12:14:26.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.773+0000 7ff87e529640 1 -- 172.21.15.82:0/1146360735 >> 172.21.15.82:0/1146360735 conn(0x7ff87809d0d0 msgr2=0x7ff8780d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:26.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.773+0000 7ff87e529640 1 -- 172.21.15.82:0/1146360735 shutdown_connections 2024-03-14T12:14:26.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.773+0000 7ff87e529640 1 -- 172.21.15.82:0/1146360735 wait complete. 2024-03-14T12:14:26.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.775+0000 7ff87e529640 1 Processor -- start 2024-03-14T12:14:26.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.775+0000 7ff87e529640 1 -- start start 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87e529640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8781f6fd0 0x7ff8780d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87e529640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8781f5260 0x7ff8780d42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87e529640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff878157b10 0x7ff878157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87e529640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780cc4b0 con 0x7ff878157b10 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87e529640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780cc330 con 0x7ff8781f6fd0 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87e529640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780cc630 con 0x7ff8781f5260 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87cac6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff878157b10 0x7ff878157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87cac6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff878157b10 0x7ff878157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39534/0 (socket says 172.21.15.82:39534) 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.776+0000 7ff87cac6640 1 -- 172.21.15.82:0/2597921693 learned_addr learned my addr 172.21.15.82:0/2597921693 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87cac6640 1 -- 172.21.15.82:0/2597921693 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8781f5260 msgr2=0x7ff8780d42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87cac6640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8781f5260 0x7ff8780d42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87cac6640 1 -- 172.21.15.82:0/2597921693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8781f6fd0 msgr2=0x7ff8780d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87cac6640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8781f6fd0 0x7ff8780d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87cac6640 1 -- 172.21.15.82:0/2597921693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878158870 con 0x7ff878157b10 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87cac6640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff878157b10 0x7ff878157f90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e200 tx=0x7ff870009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff874ff9640 1 -- 172.21.15.82:0/2597921693 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870039070 con 0x7ff878157b10 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff874ff9640 1 -- 172.21.15.82:0/2597921693 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff870044070 con 0x7ff878157b10 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff878158b50 con 0x7ff878157b10 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff874ff9640 1 -- 172.21.15.82:0/2597921693 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870009c10 con 0x7ff878157b10 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.777+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8781f73b0 con 0x7ff878157b10 2024-03-14T12:14:26.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.778+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87804f050 con 0x7ff878157b10 2024-03-14T12:14:26.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.778+0000 7ff874ff9640 1 -- 172.21.15.82:0/2597921693 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff87003e810 con 0x7ff878157b10 2024-03-14T12:14:26.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.779+0000 7ff874ff9640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff86406bc00 0x7ff86406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:26.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.779+0000 7ff874ff9640 1 -- 172.21.15.82:0/2597921693 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8700acd20 con 0x7ff878157b10 2024-03-14T12:14:26.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.779+0000 7ff877fff640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff86406bc00 0x7ff86406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:26.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.779+0000 7ff877fff640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff86406bc00 0x7ff86406e0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0079a0 tx=0x7ff86c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:26.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.783+0000 7ff874ff9640 1 -- 172.21.15.82:0/2597921693 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff87007b810 con 0x7ff878157b10 2024-03-14T12:14:26.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.962+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff87806bb40 con 0x7ff878157b10 2024-03-14T12:14:26.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.963+0000 7ff874ff9640 1 -- 172.21.15.82:0/2597921693 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff87007b110 con 0x7ff878157b10 2024-03-14T12:14:26.962 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff86406bc00 msgr2=0x7ff86406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff86406bc00 0x7ff86406e0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0079a0 tx=0x7ff86c009040 comp rx=0 tx=0).stop 2024-03-14T12:14:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff878157b10 msgr2=0x7ff878157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff878157b10 0x7ff878157f90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e200 tx=0x7ff870009750 comp rx=0 tx=0).stop 2024-03-14T12:14:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 shutdown_connections 2024-03-14T12:14:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff878157b10 0x7ff878157f90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff86406bc00 0x7ff86406e0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8781f5260 0x7ff8780d42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 --2- 172.21.15.82:0/2597921693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8781f6fd0 0x7ff8780d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 >> 172.21.15.82:0/2597921693 conn(0x7ff87809d0d0 msgr2=0x7ff87806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 shutdown_connections 2024-03-14T12:14:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:26.965+0000 7ff87e529640 1 -- 172.21.15.82:0/2597921693 wait complete. 2024-03-14T12:14:26.975 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:14:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.078+0000 7f60f3e75640 1 Processor -- start 2024-03-14T12:14:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.079+0000 7f60f3e75640 1 -- start start 2024-03-14T12:14:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.079+0000 7f60f3e75640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f60ec071740 0x7f60ec071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.079+0000 7f60f3e75640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f60ec143f60 0x7f60ec15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.079+0000 7f60f3e75640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec0b3050 con 0x7f60ec15e670 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.079+0000 7f60f3e75640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec072060 con 0x7f60ec143f60 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.079+0000 7f60f3e75640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec15ebe0 con 0x7f60ec071740 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.079+0000 7f60f2412640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f60ec143f60 0x7f60ec15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.080+0000 7f60f2412640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f60ec143f60 0x7f60ec15b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39542/0 (socket says 172.21.15.82:39542) 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.080+0000 7f60f2412640 1 -- 172.21.15.82:0/3758299716 learned_addr learned my addr 172.21.15.82:0/3758299716 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.080+0000 7f60f1c11640 1 --2- 172.21.15.82:0/3758299716 >> v2:172.21.15.164:3300/0 conn(0x7f60ec071740 0x7f60ec071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.080+0000 7f60f2412640 1 -- 172.21.15.82:0/3758299716 >> v2:172.21.15.164:3300/0 conn(0x7f60ec071740 msgr2=0x7f60ec071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.080+0000 7f60f2412640 1 --2- 172.21.15.82:0/3758299716 >> v2:172.21.15.164:3300/0 conn(0x7f60ec071740 0x7f60ec071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.080+0000 7f60f2412640 1 -- 172.21.15.82:0/3758299716 >> v1:172.21.15.164:6789/0 conn(0x7f60ec15e670 legacy=0x7f60ec0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.080+0000 7f60f2412640 1 -- 172.21.15.82:0/3758299716 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec15eec0 con 0x7f60ec143f60 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f2412640 1 --2- 172.21.15.82:0/3758299716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec143f60 0x7f60ec15b610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0d04e0 tx=0x7f60e402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f9c8a3d9e362aa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f0c0f640 1 -- 172.21.15.82:0/3758299716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e402ea90 con 0x7f60ec143f60 2024-03-14T12:14:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f0c0f640 1 -- 172.21.15.82:0/3758299716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e402ec30 con 0x7f60ec143f60 2024-03-14T12:14:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f0c0f640 1 -- 172.21.15.82:0/3758299716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4009600 con 0x7f60ec143f60 2024-03-14T12:14:27.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 -- 172.21.15.82:0/3758299716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec143f60 msgr2=0x7f60ec15b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3758299716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec143f60 0x7f60ec15b610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0d04e0 tx=0x7f60e402e0f0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 -- 172.21.15.82:0/3758299716 shutdown_connections 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3758299716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec143f60 0x7f60ec15b610 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3758299716 >> v2:172.21.15.164:3300/0 conn(0x7f60ec071740 0x7f60ec071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 -- 172.21.15.82:0/3758299716 >> 172.21.15.82:0/3758299716 conn(0x7f60ec0b1c70 msgr2=0x7f60ec0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 -- 172.21.15.82:0/3758299716 shutdown_connections 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.081+0000 7f60f3e75640 1 -- 172.21.15.82:0/3758299716 wait complete. 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.082+0000 7f60f3e75640 1 Processor -- start 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f3e75640 1 -- start start 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f3e75640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60ec071740 0x7f60ec157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f3e75640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60ec143f60 0x7f60ec13ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f3e75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec15e670 0x7f60ec06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f3e75640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec143ce0 con 0x7f60ec15e670 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f3e75640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec15f480 con 0x7f60ec071740 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f3e75640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec15b080 con 0x7f60ec143f60 2024-03-14T12:14:27.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f2412640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec15e670 0x7f60ec06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f2412640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec15e670 0x7f60ec06ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39546/0 (socket says 172.21.15.82:39546) 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f2412640 1 -- 172.21.15.82:0/3798553468 learned_addr learned my addr 172.21.15.82:0/3798553468 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f1c11640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60ec071740 0x7f60ec157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.083+0000 7f60f1410640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60ec143f60 0x7f60ec13ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f2412640 1 -- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60ec143f60 msgr2=0x7f60ec13ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f2412640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60ec143f60 0x7f60ec13ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f2412640 1 -- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60ec071740 msgr2=0x7f60ec157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f2412640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60ec071740 0x7f60ec157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f2412640 1 -- 172.21.15.82:0/3798553468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0ca1d0 con 0x7f60ec15e670 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f1c11640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60ec071740 0x7f60ec157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f2412640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec15e670 0x7f60ec06ad10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d240 tx=0x7f60e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60e27fc640 1 -- 172.21.15.82:0/3798553468 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e402ea90 con 0x7f60ec15e670 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60e27fc640 1 -- 172.21.15.82:0/3798553468 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e4045070 con 0x7f60ec15e670 2024-03-14T12:14:27.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60e27fc640 1 -- 172.21.15.82:0/3798553468 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e400b720 con 0x7f60ec15e670 2024-03-14T12:14:27.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec0ca4b0 con 0x7f60ec15e670 2024-03-14T12:14:27.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.084+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec0caa70 con 0x7f60ec15e670 2024-03-14T12:14:27.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.085+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec04f050 con 0x7f60ec15e670 2024-03-14T12:14:27.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.085+0000 7f60e27fc640 1 -- 172.21.15.82:0/3798553468 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f60e4036a60 con 0x7f60ec15e670 2024-03-14T12:14:27.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.086+0000 7f60e27fc640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d006bbd0 0x7f60d006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.086+0000 7f60e27fc640 1 -- 172.21.15.82:0/3798553468 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60e40ad950 con 0x7f60ec15e670 2024-03-14T12:14:27.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.086+0000 7f60f1c11640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d006bbd0 0x7f60d006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.086+0000 7f60f1c11640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d006bbd0 0x7f60d006e090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006fd0 tx=0x7f60e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.090+0000 7f60e27fc640 1 -- 172.21.15.82:0/3798553468 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f60e4035e40 con 0x7f60ec15e670 2024-03-14T12:14:27.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.277+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f60ec0d07b0 con 0x7f60ec15e670 2024-03-14T12:14:27.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.277+0000 7f60e27fc640 1 -- 172.21.15.82:0/3798553468 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f60e407c4e0 con 0x7f60ec15e670 2024-03-14T12:14:27.277 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:14:27.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d006bbd0 msgr2=0x7f60d006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d006bbd0 0x7f60d006e090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006fd0 tx=0x7f60e8008040 comp rx=0 tx=0).stop 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec15e670 msgr2=0x7f60ec06ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec15e670 0x7f60ec06ad10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d240 tx=0x7f60e4004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 shutdown_connections 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60ec15e670 0x7f60ec06ad10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d006bbd0 0x7f60d006e090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60ec143f60 0x7f60ec13ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 --2- 172.21.15.82:0/3798553468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60ec071740 0x7f60ec157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 >> 172.21.15.82:0/3798553468 conn(0x7f60ec0b1c70 msgr2=0x7f60ec0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 shutdown_connections 2024-03-14T12:14:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.279+0000 7f60f3e75640 1 -- 172.21.15.82:0/3798553468 wait complete. 2024-03-14T12:14:27.289 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-14T12:14:27.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:27.289 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-14T12:14:27.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:14:27.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:27.318 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-14T12:14:27.373 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:27.373 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:27.392 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:27.392 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:27.450 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:14:27.450 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:27.508 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:27.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.610+0000 7f9d447a9640 1 Processor -- start 2024-03-14T12:14:27.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.610+0000 7f9d447a9640 1 -- start start 2024-03-14T12:14:27.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d447a9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d3c15e670 0x7f9d3c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d447a9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c143f60 0x7f9d3c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d447a9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3c15ebe0 con 0x7f9d3c071740 2024-03-14T12:14:27.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d447a9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3c15eec0 con 0x7f9d3c15e670 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d447a9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c15f1a0 con 0x7f9d3c143f60 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d41d44640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d3c15e670 0x7f9d3c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d41d44640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d3c15e670 0x7f9d3c0b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39552/0 (socket says 172.21.15.82:39552) 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d41d44640 1 -- 172.21.15.82:0/3494588312 learned_addr learned my addr 172.21.15.82:0/3494588312 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d41d44640 1 -- 172.21.15.82:0/3494588312 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c143f60 msgr2=0x7f9d3c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d41d44640 1 --2- 172.21.15.82:0/3494588312 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c143f60 0x7f9d3c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d41d44640 1 -- 172.21.15.82:0/3494588312 >> v1:172.21.15.164:6789/0 conn(0x7f9d3c071740 legacy=0x7f9d3c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:27.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.611+0000 7f9d41d44640 1 -- 172.21.15.82:0/3494588312 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c15f480 con 0x7f9d3c15e670 2024-03-14T12:14:27.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.612+0000 7f9d41d44640 1 --2- 172.21.15.82:0/3494588312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c15e670 0x7f9d3c0b2940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00bb40 tx=0x7f9d2c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ba86397c9ae00b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.612+0000 7f9d41543640 1 -- 172.21.15.82:0/3494588312 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c03b070 con 0x7f9d3c15e670 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.612+0000 7f9d41543640 1 -- 172.21.15.82:0/3494588312 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c036440 con 0x7f9d3c15e670 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.612+0000 7f9d41543640 1 -- 172.21.15.82:0/3494588312 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c036810 con 0x7f9d3c15e670 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.612+0000 7f9d447a9640 1 -- 172.21.15.82:0/3494588312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c15e670 msgr2=0x7f9d3c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.612+0000 7f9d447a9640 1 --2- 172.21.15.82:0/3494588312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c15e670 0x7f9d3c0b2940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00bb40 tx=0x7f9d2c02e0c0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.613+0000 7f9d447a9640 1 -- 172.21.15.82:0/3494588312 shutdown_connections 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.613+0000 7f9d447a9640 1 --2- 172.21.15.82:0/3494588312 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c143f60 0x7f9d3c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.613+0000 7f9d447a9640 1 --2- 172.21.15.82:0/3494588312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c15e670 0x7f9d3c0b2940 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.613+0000 7f9d447a9640 1 -- 172.21.15.82:0/3494588312 >> 172.21.15.82:0/3494588312 conn(0x7f9d3c0b1c70 msgr2=0x7f9d3c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.613+0000 7f9d447a9640 1 -- 172.21.15.82:0/3494588312 shutdown_connections 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.613+0000 7f9d447a9640 1 -- 172.21.15.82:0/3494588312 wait complete. 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 Processor -- start 2024-03-14T12:14:27.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 -- start start 2024-03-14T12:14:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d3c071740 0x7f9d3c15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c143f60 0x7f9d3c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c15e670 0x7f9d3c0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c144670 con 0x7f9d3c143f60 2024-03-14T12:14:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c15b080 con 0x7f9d3c071740 2024-03-14T12:14:27.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.614+0000 7f9d447a9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c1447f0 con 0x7f9d3c15e670 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c143f60 0x7f9d3c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c143f60 0x7f9d3c0b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39558/0 (socket says 172.21.15.82:39558) 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 -- 172.21.15.82:0/4019453912 learned_addr learned my addr 172.21.15.82:0/4019453912 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 -- 172.21.15.82:0/4019453912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c15e670 msgr2=0x7f9d3c0b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c15e670 0x7f9d3c0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 -- 172.21.15.82:0/4019453912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d3c071740 msgr2=0x7f9d3c15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d3c071740 0x7f9d3c15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 -- 172.21.15.82:0/4019453912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c15c3d0 con 0x7f9d3c143f60 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d41d44640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c143f60 0x7f9d3c0b2f10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c00 tx=0x7f9d2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d32ffd640 1 -- 172.21.15.82:0/4019453912 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c046070 con 0x7f9d3c143f60 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d32ffd640 1 -- 172.21.15.82:0/4019453912 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c034a10 con 0x7f9d3c143f60 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c0ad8f0 con 0x7f9d3c143f60 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d32ffd640 1 -- 172.21.15.82:0/4019453912 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c03b070 con 0x7f9d3c143f60 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.615+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c0adeb0 con 0x7f9d3c143f60 2024-03-14T12:14:27.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.616+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3c04f050 con 0x7f9d3c143f60 2024-03-14T12:14:27.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.617+0000 7f9d32ffd640 1 -- 172.21.15.82:0/4019453912 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9d2c049070 con 0x7f9d3c143f60 2024-03-14T12:14:27.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.617+0000 7f9d32ffd640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d2006bbd0 0x7f9d2006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.617+0000 7f9d32ffd640 1 -- 172.21.15.82:0/4019453912 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d2c0acaf0 con 0x7f9d3c143f60 2024-03-14T12:14:27.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.617+0000 7f9d42545640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d2006bbd0 0x7f9d2006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.617+0000 7f9d42545640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d2006bbd0 0x7f9d2006e090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006fd0 tx=0x7f9d38006bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.622+0000 7f9d32ffd640 1 -- 172.21.15.82:0/4019453912 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9d3c15c3d0 con 0x7f9d3c143f60 2024-03-14T12:14:27.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.794+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d3c0d07b0 con 0x7f9d3c143f60 2024-03-14T12:14:27.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.795+0000 7f9d32ffd640 1 -- 172.21.15.82:0/4019453912 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f9d2c07b670 con 0x7f9d3c143f60 2024-03-14T12:14:27.795 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:27.795 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d2006bbd0 msgr2=0x7f9d2006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d2006bbd0 0x7f9d2006e090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006fd0 tx=0x7f9d38006bc0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c143f60 msgr2=0x7f9d3c0b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c143f60 0x7f9d3c0b2f10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c00 tx=0x7f9d2c004730 comp rx=0 tx=0).stop 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 shutdown_connections 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d2006bbd0 0x7f9d2006e090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d3c15e670 0x7f9d3c0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3c143f60 0x7f9d3c0b2f10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 --2- 172.21.15.82:0/4019453912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d3c071740 0x7f9d3c15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 >> 172.21.15.82:0/4019453912 conn(0x7f9d3c0b1c70 msgr2=0x7f9d3c0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 shutdown_connections 2024-03-14T12:14:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.797+0000 7f9d447a9640 1 -- 172.21.15.82:0/4019453912 wait complete. 2024-03-14T12:14:27.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:27.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:27.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.911+0000 7f6979c92640 1 Processor -- start 2024-03-14T12:14:27.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.911+0000 7f6979c92640 1 -- start start 2024-03-14T12:14:27.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f6979c92640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6974145030 0x7f69740cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f6979c92640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69740d04e0 con 0x7f69740b29b0 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f6979c92640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69740b48a0 con 0x7f69740cfbc0 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f6979c92640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69740ce2e0 con 0x7f6974145030 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f6973fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f69740b29b0 0x7f69740ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50966/0 (socket says 172.21.15.82:50966) 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f6973fff640 1 -- 172.21.15.82:0/2194019210 learned_addr learned my addr 172.21.15.82:0/2194019210 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f6972ffd640 1 --2- 172.21.15.82:0/2194019210 >> v2:172.21.15.164:3300/0 conn(0x7f6974145030 0x7f69740cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738666738 0 0) 0x7f69740d04e0 con 0x7f69740b29b0 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f695c003520 con 0x7f69740b29b0 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1732521930 0 0) 0x7f695c003520 con 0x7f69740b29b0 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 >> v2:172.21.15.164:3300/0 conn(0x7f6974145030 msgr2=0x7f69740cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 --2- 172.21.15.82:0/2194019210 >> v2:172.21.15.164:3300/0 conn(0x7f6974145030 0x7f69740cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 >> v1:172.21.15.164:6789/0 conn(0x7f69740cfbc0 legacy=0x7f6974160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69740b4b80 con 0x7f69740b29b0 2024-03-14T12:14:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f696c0019f0 con 0x7f69740b29b0 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 >> v1:172.21.15.82:6789/0 conn(0x7f69740b29b0 legacy=0x7f69740ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69740ce2e0 con 0x7f695c0039b0 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c00c060 con 0x7f695c0051c0 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.912+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c00c260 con 0x7f695c004800 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f69737fe640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f6973fff640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f6972ffd640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f69737fe640 1 -- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c004800 msgr2=0x7f695c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f69737fe640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f69737fe640 1 -- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695c0051c0 msgr2=0x7f695c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f69737fe640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f6973fff640 1 -- 172.21.15.82:0/2194019210 reap_dead start 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f69737fe640 1 -- 172.21.15.82:0/2194019210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c003520 con 0x7f695c0039b0 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.913+0000 7f69737fe640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6964001750 tx=0x7f695c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d426fa9745836066 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964001370 con 0x7f695c0039b0 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69640052d0 con 0x7f695c0039b0 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f69727fc640 1 -- 172.21.15.82:0/2194019210 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964003830 con 0x7f695c0039b0 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f6979c92640 1 -- 172.21.15.82:0/2194019210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695c0039b0 msgr2=0x7f695c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f6979c92640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6964001750 tx=0x7f695c005830 comp rx=0 tx=0).stop 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f6979c92640 1 -- 172.21.15.82:0/2194019210 shutdown_connections 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f6979c92640 1 --2- 172.21.15.82:0/2194019210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f695c0039b0 0x7f695c003dd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f6979c92640 1 -- 172.21.15.82:0/2194019210 >> 172.21.15.82:0/2194019210 conn(0x7f697406aef0 msgr2=0x7f697406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f6979c92640 1 -- 172.21.15.82:0/2194019210 shutdown_connections 2024-03-14T12:14:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.914+0000 7f6979c92640 1 -- 172.21.15.82:0/2194019210 wait complete. 2024-03-14T12:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.915+0000 7f6979c92640 1 Processor -- start 2024-03-14T12:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6979c92640 1 -- start start 2024-03-14T12:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6979c92640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c0039b0 0x7f6974138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6979c92640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697413ecb0 0x7f697414db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6979c92640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697413f110 0x7f697414e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6979c92640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6974144c60 con 0x7f697413f110 2024-03-14T12:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6979c92640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69740cdf70 con 0x7f697413ecb0 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6979c92640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6974144de0 con 0x7f695c0039b0 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6973fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697413f110 0x7f697414e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6973fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697413f110 0x7f697414e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39582/0 (socket says 172.21.15.82:39582) 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.916+0000 7f6973fff640 1 -- 172.21.15.82:0/3833054335 learned_addr learned my addr 172.21.15.82:0/3833054335 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f69737fe640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c0039b0 0x7f6974138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6972ffd640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697413ecb0 0x7f697414db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6973fff640 1 -- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c0039b0 msgr2=0x7f6974138f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6973fff640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c0039b0 0x7f6974138f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6973fff640 1 -- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697413ecb0 msgr2=0x7f697414db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6973fff640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697413ecb0 0x7f697414db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6973fff640 1 -- 172.21.15.82:0/3833054335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697413bbf0 con 0x7f697413f110 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f69737fe640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c0039b0 0x7f6974138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6973fff640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697413f110 0x7f697414e0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6974071c30 tx=0x7f696c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f695bfff640 1 -- 172.21.15.82:0/3833054335 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00ab00 con 0x7f697413f110 2024-03-14T12:14:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f695bfff640 1 -- 172.21.15.82:0/3833054335 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c018070 con 0x7f697413f110 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697413c920 con 0x7f697413f110 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f695bfff640 1 -- 172.21.15.82:0/3833054335 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0133f0 con 0x7f697413f110 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.917+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697413cee0 con 0x7f697413f110 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.918+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697404f050 con 0x7f697413f110 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.918+0000 7f695bfff640 1 -- 172.21.15.82:0/3833054335 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f696c013590 con 0x7f697413f110 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.919+0000 7f695bfff640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f696006bc00 0x7f696006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.919+0000 7f695bfff640 1 -- 172.21.15.82:0/3833054335 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f696c08afd0 con 0x7f697413f110 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.919+0000 7f69737fe640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f696006bc00 0x7f696006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.919+0000 7f69737fe640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f696006bc00 0x7f696006e0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6964033000 tx=0x7f6964009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:27.923+0000 7f695bfff640 1 -- 172.21.15.82:0/3833054335 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f696c059a30 con 0x7f697413f110 2024-03-14T12:14:28.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.092+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f697413d0b0 con 0x7f697413f110 2024-03-14T12:14:28.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.093+0000 7f695bfff640 1 -- 172.21.15.82:0/3833054335 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f696c059330 con 0x7f697413f110 2024-03-14T12:14:28.092 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f696006bc00 msgr2=0x7f696006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f696006bc00 0x7f696006e0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6964033000 tx=0x7f6964009030 comp rx=0 tx=0).stop 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697413f110 msgr2=0x7f697414e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697413f110 0x7f697414e0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6974071c30 tx=0x7f696c007a90 comp rx=0 tx=0).stop 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 shutdown_connections 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697413f110 0x7f697414e0c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f697413ecb0 0x7f697414db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f696006bc00 0x7f696006e0c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 --2- 172.21.15.82:0/3833054335 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f695c0039b0 0x7f6974138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 >> 172.21.15.82:0/3833054335 conn(0x7f697406aef0 msgr2=0x7f69740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:28.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 shutdown_connections 2024-03-14T12:14:28.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.095+0000 7f6979c92640 1 -- 172.21.15.82:0/3833054335 wait complete. 2024-03-14T12:14:28.105 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:28.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.208+0000 7fc7adadc640 1 Processor -- start 2024-03-14T12:14:28.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.208+0000 7fc7adadc640 1 -- start start 2024-03-14T12:14:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7adadc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7a815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7adadc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a80b34e0 con 0x7fc7a81446f0 2024-03-14T12:14:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7adadc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a815f4a0 con 0x7fc7a80cfe30 2024-03-14T12:14:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7adadc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b3780 con 0x7fc7a80d0210 2024-03-14T12:14:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7a6ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc7a81446f0 0x7fc7a815e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50970/0 (socket says 172.21.15.82:50970) 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7a6ffd640 1 -- 172.21.15.82:0/1071743518 learned_addr learned my addr 172.21.15.82:0/1071743518 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7a7fff640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7a815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426321305 0 0) 0x7fc7a80b34e0 con 0x7fc7a81446f0 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.209+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc790003520 con 0x7fc7a81446f0 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669181625 0 0) 0x7fc7a815f4a0 con 0x7fc7a80cfe30 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a80b34e0 con 0x7fc7a80cfe30 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2219980351 0 0) 0x7fc790003520 con 0x7fc7a81446f0 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 msgr2=0x7fc7a815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7a815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 >> v1:172.21.15.164:6789/0 conn(0x7fc7a80cfe30 legacy=0x7fc7a8143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a80b3a60 con 0x7fc7a81446f0 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fc79c001670 con 0x7fc7a81446f0 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a7fff640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7a815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 >> v1:172.21.15.82:6789/0 conn(0x7fc7a81446f0 legacy=0x7fc7a815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc790003b70 0x7fc790003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc790004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b3780 con 0x7fc790004800 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79000ba70 con 0x7fc790003b70 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79000bc40 con 0x7fc7a80d0210 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a6ffd640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc790004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a77fe640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc790003b70 0x7fc790003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a7fff640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a6ffd640 1 -- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 msgr2=0x7fc7900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a6ffd640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a6ffd640 1 -- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc790003b70 msgr2=0x7fc790003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a6ffd640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc790003b70 0x7fc790003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a6ffd640 1 -- 172.21.15.82:0/1071743518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790003520 con 0x7fc790004800 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a77fe640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc790003b70 0x7fc790003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.210+0000 7fc7a7fff640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80d0210 0x7fc7900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.211+0000 7fc7a6ffd640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc790004c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007f90 tx=0x7fc790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c658480885bb3f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.211+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c007be0 con 0x7fc790004800 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.211+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c030070 con 0x7fc790004800 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.211+0000 7fc7a67fc640 1 -- 172.21.15.82:0/1071743518 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c005d30 con 0x7fc790004800 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.212+0000 7fc7adadc640 1 -- 172.21.15.82:0/1071743518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 msgr2=0x7fc790004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.212+0000 7fc7adadc640 1 --2- 172.21.15.82:0/1071743518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc790004c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007f90 tx=0x7fc790005830 comp rx=0 tx=0).stop 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.212+0000 7fc7a7fff640 1 -- 172.21.15.82:0/1071743518 reap_dead start 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.212+0000 7fc7adadc640 1 -- 172.21.15.82:0/1071743518 shutdown_connections 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.212+0000 7fc7adadc640 1 -- 172.21.15.82:0/1071743518 >> 172.21.15.82:0/1071743518 conn(0x7fc7a806aef0 msgr2=0x7fc7a806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.212+0000 7fc7adadc640 1 -- 172.21.15.82:0/1071743518 shutdown_connections 2024-03-14T12:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.212+0000 7fc7adadc640 1 -- 172.21.15.82:0/1071743518 wait complete. 2024-03-14T12:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.213+0000 7fc7adadc640 1 Processor -- start 2024-03-14T12:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.213+0000 7fc7adadc640 1 -- start start 2024-03-14T12:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc7a8143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8154f50 0x7fc7a80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80b1010 0x7fc7a81531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b43e0 con 0x7fc790004800 2024-03-14T12:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b4260 con 0x7fc7a8154f50 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a815ee80 con 0x7fc7a80b1010 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc7a8143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc7a8143090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39598/0 (socket says 172.21.15.82:39598) 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 -- 172.21.15.82:0/3688556039 learned_addr learned my addr 172.21.15.82:0/3688556039 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a7fff640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80b1010 0x7fc7a81531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 -- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80b1010 msgr2=0x7fc7a81531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a6ffd640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8154f50 0x7fc7a80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80b1010 0x7fc7a81531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 -- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8154f50 msgr2=0x7fc7a80b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8154f50 0x7fc7a80b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 -- 172.21.15.82:0/3688556039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a81435d0 con 0x7fc790004800 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a7fff640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80b1010 0x7fc7a81531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7a77fe640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc7a8143090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc79800e780 tx=0x7fc79800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc78ffff640 1 -- 172.21.15.82:0/3688556039 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7980050d0 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc78ffff640 1 -- 172.21.15.82:0/3688556039 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc798005270 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a8153eb0 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc78ffff640 1 -- 172.21.15.82:0/3688556039 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798005440 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.214+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a8154470 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.215+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a804f050 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.216+0000 7fc78ffff640 1 -- 172.21.15.82:0/3688556039 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc79801a020 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.216+0000 7fc78ffff640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc79406bbd0 0x7fc79406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.216+0000 7fc7a6ffd640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc79406bbd0 0x7fc79406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.216+0000 7fc78ffff640 1 -- 172.21.15.82:0/3688556039 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc79808a260 con 0x7fc790004800 2024-03-14T12:14:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.217+0000 7fc7a6ffd640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc79406bbd0 0x7fc79406e090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc79c034000 tx=0x7fc79c007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.221+0000 7fc78ffff640 1 -- 172.21.15.82:0/3688556039 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc7a81435d0 con 0x7fc790004800 2024-03-14T12:14:28.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.391+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7a806e800 con 0x7fc790004800 2024-03-14T12:14:28.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.391+0000 7fc78ffff640 1 -- 172.21.15.82:0/3688556039 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc798058e10 con 0x7fc790004800 2024-03-14T12:14:28.391 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:28.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc79406bbd0 msgr2=0x7fc79406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc79406bbd0 0x7fc79406e090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc79c034000 tx=0x7fc79c007000 comp rx=0 tx=0).stop 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 msgr2=0x7fc7a8143090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc7a8143090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc79800e780 tx=0x7fc79800ec40 comp rx=0 tx=0).stop 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 shutdown_connections 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc79406bbd0 0x7fc79406e090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7a80b1010 0x7fc7a81531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7a8154f50 0x7fc7a80b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 --2- 172.21.15.82:0/3688556039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc790004800 0x7fc7a8143090 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 >> 172.21.15.82:0/3688556039 conn(0x7fc7a806aef0 msgr2=0x7fc7a80af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 shutdown_connections 2024-03-14T12:14:28.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.393+0000 7fc7adadc640 1 -- 172.21.15.82:0/3688556039 wait complete. 2024-03-14T12:14:28.403 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.507+0000 7f3ebb3ab640 1 Processor -- start 2024-03-14T12:14:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.507+0000 7f3ebb3ab640 1 -- start start 2024-03-14T12:14:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.507+0000 7f3ebb3ab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb41617c0 0x7f3eb40b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3ebb3ab640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3eb40b36a0 0x7f3eb40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3ebb3ab640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb4161ba0 con 0x7f3eb40b09c0 2024-03-14T12:14:28.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3ebb3ab640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb40b4040 con 0x7f3eb40b36a0 2024-03-14T12:14:28.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3ebb3ab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40ce730 con 0x7f3eb41617c0 2024-03-14T12:14:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9147640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb41617c0 0x7f3eb40b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb8946640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3eb40b36a0 0x7f3eb40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb8946640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3eb40b36a0 0x7f3eb40b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:49688/0 (socket says 172.21.15.82:49688) 2024-03-14T12:14:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9147640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb41617c0 0x7f3eb40b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:38758/0 (socket says 172.21.15.82:38758) 2024-03-14T12:14:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9948640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3eb40b09c0 0x7f3eb40b0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:51352/0 (socket says 172.21.15.82:51352) 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9147640 1 -- 172.21.15.82:0/3000697907 learned_addr learned my addr 172.21.15.82:0/3000697907 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9147640 1 -- 172.21.15.82:0/3000697907 >> v2:172.21.15.164:3300/0 conn(0x7f3eb40b36a0 msgr2=0x7f3eb40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9147640 1 --2- 172.21.15.82:0/3000697907 >> v2:172.21.15.164:3300/0 conn(0x7f3eb40b36a0 0x7f3eb40b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9147640 1 -- 172.21.15.82:0/3000697907 >> v1:172.21.15.164:6789/0 conn(0x7f3eb40b09c0 legacy=0x7f3eb40b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.508+0000 7f3eb9147640 1 -- 172.21.15.82:0/3000697907 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40d4ba0 con 0x7f3eb41617c0 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.509+0000 7f3eb9147640 1 --2- 172.21.15.82:0/3000697907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb41617c0 0x7f3eb40b23b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009e50 tx=0x7f3eb002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df02694732957a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.509+0000 7f3eabfff640 1 -- 172.21.15.82:0/3000697907 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb002cb00 con 0x7f3eb41617c0 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.509+0000 7f3eabfff640 1 -- 172.21.15.82:0/3000697907 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb002cca0 con 0x7f3eb41617c0 2024-03-14T12:14:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.509+0000 7f3eabfff640 1 -- 172.21.15.82:0/3000697907 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0034610 con 0x7f3eb41617c0 2024-03-14T12:14:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/3000697907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb41617c0 msgr2=0x7f3eb40b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/3000697907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb41617c0 0x7f3eb40b23b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009e50 tx=0x7f3eb002bf60 comp rx=0 tx=0).stop 2024-03-14T12:14:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/3000697907 shutdown_connections 2024-03-14T12:14:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/3000697907 >> v2:172.21.15.164:3300/0 conn(0x7f3eb40b36a0 0x7f3eb40b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/3000697907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb41617c0 0x7f3eb40b23b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/3000697907 >> 172.21.15.82:0/3000697907 conn(0x7f3eb40d3d70 msgr2=0x7f3eb40d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/3000697907 shutdown_connections 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.510+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/3000697907 wait complete. 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.511+0000 7f3ebb3ab640 1 Processor -- start 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.511+0000 7f3ebb3ab640 1 -- start start 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3ebb3ab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb40b09c0 0x7f3eb4158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3ebb3ab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb40b36a0 0x7f3eb415fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3ebb3ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb41617c0 0x7f3eb4160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3ebb3ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40d23f0 con 0x7f3eb41617c0 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3ebb3ab640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40b2fe0 con 0x7f3eb40b09c0 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3ebb3ab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40d26d0 con 0x7f3eb40b36a0 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb41617c0 0x7f3eb4160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb41617c0 0x7f3eb4160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39610/0 (socket says 172.21.15.82:39610) 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 -- 172.21.15.82:0/681796445 learned_addr learned my addr 172.21.15.82:0/681796445 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb8946640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb40b36a0 0x7f3eb415fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9147640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb40b09c0 0x7f3eb4158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 -- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb40b36a0 msgr2=0x7f3eb415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb40b36a0 0x7f3eb415fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 -- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb40b09c0 msgr2=0x7f3eb4158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb40b09c0 0x7f3eb4158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9948640 1 -- 172.21.15.82:0/681796445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40b7770 con 0x7f3eb41617c0 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.512+0000 7f3eb9147640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb40b09c0 0x7f3eb4158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.513+0000 7f3eb9948640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb41617c0 0x7f3eb4160120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c400 tx=0x7f3eac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.513+0000 7f3ea9ffb640 1 -- 172.21.15.82:0/681796445 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac0074e0 con 0x7f3eb41617c0 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.513+0000 7f3ea9ffb640 1 -- 172.21.15.82:0/681796445 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac007680 con 0x7f3eb41617c0 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.513+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb40b7a50 con 0x7f3eb41617c0 2024-03-14T12:14:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.513+0000 7f3ea9ffb640 1 -- 172.21.15.82:0/681796445 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac0049e0 con 0x7f3eb41617c0 2024-03-14T12:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.513+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb40b8010 con 0x7f3eb41617c0 2024-03-14T12:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.514+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb404f050 con 0x7f3eb41617c0 2024-03-14T12:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.514+0000 7f3ea9ffb640 1 -- 172.21.15.82:0/681796445 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3eac01a020 con 0x7f3eb41617c0 2024-03-14T12:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.515+0000 7f3ea9ffb640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e9806bbd0 0x7f3e9806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.515+0000 7f3eb9147640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e9806bbd0 0x7f3e9806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.515+0000 7f3ea9ffb640 1 -- 172.21.15.82:0/681796445 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3eac08a100 con 0x7f3eb41617c0 2024-03-14T12:14:28.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.515+0000 7f3eb9147640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e9806bbd0 0x7f3e9806e090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0035000 tx=0x7f3eb0030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.519+0000 7f3ea9ffb640 1 -- 172.21.15.82:0/681796445 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3eb4161ba0 con 0x7f3eb41617c0 2024-03-14T12:14:28.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.693+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3eb40d0920 con 0x7f3eb41617c0 2024-03-14T12:14:28.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.693+0000 7f3ea9ffb640 1 -- 172.21.15.82:0/681796445 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3eac058c80 con 0x7f3eb41617c0 2024-03-14T12:14:28.692 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:28.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e9806bbd0 msgr2=0x7f3e9806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e9806bbd0 0x7f3e9806e090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0035000 tx=0x7f3eb0030000 comp rx=0 tx=0).stop 2024-03-14T12:14:28.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb41617c0 msgr2=0x7f3eb4160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb41617c0 0x7f3eb4160120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c400 tx=0x7f3eac00c8c0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 shutdown_connections 2024-03-14T12:14:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb41617c0 0x7f3eb4160120 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e9806bbd0 0x7f3e9806e090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb40b36a0 0x7f3eb415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 --2- 172.21.15.82:0/681796445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb40b09c0 0x7f3eb4158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 >> 172.21.15.82:0/681796445 conn(0x7f3eb40d3d70 msgr2=0x7f3eb406c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 shutdown_connections 2024-03-14T12:14:28.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.695+0000 7f3ebb3ab640 1 -- 172.21.15.82:0/681796445 wait complete. 2024-03-14T12:14:28.705 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:14:28.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.807+0000 7f200a8f8640 1 Processor -- start 2024-03-14T12:14:28.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f200a8f8640 1 -- start start 2024-03-14T12:14:28.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f200a8f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040d0760 0x7f20040d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f200a8f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20040b7720 0x7f20040b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f200a8f8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20040d1080 con 0x7f20040cf370 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f200a8f8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20040cf8c0 con 0x7f20040b7720 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f200a8f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20040cfba0 con 0x7f20040d0760 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f2008e95640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20040b7720 0x7f20040b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f2008e95640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20040b7720 0x7f20040b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39616/0 (socket says 172.21.15.82:39616) 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.808+0000 7f2008e95640 1 -- 172.21.15.82:0/577907757 learned_addr learned my addr 172.21.15.82:0/577907757 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.809+0000 7f2003fff640 1 --2- 172.21.15.82:0/577907757 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040d0760 0x7f20040d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.809+0000 7f2008e95640 1 -- 172.21.15.82:0/577907757 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040d0760 msgr2=0x7f20040d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.809+0000 7f2008e95640 1 --2- 172.21.15.82:0/577907757 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040d0760 0x7f20040d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.809+0000 7f2008e95640 1 -- 172.21.15.82:0/577907757 >> v1:172.21.15.82:6789/0 conn(0x7f20040cf370 legacy=0x7f20040d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.809+0000 7f2008e95640 1 -- 172.21.15.82:0/577907757 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20040b6c40 con 0x7f20040b7720 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.809+0000 7f2008e95640 1 --2- 172.21.15.82:0/577907757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040b7720 0x7f20040b7b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009e50 tx=0x7f1ffc02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3876d5c1224cec server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.809+0000 7f2002ffd640 1 -- 172.21.15.82:0/577907757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc02cb90 con 0x7f20040b7720 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f2002ffd640 1 -- 172.21.15.82:0/577907757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc02cd30 con 0x7f20040b7720 2024-03-14T12:14:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f2002ffd640 1 -- 172.21.15.82:0/577907757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc034610 con 0x7f20040b7720 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 -- 172.21.15.82:0/577907757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040b7720 msgr2=0x7f20040b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 --2- 172.21.15.82:0/577907757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040b7720 0x7f20040b7b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009e50 tx=0x7f1ffc02bff0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 -- 172.21.15.82:0/577907757 shutdown_connections 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 --2- 172.21.15.82:0/577907757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040b7720 0x7f20040b7b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 --2- 172.21.15.82:0/577907757 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040d0760 0x7f20040d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 -- 172.21.15.82:0/577907757 >> 172.21.15.82:0/577907757 conn(0x7f200409d030 msgr2=0x7f20040d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 -- 172.21.15.82:0/577907757 shutdown_connections 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.810+0000 7f200a8f8640 1 -- 172.21.15.82:0/577907757 wait complete. 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.811+0000 7f200a8f8640 1 Processor -- start 2024-03-14T12:14:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.811+0000 7f200a8f8640 1 -- start start 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f200a8f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20040b7720 0x7f20041540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f200a8f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040cf370 0x7f20040cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f200a8f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040d0760 0x7f20040c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f200a8f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20041617a0 con 0x7f20040d0760 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f200a8f8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20040b7200 con 0x7f20040b7720 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f200a8f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2004161920 con 0x7f20040cf370 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f2008e95640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040d0760 0x7f20040c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f2008e95640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040d0760 0x7f20040c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39632/0 (socket says 172.21.15.82:39632) 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f2008e95640 1 -- 172.21.15.82:0/1273273413 learned_addr learned my addr 172.21.15.82:0/1273273413 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f20037fe640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040cf370 0x7f20040cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.812+0000 7f2003fff640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20040b7720 0x7f20041540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2008e95640 1 -- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040cf370 msgr2=0x7f20040cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2008e95640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040cf370 0x7f20040cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2008e95640 1 -- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20040b7720 msgr2=0x7f20041540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2008e95640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20040b7720 0x7f20041540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2008e95640 1 -- 172.21.15.82:0/1273273413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20040cd660 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f20037fe640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040cf370 0x7f20040cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2003fff640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20040b7720 0x7f20041540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2008e95640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040d0760 0x7f20040c9b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009e50 tx=0x7f1ffc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2000ff9640 1 -- 172.21.15.82:0/1273273413 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc043020 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2000ff9640 1 -- 172.21.15.82:0/1273273413 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc03e450 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20040ccdc0 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f2000ff9640 1 -- 172.21.15.82:0/1273273413 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc009510 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.813+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20040cd380 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.814+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200404f050 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.814+0000 7f2000ff9640 1 -- 172.21.15.82:0/1273273413 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1ffc046020 con 0x7f20040d0760 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.815+0000 7f2000ff9640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ff006bbd0 0x7f1ff006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.815+0000 7f2000ff9640 1 -- 172.21.15.82:0/1273273413 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ffc0ad190 con 0x7f20040d0760 2024-03-14T12:14:28.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.815+0000 7f2003fff640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ff006bbd0 0x7f1ff006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:28.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.815+0000 7f2003fff640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ff006bbd0 0x7f1ff006e090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4006fd0 tx=0x7f1ff4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:28.819+0000 7f2000ff9640 1 -- 172.21.15.82:0/1273273413 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1ffc04d030 con 0x7f20040d0760 2024-03-14T12:14:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.005+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2004073000 con 0x7f20040d0760 2024-03-14T12:14:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.005+0000 7f2000ff9640 1 -- 172.21.15.82:0/1273273413 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2004073000 con 0x7f20040d0760 2024-03-14T12:14:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:14:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ff006bbd0 msgr2=0x7f1ff006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ff006bbd0 0x7f1ff006e090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4006fd0 tx=0x7f1ff4008040 comp rx=0 tx=0).stop 2024-03-14T12:14:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040d0760 msgr2=0x7f20040c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040d0760 0x7f20040c9b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009e50 tx=0x7f1ffc004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 shutdown_connections 2024-03-14T12:14:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20040d0760 0x7f20040c9b50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ff006bbd0 0x7f1ff006e090 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20040cf370 0x7f20040cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 --2- 172.21.15.82:0/1273273413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20040b7720 0x7f20041540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 >> 172.21.15.82:0/1273273413 conn(0x7f200409d030 msgr2=0x7f20040d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 shutdown_connections 2024-03-14T12:14:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.007+0000 7f200a8f8640 1 -- 172.21.15.82:0/1273273413 wait complete. 2024-03-14T12:14:29.017 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-14T12:14:29.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:29.017 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-14T12:14:29.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:14:29.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:29.050 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-14T12:14:29.109 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-14T12:14:29.109 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:29.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1cd52a4640 1 Processor -- start 2024-03-14T12:14:29.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1cd52a4640 1 -- start start 2024-03-14T12:14:29.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1cd52a4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1cd00d0760 0x7f1cd00d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1cd52a4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cd00b7720 0x7f1cd00b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1cd52a4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd00d1080 con 0x7f1cd00cf370 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1cd52a4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd00cf8c0 con 0x7f1cd00b7720 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1cd52a4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd00cfba0 con 0x7f1cd00d0760 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1ccf7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cd00b7720 0x7f1cd00b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1ccf7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cd00b7720 0x7f1cd00b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39638/0 (socket says 172.21.15.82:39638) 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.211+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/2914460274 learned_addr learned my addr 172.21.15.82:0/2914460274 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.212+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/2914460274 >> v2:172.21.15.164:3300/0 conn(0x7f1cd00d0760 msgr2=0x7f1cd00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.212+0000 7f1ccf7fe640 1 --2- 172.21.15.82:0/2914460274 >> v2:172.21.15.164:3300/0 conn(0x7f1cd00d0760 0x7f1cd00d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.212+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/2914460274 >> v1:172.21.15.82:6789/0 conn(0x7f1cd00cf370 legacy=0x7f1cd00d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.212+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/2914460274 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd00b6c40 con 0x7f1cd00b7720 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.212+0000 7f1ccf7fe640 1 --2- 172.21.15.82:0/2914460274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd00b7b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cd008ca90 tx=0x7f1cc802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907e97c38c4348c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1ccdffb640 1 -- 172.21.15.82:0/2914460274 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc802cbb0 con 0x7f1cd00b7720 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1ccdffb640 1 -- 172.21.15.82:0/2914460274 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc802cd50 con 0x7f1cd00b7720 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1ccdffb640 1 -- 172.21.15.82:0/2914460274 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8034610 con 0x7f1cd00b7720 2024-03-14T12:14:29.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 -- 172.21.15.82:0/2914460274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 msgr2=0x7f1cd00b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/2914460274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd00b7b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cd008ca90 tx=0x7f1cc802c010 comp rx=0 tx=0).stop 2024-03-14T12:14:29.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 -- 172.21.15.82:0/2914460274 shutdown_connections 2024-03-14T12:14:29.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/2914460274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd00b7b50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/2914460274 >> v2:172.21.15.164:3300/0 conn(0x7f1cd00d0760 0x7f1cd00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 -- 172.21.15.82:0/2914460274 >> 172.21.15.82:0/2914460274 conn(0x7f1cd009d030 msgr2=0x7f1cd00d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:29.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 -- 172.21.15.82:0/2914460274 shutdown_connections 2024-03-14T12:14:29.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.213+0000 7f1cd52a4640 1 -- 172.21.15.82:0/2914460274 wait complete. 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.214+0000 7f1cd52a4640 1 Processor -- start 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cd52a4640 1 -- start start 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cd52a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd0155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cd52a4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1cd00cf370 0x7f1cd00cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cd52a4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cd00d0760 0x7f1cd00cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cd52a4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01617a0 con 0x7f1cd00b7720 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cd52a4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00b7200 con 0x7f1cd00d0760 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cd52a4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0161920 con 0x7f1cd00cf370 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cceffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd0155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cceffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd0155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39648/0 (socket says 172.21.15.82:39648) 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1cceffd640 1 -- 172.21.15.82:0/1775459791 learned_addr learned my addr 172.21.15.82:0/1775459791 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:29.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.215+0000 7f1ccf7fe640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cd00d0760 0x7f1cd00cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cce7fc640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1cd00cf370 0x7f1cd00cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cceffd640 1 -- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1cd00cf370 msgr2=0x7f1cd00cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cceffd640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1cd00cf370 0x7f1cd00cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cceffd640 1 -- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cd00d0760 msgr2=0x7f1cd00cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cceffd640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cd00d0760 0x7f1cd00cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cceffd640 1 -- 172.21.15.82:0/1775459791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd01563a0 con 0x7f1cd00b7720 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1ccf7fe640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cd00d0760 0x7f1cd00cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cceffd640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd0155d60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b410 tx=0x7f1cc400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cb77fe640 1 -- 172.21.15.82:0/1775459791 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4015020 con 0x7f1cd00b7720 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cb77fe640 1 -- 172.21.15.82:0/1775459791 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc4011850 con 0x7f1cd00b7720 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd0156680 con 0x7f1cd00b7720 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cb77fe640 1 -- 172.21.15.82:0/1775459791 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4011c40 con 0x7f1cd00b7720 2024-03-14T12:14:29.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.216+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0156c40 con 0x7f1cd00b7720 2024-03-14T12:14:29.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.217+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd004f050 con 0x7f1cd00b7720 2024-03-14T12:14:29.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.217+0000 7f1cb77fe640 1 -- 172.21.15.82:0/1775459791 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1cc4018020 con 0x7f1cd00b7720 2024-03-14T12:14:29.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.218+0000 7f1cb77fe640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1cbc06bc00 0x7f1cbc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.218+0000 7f1cce7fc640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1cbc06bc00 0x7f1cbc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.218+0000 7f1cb77fe640 1 -- 172.21.15.82:0/1775459791 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cc408aa20 con 0x7f1cd00b7720 2024-03-14T12:14:29.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.218+0000 7f1cce7fc640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1cbc06bc00 0x7f1cbc06e0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00cad80 tx=0x7f1cc0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.222+0000 7f1cb77fe640 1 -- 172.21.15.82:0/1775459791 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1cc401f020 con 0x7f1cd00b7720 2024-03-14T12:14:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.429+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cd00d0540 con 0x7f1cd00b7720 2024-03-14T12:14:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.429+0000 7f1cb77fe640 1 -- 172.21.15.82:0/1775459791 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f1cc40172c0 con 0x7f1cd00b7720 2024-03-14T12:14:29.429 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:29.429 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:14:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1cbc06bc00 msgr2=0x7f1cbc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1cbc06bc00 0x7f1cbc06e0c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00cad80 tx=0x7f1cc0009040 comp rx=0 tx=0).stop 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 msgr2=0x7f1cd0155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd0155d60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b410 tx=0x7f1cc400b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 shutdown_connections 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cd00d0760 0x7f1cd00cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1cbc06bc00 0x7f1cbc06e0c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1cd00cf370 0x7f1cd00cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 --2- 172.21.15.82:0/1775459791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd00b7720 0x7f1cd0155d60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 >> 172.21.15.82:0/1775459791 conn(0x7f1cd009d030 msgr2=0x7f1cd00b9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 shutdown_connections 2024-03-14T12:14:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.431+0000 7f1cd52a4640 1 -- 172.21.15.82:0/1775459791 wait complete. 2024-03-14T12:14:29.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:29.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.543+0000 7f06c2638640 1 Processor -- start 2024-03-14T12:14:29.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.543+0000 7f06c2638640 1 -- start start 2024-03-14T12:14:29.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06c2638640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc1551d0 0x7f06bc0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06c2638640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc0b60f0 con 0x7f06bc0b5bb0 2024-03-14T12:14:29.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06c2638640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc0b63d0 con 0x7f06bc162190 2024-03-14T12:14:29.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06c2638640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0b6d00 con 0x7f06bc1551d0 2024-03-14T12:14:29.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06bb7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f06bc0b5bb0 0x7f06bc161a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51006/0 (socket says 172.21.15.82:51006) 2024-03-14T12:14:29.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06bb7fe640 1 -- 172.21.15.82:0/2384040236 learned_addr learned my addr 172.21.15.82:0/2384040236 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:29.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326689519 0 0) 0x7f06bc0b60f0 con 0x7f06bc0b5bb0 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a4003520 con 0x7f06bc0b5bb0 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06bbfff640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc1551d0 0x7f06bc0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4540348 0 0) 0x7f06a4003520 con 0x7f06bc0b5bb0 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc1551d0 msgr2=0x7f06bc0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc1551d0 0x7f06bc0b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 >> v1:172.21.15.164:6789/0 conn(0x7f06bc162190 legacy=0x7f06bc06f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0b66b0 con 0x7f06bc0b5bb0 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f06b0001670 con 0x7f06bc0b5bb0 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 >> v1:172.21.15.82:6789/0 conn(0x7f06bc0b5bb0 legacy=0x7f06bc161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06a40039e0 0x7f06a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06a40051c0 0x7f06a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a400c090 con 0x7f06a40051c0 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.544+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a400c210 con 0x7f06a4004800 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a400c410 con 0x7f06a40039e0 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06bbfff640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc1551d0 0x7f06bc0b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06c0bd5640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06a40051c0 0x7f06a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06bb7fe640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06bbfff640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06a40039e0 0x7f06a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06c0bd5640 1 -- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06a40039e0 msgr2=0x7f06a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06c0bd5640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06a40039e0 0x7f06a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06c0bd5640 1 -- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a4004800 msgr2=0x7f06a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06c0bd5640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06c0bd5640 1 -- 172.21.15.82:0/2384040236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4003520 con 0x7f06a40051c0 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.545+0000 7f06bb7fe640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a4004800 0x7f06a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c0bd5640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06a40051c0 0x7f06a400b9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f06b4001e50 tx=0x7f06a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276adc6a93aee204 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4001a70 con 0x7f06a40051c0 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b4003b70 con 0x7f06a40051c0 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06baffd640 1 -- 172.21.15.82:0/2384040236 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b40050d0 con 0x7f06a40051c0 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c2638640 1 -- 172.21.15.82:0/2384040236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06a40051c0 msgr2=0x7f06a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c2638640 1 --2- 172.21.15.82:0/2384040236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06a40051c0 0x7f06a400b9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f06b4001e50 tx=0x7f06a4005830 comp rx=0 tx=0).stop 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c0bd5640 1 -- 172.21.15.82:0/2384040236 reap_dead start 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c2638640 1 -- 172.21.15.82:0/2384040236 shutdown_connections 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c2638640 1 -- 172.21.15.82:0/2384040236 >> 172.21.15.82:0/2384040236 conn(0x7f06bc072ee0 msgr2=0x7f06bc09a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c2638640 1 -- 172.21.15.82:0/2384040236 shutdown_connections 2024-03-14T12:14:29.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.546+0000 7f06c2638640 1 -- 172.21.15.82:0/2384040236 wait complete. 2024-03-14T12:14:29.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 Processor -- start 2024-03-14T12:14:29.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 -- start start 2024-03-14T12:14:29.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06bc0b3fd0 0x7f06bc0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06bc0b3000 0x7f06bc1e7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc0b43b0 0x7f06bc14c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc160cc0 con 0x7f06bc0b3000 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc160b40 con 0x7f06bc0b3fd0 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06c2638640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc160e40 con 0x7f06bc0b43b0 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06bb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06bc0b3000 0x7f06bc1e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06bb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06bc0b3000 0x7f06bc1e7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39678/0 (socket says 172.21.15.82:39678) 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.548+0000 7f06bb7fe640 1 -- 172.21.15.82:0/150814660 learned_addr learned my addr 172.21.15.82:0/150814660 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bb7fe640 1 -- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc0b43b0 msgr2=0x7f06bc14c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bb7fe640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc0b43b0 0x7f06bc14c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bb7fe640 1 -- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06bc0b3fd0 msgr2=0x7f06bc0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bbfff640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06bc0b3fd0 0x7f06bc0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bb7fe640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06bc0b3fd0 0x7f06bc0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bb7fe640 1 -- 172.21.15.82:0/150814660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0b1f60 con 0x7f06bc0b3000 2024-03-14T12:14:29.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bbfff640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06bc0b3fd0 0x7f06bc0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06bb7fe640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06bc0b3000 0x7f06bc1e7850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f06b000e840 tx=0x7f06b000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06b8ff9640 1 -- 172.21.15.82:0/150814660 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000c8d0 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06b8ff9640 1 -- 172.21.15.82:0/150814660 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b0018070 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc0b2240 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06b8ff9640 1 -- 172.21.15.82:0/150814660 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000cc70 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.549+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc0b2800 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.550+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc04f050 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.551+0000 7f06b8ff9640 1 -- 172.21.15.82:0/150814660 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f06b001d460 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.551+0000 7f06b8ff9640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f06a806bbd0 0x7f06a806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.551+0000 7f06b8ff9640 1 -- 172.21.15.82:0/150814660 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06b008a9e0 con 0x7f06bc0b3000 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.551+0000 7f06bbfff640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f06a806bbd0 0x7f06a806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.551+0000 7f06bbfff640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f06a806bbd0 0x7f06a806e090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.555+0000 7f06b8ff9640 1 -- 172.21.15.82:0/150814660 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f06b00594d0 con 0x7f06bc0b3000 2024-03-14T12:14:29.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.730+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06bc0b6ff0 con 0x7f06bc0b3000 2024-03-14T12:14:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.731+0000 7f06b8ff9640 1 -- 172.21.15.82:0/150814660 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f06b0058dd0 con 0x7f06bc0b3000 2024-03-14T12:14:29.730 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:29.730 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f06a806bbd0 msgr2=0x7f06a806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f06a806bbd0 0x7f06a806e090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac008040 comp rx=0 tx=0).stop 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06bc0b3000 msgr2=0x7f06bc1e7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06bc0b3000 0x7f06bc1e7850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f06b000e840 tx=0x7f06b000ed00 comp rx=0 tx=0).stop 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 shutdown_connections 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f06a806bbd0 0x7f06a806e090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06bc0b43b0 0x7f06bc14c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06bc0b3000 0x7f06bc1e7850 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 --2- 172.21.15.82:0/150814660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06bc0b3fd0 0x7f06bc0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 >> 172.21.15.82:0/150814660 conn(0x7f06bc072ee0 msgr2=0x7f06bc0b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:29.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 shutdown_connections 2024-03-14T12:14:29.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.733+0000 7f06c2638640 1 -- 172.21.15.82:0/150814660 wait complete. 2024-03-14T12:14:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:13:49.489722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:13:49.874257+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:29.744 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:29.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.848+0000 7f31b15f1640 1 Processor -- start 2024-03-14T12:14:29.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.848+0000 7f31b15f1640 1 -- start start 2024-03-14T12:14:29.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31b15f1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31ac15dff0 0x7f31ac0d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31b15f1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ac0d25a0 con 0x7f31ac0d12a0 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31b15f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ac0d2880 con 0x7f31ac0d0e10 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31b15f1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ac0d2350 con 0x7f31ac15dff0 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31aa7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f31ac0d12a0 0x7f31ac0d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51020/0 (socket says 172.21.15.82:51020) 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31aa7fc640 1 -- 172.21.15.82:0/3947094171 learned_addr learned my addr 172.21.15.82:0/3947094171 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31aaffd640 1 --2- 172.21.15.82:0/3947094171 >> v2:172.21.15.164:3300/0 conn(0x7f31ac15dff0 0x7f31ac0d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475494556 0 0) 0x7f31ac0d25a0 con 0x7f31ac0d12a0 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3198003520 con 0x7f31ac0d12a0 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1497336211 0 0) 0x7f3198003520 con 0x7f31ac0d12a0 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 >> v2:172.21.15.164:3300/0 conn(0x7f31ac15dff0 msgr2=0x7f31ac0d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 --2- 172.21.15.82:0/3947094171 >> v2:172.21.15.164:3300/0 conn(0x7f31ac15dff0 0x7f31ac0d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 >> v1:172.21.15.164:6789/0 conn(0x7f31ac0d0e10 legacy=0x7f31ac0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac0d2b60 con 0x7f31ac0d12a0 2024-03-14T12:14:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f3194001670 con 0x7f31ac0d12a0 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.849+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 >> v1:172.21.15.82:6789/0 conn(0x7f31ac0d12a0 legacy=0x7f31ac0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31a9ffb640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31980039b0 0x7f3198003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31a9ffb640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3198004800 0x7f3198004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31a9ffb640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31980051c0 0x7f319800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac0d2350 con 0x7f3198004800 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319800c090 con 0x7f31980039b0 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319800c290 con 0x7f31980051c0 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31aa7fc640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3198004800 0x7f3198004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31aaffd640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31980039b0 0x7f3198003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31ab7fe640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31980051c0 0x7f319800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31aa7fc640 1 -- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31980051c0 msgr2=0x7f319800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31aa7fc640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31980051c0 0x7f319800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31aa7fc640 1 -- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31980039b0 msgr2=0x7f3198003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31aa7fc640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31980039b0 0x7f3198003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31aa7fc640 1 -- 172.21.15.82:0/3947094171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198003520 con 0x7f3198004800 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.850+0000 7f31ab7fe640 1 -- 172.21.15.82:0/3947094171 reap_dead start 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31aa7fc640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3198004800 0x7f3198004c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f319400c210 tx=0x7f3198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3c72d7f8411c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194007cf0 con 0x7f3198004800 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194030070 con 0x7f3198004800 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31a9ffb640 1 -- 172.21.15.82:0/3947094171 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194005d30 con 0x7f3198004800 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31b15f1640 1 -- 172.21.15.82:0/3947094171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3198004800 msgr2=0x7f3198004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31b15f1640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3198004800 0x7f3198004c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f319400c210 tx=0x7f3198005830 comp rx=0 tx=0).stop 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31b15f1640 1 -- 172.21.15.82:0/3947094171 shutdown_connections 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31b15f1640 1 --2- 172.21.15.82:0/3947094171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3198004800 0x7f3198004c80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31b15f1640 1 -- 172.21.15.82:0/3947094171 >> 172.21.15.82:0/3947094171 conn(0x7f31ac09d0d0 msgr2=0x7f31ac0d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31b15f1640 1 -- 172.21.15.82:0/3947094171 shutdown_connections 2024-03-14T12:14:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.851+0000 7f31b15f1640 1 -- 172.21.15.82:0/3947094171 wait complete. 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 Processor -- start 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 -- start start 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3198004800 0x7f31ac0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31ac1606e0 0x7f31ac159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31ac0b4040 0x7f31ac15f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac0d0880 con 0x7f31ac1606e0 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac0d0700 con 0x7f3198004800 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31b15f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac0d0a00 con 0x7f31ac0b4040 2024-03-14T12:14:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31aa7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31ac1606e0 0x7f31ac159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31aa7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31ac1606e0 0x7f31ac159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39688/0 (socket says 172.21.15.82:39688) 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.853+0000 7f31aa7fc640 1 -- 172.21.15.82:0/2787085234 learned_addr learned my addr 172.21.15.82:0/2787085234 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31aaffd640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3198004800 0x7f31ac0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31ab7fe640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31ac0b4040 0x7f31ac15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31aa7fc640 1 -- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31ac0b4040 msgr2=0x7f31ac15f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31aa7fc640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31ac0b4040 0x7f31ac15f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31aa7fc640 1 -- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3198004800 msgr2=0x7f31ac0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31aa7fc640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3198004800 0x7f31ac0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31aa7fc640 1 -- 172.21.15.82:0/2787085234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac0b2fa0 con 0x7f31ac1606e0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31aa7fc640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31ac1606e0 0x7f31ac159e20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f319400c300 tx=0x7f31ac0cae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31a37fe640 1 -- 172.21.15.82:0/2787085234 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194001c30 con 0x7f31ac1606e0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31a37fe640 1 -- 172.21.15.82:0/2787085234 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194004bf0 con 0x7f31ac1606e0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ac0b3280 con 0x7f31ac1606e0 2024-03-14T12:14:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31a37fe640 1 -- 172.21.15.82:0/2787085234 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194007070 con 0x7f31ac1606e0 2024-03-14T12:14:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.854+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ac0b3840 con 0x7f31ac1606e0 2024-03-14T12:14:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.855+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ac04f050 con 0x7f31ac1606e0 2024-03-14T12:14:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.855+0000 7f31a37fe640 1 -- 172.21.15.82:0/2787085234 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3194048020 con 0x7f31ac1606e0 2024-03-14T12:14:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.856+0000 7f31a37fe640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f318c06bbd0 0x7f318c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.856+0000 7f31aaffd640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f318c06bbd0 0x7f318c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.856+0000 7f31a37fe640 1 -- 172.21.15.82:0/2787085234 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31940abcb0 con 0x7f31ac1606e0 2024-03-14T12:14:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.856+0000 7f31aaffd640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f318c06bbd0 0x7f318c06e090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f319c007520 tx=0x7f319c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:29.860+0000 7f31a37fe640 1 -- 172.21.15.82:0/2787085234 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f319407a910 con 0x7f31ac1606e0 2024-03-14T12:14:30.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.031+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31ac0b9be0 con 0x7f31ac1606e0 2024-03-14T12:14:30.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.031+0000 7f31a37fe640 1 -- 172.21.15.82:0/2787085234 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f319407a210 con 0x7f31ac1606e0 2024-03-14T12:14:30.031 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.033+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f318c06bbd0 msgr2=0x7f318c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f318c06bbd0 0x7f318c06e090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f319c007520 tx=0x7f319c00b040 comp rx=0 tx=0).stop 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31ac1606e0 msgr2=0x7f31ac159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31ac1606e0 0x7f31ac159e20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f319400c300 tx=0x7f31ac0cae40 comp rx=0 tx=0).stop 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 shutdown_connections 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f318c06bbd0 0x7f318c06e090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31ac0b4040 0x7f31ac15f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31ac1606e0 0x7f31ac159e20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 --2- 172.21.15.82:0/2787085234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3198004800 0x7f31ac0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 >> 172.21.15.82:0/2787085234 conn(0x7f31ac09d0d0 msgr2=0x7f31ac0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:30.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 shutdown_connections 2024-03-14T12:14:30.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.034+0000 7f31b15f1640 1 -- 172.21.15.82:0/2787085234 wait complete. 2024-03-14T12:14:30.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.149+0000 7fa5997e1640 1 Processor -- start 2024-03-14T12:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.149+0000 7fa5997e1640 1 -- start start 2024-03-14T12:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5997e1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa5940c7400 0x7fa5941623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5997e1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa5940afa80 0x7fa5940aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5997e1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59415e3d0 con 0x7fa59415dff0 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5997e1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940d18a0 con 0x7fa5940c7400 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5997e1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa594161be0 con 0x7fa5940afa80 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5927fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa5940c7400 0x7fa5941623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5927fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa5940c7400 0x7fa5941623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39694/0 (socket says 172.21.15.82:39694) 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5927fc640 1 -- 172.21.15.82:0/49885840 learned_addr learned my addr 172.21.15.82:0/49885840 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5937fe640 1 --2- 172.21.15.82:0/49885840 >> v2:172.21.15.164:3300/0 conn(0x7fa5940afa80 0x7fa5940aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5927fc640 1 -- 172.21.15.82:0/49885840 >> v2:172.21.15.164:3300/0 conn(0x7fa5940afa80 msgr2=0x7fa5940aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5927fc640 1 --2- 172.21.15.82:0/49885840 >> v2:172.21.15.164:3300/0 conn(0x7fa5940afa80 0x7fa5940aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5927fc640 1 -- 172.21.15.82:0/49885840 >> v1:172.21.15.164:6789/0 conn(0x7fa59415dff0 legacy=0x7fa5940d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.150+0000 7fa5927fc640 1 -- 172.21.15.82:0/49885840 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940ce6d0 con 0x7fa5940c7400 2024-03-14T12:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa5927fc640 1 --2- 172.21.15.82:0/49885840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940c7400 0x7fa5941623a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa5940d0aa0 tx=0x7fa58802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfd450f7b8d445e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa591ffb640 1 -- 172.21.15.82:0/49885840 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588035020 con 0x7fa5940c7400 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa591ffb640 1 -- 172.21.15.82:0/49885840 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa588031870 con 0x7fa5940c7400 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa591ffb640 1 -- 172.21.15.82:0/49885840 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588031c40 con 0x7fa5940c7400 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa5997e1640 1 -- 172.21.15.82:0/49885840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940c7400 msgr2=0x7fa5941623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa5997e1640 1 --2- 172.21.15.82:0/49885840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940c7400 0x7fa5941623a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa5940d0aa0 tx=0x7fa58802b910 comp rx=0 tx=0).stop 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa5997e1640 1 -- 172.21.15.82:0/49885840 shutdown_connections 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa5997e1640 1 --2- 172.21.15.82:0/49885840 >> v2:172.21.15.164:3300/0 conn(0x7fa5940afa80 0x7fa5940aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa5997e1640 1 --2- 172.21.15.82:0/49885840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940c7400 0x7fa5941623a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.151+0000 7fa5997e1640 1 -- 172.21.15.82:0/49885840 >> 172.21.15.82:0/49885840 conn(0x7fa5940d0b00 msgr2=0x7fa5940d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.152+0000 7fa5997e1640 1 -- 172.21.15.82:0/49885840 shutdown_connections 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.152+0000 7fa5997e1640 1 -- 172.21.15.82:0/49885840 wait complete. 2024-03-14T12:14:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.153+0000 7fa5997e1640 1 Processor -- start 2024-03-14T12:14:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.153+0000 7fa5997e1640 1 -- start start 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.153+0000 7fa5997e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940afa80 0x7fa5940af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa5997e1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5940c7400 0x7fa59406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa5997e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59415dff0 0x7fa59406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa5997e1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940cec90 con 0x7fa5940afa80 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa5997e1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594161e80 con 0x7fa59415dff0 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa5997e1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940cee10 con 0x7fa5940c7400 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940afa80 0x7fa5940af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940afa80 0x7fa5940af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39706/0 (socket says 172.21.15.82:39706) 2024-03-14T12:14:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 -- 172.21.15.82:0/1444264237 learned_addr learned my addr 172.21.15.82:0/1444264237 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 -- 172.21.15.82:0/1444264237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5940c7400 msgr2=0x7fa59406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5940c7400 0x7fa59406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 -- 172.21.15.82:0/1444264237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59415dff0 msgr2=0x7fa59406ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59415dff0 0x7fa59406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 -- 172.21.15.82:0/1444264237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59415c310 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa592ffd640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940afa80 0x7fa5940af350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00ba80 tx=0x7fa58c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa57b7fe640 1 -- 172.21.15.82:0/1444264237 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c019070 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa57b7fe640 1 -- 172.21.15.82:0/1444264237 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00cc90 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59415c5f0 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa57b7fe640 1 -- 172.21.15.82:0/1444264237 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c014680 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.154+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59415cbb0 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.155+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59404f050 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.156+0000 7fa57b7fe640 1 -- 172.21.15.82:0/1444264237 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa58c007370 con 0x7fa5940afa80 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.156+0000 7fa57b7fe640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa58006bc00 0x7fa58006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.156+0000 7fa57b7fe640 1 -- 172.21.15.82:0/1444264237 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa58c08a740 con 0x7fa5940afa80 2024-03-14T12:14:30.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.156+0000 7fa5927fc640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa58006bc00 0x7fa58006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.156+0000 7fa5927fc640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa58006bc00 0x7fa58006e0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa588036000 tx=0x7fa588030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.160+0000 7fa57b7fe640 1 -- 172.21.15.82:0/1444264237 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa58c059230 con 0x7fa5940afa80 2024-03-14T12:14:30.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.330+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa59406b9f0 con 0x7fa5940afa80 2024-03-14T12:14:30.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.330+0000 7fa57b7fe640 1 -- 172.21.15.82:0/1444264237 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa58c058b30 con 0x7fa5940afa80 2024-03-14T12:14:30.330 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa58006bc00 msgr2=0x7fa58006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa58006bc00 0x7fa58006e0c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa588036000 tx=0x7fa588030000 comp rx=0 tx=0).stop 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940afa80 msgr2=0x7fa5940af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940afa80 0x7fa5940af350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00ba80 tx=0x7fa58c00bf40 comp rx=0 tx=0).stop 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 shutdown_connections 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59415dff0 0x7fa59406ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa58006bc00 0x7fa58006e0c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5940c7400 0x7fa59406e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 --2- 172.21.15.82:0/1444264237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5940afa80 0x7fa5940af350 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.332+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 >> 172.21.15.82:0/1444264237 conn(0x7fa5940d0b00 msgr2=0x7fa59406dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.333+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 shutdown_connections 2024-03-14T12:14:30.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.333+0000 7fa5997e1640 1 -- 172.21.15.82:0/1444264237 wait complete. 2024-03-14T12:14:30.342 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:14:30.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.447+0000 7f20ff02b640 1 Processor -- start 2024-03-14T12:14:30.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.447+0000 7f20ff02b640 1 -- start start 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.447+0000 7f20ff02b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f20f81446f0 0x7f20f815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.447+0000 7f20ff02b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f80d0210 0x7f20f815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.447+0000 7f20ff02b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f80b34e0 con 0x7f20f80cfe30 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.447+0000 7f20ff02b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f815f1f0 con 0x7f20f81446f0 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.447+0000 7f20ff02b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f815f4a0 con 0x7f20f80d0210 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20effff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f20f81446f0 0x7f20f815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20fd5c8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f80d0210 0x7f20f815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20fcdc7640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f20f80cfe30 0x7f20f8143fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:51390/0 (socket says 172.21.15.82:51390) 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20fcdc7640 1 -- 172.21.15.82:0/3928686398 learned_addr learned my addr 172.21.15.82:0/3928686398 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:30.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20effff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f20f81446f0 0x7f20f815e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:49798/0 (socket says 172.21.15.82:49798) 2024-03-14T12:14:30.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20fd5c8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f80d0210 0x7f20f815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:38868/0 (socket says 172.21.15.82:38868) 2024-03-14T12:14:30.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20effff640 1 -- 172.21.15.82:0/3928686398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f80d0210 msgr2=0x7f20f815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:30.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20effff640 1 --2- 172.21.15.82:0/3928686398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f80d0210 0x7f20f815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20effff640 1 -- 172.21.15.82:0/3928686398 >> v1:172.21.15.164:6789/0 conn(0x7f20f80cfe30 legacy=0x7f20f8143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20ef7fe640 1 -- 172.21.15.82:0/3928686398 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335175158 0 0) 0x7f20f80b34e0 con 0x7f20f80cfe30 2024-03-14T12:14:30.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.448+0000 7f20effff640 1 -- 172.21.15.82:0/3928686398 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80b3780 con 0x7f20f81446f0 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.449+0000 7f20effff640 1 --2- 172.21.15.82:0/3928686398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f81446f0 0x7f20f815e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097a0 tx=0x7f20e802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe9a7a90a56f85b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.449+0000 7f20ef7fe640 1 -- 172.21.15.82:0/3928686398 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8035020 con 0x7f20f81446f0 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.449+0000 7f20ef7fe640 1 -- 172.21.15.82:0/3928686398 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8031830 con 0x7f20f81446f0 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.449+0000 7f20ef7fe640 1 -- 172.21.15.82:0/3928686398 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8031c00 con 0x7f20f81446f0 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 -- 172.21.15.82:0/3928686398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f81446f0 msgr2=0x7f20f815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 --2- 172.21.15.82:0/3928686398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f81446f0 0x7f20f815e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097a0 tx=0x7f20e802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 -- 172.21.15.82:0/3928686398 shutdown_connections 2024-03-14T12:14:30.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 --2- 172.21.15.82:0/3928686398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f80d0210 0x7f20f815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 --2- 172.21.15.82:0/3928686398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f81446f0 0x7f20f815e770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 -- 172.21.15.82:0/3928686398 >> 172.21.15.82:0/3928686398 conn(0x7f20f806aef0 msgr2=0x7f20f806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:30.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 -- 172.21.15.82:0/3928686398 shutdown_connections 2024-03-14T12:14:30.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.450+0000 7f20ff02b640 1 -- 172.21.15.82:0/3928686398 wait complete. 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.451+0000 7f20ff02b640 1 Processor -- start 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.451+0000 7f20ff02b640 1 -- start start 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f80cfe30 0x7f20f813d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f80d0210 0x7f20f81599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f81446f0 0x7f20f8159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b4260 con 0x7f20f80d0210 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b3d40 con 0x7f20f80cfe30 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b43e0 con 0x7f20f81446f0 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f80d0210 0x7f20f81599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f80d0210 0x7f20f81599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39722/0 (socket says 172.21.15.82:39722) 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 -- 172.21.15.82:0/1018657567 learned_addr learned my addr 172.21.15.82:0/1018657567 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20fcdc7640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f80cfe30 0x7f20f813d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20fd5c8640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f81446f0 0x7f20f8159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 -- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f81446f0 msgr2=0x7f20f8159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f81446f0 0x7f20f8159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 -- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f80cfe30 msgr2=0x7f20f813d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f80cfe30 0x7f20f813d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 -- 172.21.15.82:0/1018657567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f81f5d40 con 0x7f20f80d0210 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20effff640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f80d0210 0x7f20f81599a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097a0 tx=0x7f20e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ed7fa640 1 -- 172.21.15.82:0/1018657567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8035040 con 0x7f20f80d0210 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f81f6020 con 0x7f20f80d0210 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ed7fa640 1 -- 172.21.15.82:0/1018657567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8004230 con 0x7f20f80d0210 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ed7fa640 1 -- 172.21.15.82:0/1018657567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e803a640 con 0x7f20f80d0210 2024-03-14T12:14:30.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.452+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f81f65e0 con 0x7f20f80d0210 2024-03-14T12:14:30.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.453+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f804f050 con 0x7f20f80d0210 2024-03-14T12:14:30.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.454+0000 7f20ed7fa640 1 -- 172.21.15.82:0/1018657567 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f20e8038020 con 0x7f20f80d0210 2024-03-14T12:14:30.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.454+0000 7f20ed7fa640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f20e406bbd0 0x7f20e406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.454+0000 7f20ed7fa640 1 -- 172.21.15.82:0/1018657567 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e80ac670 con 0x7f20f80d0210 2024-03-14T12:14:30.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.454+0000 7f20fcdc7640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f20e406bbd0 0x7f20e406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.455+0000 7f20fcdc7640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f20e406bbd0 0x7f20e406e090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20f4000f60 tx=0x7f20f4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.458+0000 7f20ed7fa640 1 -- 172.21.15.82:0/1018657567 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f20e807ab20 con 0x7f20f80d0210 2024-03-14T12:14:30.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.631+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20f806d300 con 0x7f20f80d0210 2024-03-14T12:14:30.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.631+0000 7f20ed7fa640 1 -- 172.21.15.82:0/1018657567 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20e807ebc0 con 0x7f20f80d0210 2024-03-14T12:14:30.631 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:14:30.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f20e406bbd0 msgr2=0x7f20e406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f20e406bbd0 0x7f20e406e090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20f4000f60 tx=0x7f20f4009040 comp rx=0 tx=0).stop 2024-03-14T12:14:30.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f80d0210 msgr2=0x7f20f81599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f80d0210 0x7f20f81599a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097a0 tx=0x7f20e80046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 shutdown_connections 2024-03-14T12:14:30.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f20e406bbd0 0x7f20e406e090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f20f81446f0 0x7f20f8159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f80d0210 0x7f20f81599a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 --2- 172.21.15.82:0/1018657567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20f80cfe30 0x7f20f813d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 >> 172.21.15.82:0/1018657567 conn(0x7f20f806aef0 msgr2=0x7f20f806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:30.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 shutdown_connections 2024-03-14T12:14:30.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.634+0000 7f20ff02b640 1 -- 172.21.15.82:0/1018657567 wait complete. 2024-03-14T12:14:30.643 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:30.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.745+0000 7ff6ec697640 1 Processor -- start 2024-03-14T12:14:30.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.745+0000 7ff6ec697640 1 -- start start 2024-03-14T12:14:30.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6ec697640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff6e40d1810 0x7ff6e406f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6ec697640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 0x7ff6e406b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6ec697640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e406ea90 con 0x7ff6e415bf60 2024-03-14T12:14:30.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6ec697640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e4072000 con 0x7ff6e40d1810 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6ec697640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406ed30 con 0x7ff6e406ad10 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6ea433640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff6e415bf60 0x7ff6e4072490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51026/0 (socket says 172.21.15.82:51026) 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6ea433640 1 -- 172.21.15.82:0/2704150550 learned_addr learned my addr 172.21.15.82:0/2704150550 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9c32640 1 --2- 172.21.15.82:0/2704150550 >> v2:172.21.15.164:3300/0 conn(0x7ff6e40d1810 0x7ff6e406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6eac34640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 0x7ff6e406b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894915839 0 0) 0x7ff6e406ea90 con 0x7ff6e415bf60 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d0003520 con 0x7ff6e415bf60 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2812549033 0 0) 0x7ff6d0003520 con 0x7ff6e415bf60 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 msgr2=0x7ff6e406b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 0x7ff6e406b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 >> v2:172.21.15.164:3300/0 conn(0x7ff6e40d1810 msgr2=0x7ff6e406f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 --2- 172.21.15.82:0/2704150550 >> v2:172.21.15.164:3300/0 conn(0x7ff6e40d1810 0x7ff6e406f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40719b0 con 0x7ff6e415bf60 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7ff6e0001670 con 0x7ff6e415bf60 2024-03-14T12:14:30.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 >> v1:172.21.15.82:6789/0 conn(0x7ff6e415bf60 legacy=0x7ff6e4072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d00039b0 0x7ff6d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d0004800 0x7ff6d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 0x7ff6d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4072000 con 0x7ff6d00039b0 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406ed30 con 0x7ff6d0004800 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.746+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d000b9c0 con 0x7ff6e406ad10 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6ea433640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d00039b0 0x7ff6d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6e9c32640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d0004800 0x7ff6d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6eac34640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 0x7ff6d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6ea433640 1 -- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 msgr2=0x7ff6d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6ea433640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e406ad10 0x7ff6d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6ea433640 1 -- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d0004800 msgr2=0x7ff6d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6ea433640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d0004800 0x7ff6d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6ea433640 1 -- 172.21.15.82:0/2704150550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d0003520 con 0x7ff6d00039b0 2024-03-14T12:14:30.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.747+0000 7ff6ea433640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d00039b0 0x7ff6d0003dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c240 tx=0x7ff6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98d4d7e51829b87 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0007d50 con 0x7ff6d00039b0 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e0030070 con 0x7ff6d00039b0 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6e9431640 1 -- 172.21.15.82:0/2704150550 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e002cda0 con 0x7ff6d00039b0 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6ec697640 1 -- 172.21.15.82:0/2704150550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d00039b0 msgr2=0x7ff6d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6ec697640 1 --2- 172.21.15.82:0/2704150550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d00039b0 0x7ff6d0003dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c240 tx=0x7ff6d0005830 comp rx=0 tx=0).stop 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6eac34640 1 -- 172.21.15.82:0/2704150550 reap_dead start 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6ec697640 1 -- 172.21.15.82:0/2704150550 shutdown_connections 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6ec697640 1 -- 172.21.15.82:0/2704150550 >> 172.21.15.82:0/2704150550 conn(0x7ff6e40d0b60 msgr2=0x7ff6e40d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6ec697640 1 -- 172.21.15.82:0/2704150550 shutdown_connections 2024-03-14T12:14:30.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.748+0000 7ff6ec697640 1 -- 172.21.15.82:0/2704150550 wait complete. 2024-03-14T12:14:30.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.749+0000 7ff6ec697640 1 Processor -- start 2024-03-14T12:14:30.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.749+0000 7ff6ec697640 1 -- start start 2024-03-14T12:14:30.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6ec697640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d00039b0 0x7ff6e415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6ec697640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40c4570 0x7ff6e414e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6ec697640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e414e910 0x7ff6e40c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6ec697640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406b340 con 0x7ff6e40c4570 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6ec697640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406b1c0 con 0x7ff6d00039b0 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6ec697640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406b4c0 con 0x7ff6e414e910 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6e9c32640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40c4570 0x7ff6e414e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6e9c32640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40c4570 0x7ff6e414e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39732/0 (socket says 172.21.15.82:39732) 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6e9c32640 1 -- 172.21.15.82:0/2234613698 learned_addr learned my addr 172.21.15.82:0/2234613698 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6eac34640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e414e910 0x7ff6e40c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6ea433640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d00039b0 0x7ff6e415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6e9c32640 1 -- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e414e910 msgr2=0x7ff6e40c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6e9c32640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e414e910 0x7ff6e40c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6e9c32640 1 -- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d00039b0 msgr2=0x7ff6e415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.750+0000 7ff6e9c32640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d00039b0 0x7ff6e415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.751+0000 7ff6e9c32640 1 -- 172.21.15.82:0/2234613698 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e414bf90 con 0x7ff6e40c4570 2024-03-14T12:14:30.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.751+0000 7ff6e9c32640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40c4570 0x7ff6e414e3d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40119c0 tx=0x7ff6d4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.751+0000 7ff6daffd640 1 -- 172.21.15.82:0/2234613698 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400faf0 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.751+0000 7ff6daffd640 1 -- 172.21.15.82:0/2234613698 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400fc90 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.751+0000 7ff6daffd640 1 -- 172.21.15.82:0/2234613698 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4014440 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.751+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e414c270 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.751+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e414c830 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.752+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e404f050 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.752+0000 7ff6daffd640 1 -- 172.21.15.82:0/2234613698 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff6d4012040 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.753+0000 7ff6daffd640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff6c806bbd0 0x7ff6c806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.753+0000 7ff6daffd640 1 -- 172.21.15.82:0/2234613698 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6d408af10 con 0x7ff6e40c4570 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.753+0000 7ff6ea433640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff6c806bbd0 0x7ff6c806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:30.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.753+0000 7ff6ea433640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff6c806bbd0 0x7ff6c806e090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0034000 tx=0x7ff6e0007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:30.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.757+0000 7ff6daffd640 1 -- 172.21.15.82:0/2234613698 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff6d4019030 con 0x7ff6e40c4570 2024-03-14T12:14:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.965+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e40743a0 con 0x7ff6e40c4570 2024-03-14T12:14:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.965+0000 7ff6daffd640 1 -- 172.21.15.82:0/2234613698 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7ff6d40593b0 con 0x7ff6e40c4570 2024-03-14T12:14:30.964 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:30.965 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:14:30.965 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:14:30.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff6c806bbd0 msgr2=0x7ff6c806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff6c806bbd0 0x7ff6c806e090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0034000 tx=0x7ff6e0007000 comp rx=0 tx=0).stop 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40c4570 msgr2=0x7ff6e414e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40c4570 0x7ff6e414e3d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40119c0 tx=0x7ff6d4011e80 comp rx=0 tx=0).stop 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 shutdown_connections 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff6c806bbd0 0x7ff6c806e090 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6e414e910 0x7ff6e40c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40c4570 0x7ff6e414e3d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 --2- 172.21.15.82:0/2234613698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6d00039b0 0x7ff6e415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 >> 172.21.15.82:0/2234613698 conn(0x7ff6e40d0b60 msgr2=0x7ff6e415abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 shutdown_connections 2024-03-14T12:14:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:30.967+0000 7ff6ec697640 1 -- 172.21.15.82:0/2234613698 wait complete. 2024-03-14T12:14:30.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.078+0000 7f5836fa2640 1 Processor -- start 2024-03-14T12:14:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.078+0000 7f5836fa2640 1 -- start start 2024-03-14T12:14:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5836fa2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300b4400 0x7f5830165830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5836fa2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58300b33a0 con 0x7f58300aede0 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5836fa2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58300b3640 con 0x7f5830071610 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5836fa2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58300b3920 con 0x7f58300b4400 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5835fa0640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f58300aede0 0x7f5830070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51042/0 (socket says 172.21.15.82:51042) 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5835fa0640 1 -- 172.21.15.82:0/1209221157 learned_addr learned my addr 172.21.15.82:0/1209221157 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f58367a1640 1 --2- 172.21.15.82:0/1209221157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300b4400 0x7f5830165830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898449936 0 0) 0x7f58300b33a0 con 0x7f58300aede0 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f581c003520 con 0x7f58300aede0 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284442208 0 0) 0x7f58300b3640 con 0x7f5830071610 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58300b33a0 con 0x7f5830071610 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258933812 0 0) 0x7f581c003520 con 0x7f58300aede0 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f58367a1640 1 -- 172.21.15.82:0/1209221157 >> v1:172.21.15.164:6789/0 conn(0x7f5830071610 legacy=0x7f583006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f58367a1640 1 -- 172.21.15.82:0/1209221157 >> v1:172.21.15.82:6789/0 conn(0x7f58300aede0 legacy=0x7f5830070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.079+0000 7f58367a1640 1 -- 172.21.15.82:0/1209221157 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58300b3c00 con 0x7f58300b4400 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.080+0000 7f58367a1640 1 --2- 172.21.15.82:0/1209221157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300b4400 0x7f5830165830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5828009dc0 tx=0x7f582802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b090b4651626967f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.080+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828039070 con 0x7f58300b4400 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.080+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582802c910 con 0x7f58300b4400 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.080+0000 7f5834f9e640 1 -- 172.21.15.82:0/1209221157 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c001690 con 0x7f58300b4400 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.081+0000 7f5836fa2640 1 -- 172.21.15.82:0/1209221157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300b4400 msgr2=0x7f5830165830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.081+0000 7f5836fa2640 1 --2- 172.21.15.82:0/1209221157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300b4400 0x7f5830165830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5828009dc0 tx=0x7f582802bde0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.081+0000 7f5836fa2640 1 -- 172.21.15.82:0/1209221157 shutdown_connections 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.081+0000 7f5836fa2640 1 --2- 172.21.15.82:0/1209221157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300b4400 0x7f5830165830 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.081+0000 7f5836fa2640 1 -- 172.21.15.82:0/1209221157 >> 172.21.15.82:0/1209221157 conn(0x7f58300cf760 msgr2=0x7f58300cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:31.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.081+0000 7f5836fa2640 1 -- 172.21.15.82:0/1209221157 shutdown_connections 2024-03-14T12:14:31.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.081+0000 7f5836fa2640 1 -- 172.21.15.82:0/1209221157 wait complete. 2024-03-14T12:14:31.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.082+0000 7f5836fa2640 1 Processor -- start 2024-03-14T12:14:31.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.082+0000 7f5836fa2640 1 -- start start 2024-03-14T12:14:31.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5836fa2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5830071610 0x7f58300ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5836fa2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58300aede0 0x7f5830145250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5836fa2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300b4400 0x7f5830158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5836fa2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583006e420 con 0x7f58300aede0 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5836fa2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58300af1c0 con 0x7f58300b4400 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5836fa2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583006e5a0 con 0x7f5830071610 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58300aede0 0x7f5830145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58300aede0 0x7f5830145250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39746/0 (socket says 172.21.15.82:39746) 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 -- 172.21.15.82:0/3068378785 learned_addr learned my addr 172.21.15.82:0/3068378785 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f58367a1640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300b4400 0x7f5830158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5835fa0640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5830071610 0x7f58300ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 -- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5830071610 msgr2=0x7f58300ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5830071610 0x7f58300ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 -- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300b4400 msgr2=0x7f5830158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300b4400 0x7f5830158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 -- 172.21.15.82:0/3068378785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583006cf90 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f5835fa0640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5830071610 0x7f58300ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f583579f640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58300aede0 0x7f5830145250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f582000c4c0 tx=0x7f582000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f58267fc640 1 -- 172.21.15.82:0/3068378785 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200052d0 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.083+0000 7f58267fc640 1 -- 172.21.15.82:0/3068378785 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820005470 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.084+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58301f71b0 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.084+0000 7f58267fc640 1 -- 172.21.15.82:0/3068378785 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820012870 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.084+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58301f7770 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.084+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5830075210 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.085+0000 7f58267fc640 1 -- 172.21.15.82:0/3068378785 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5820012a10 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.085+0000 7f58267fc640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f581406bbd0 0x7f581406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.085+0000 7f58267fc640 1 -- 172.21.15.82:0/3068378785 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f582008a6b0 con 0x7f58300aede0 2024-03-14T12:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.085+0000 7f5835fa0640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f581406bbd0 0x7f581406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.086+0000 7f5835fa0640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f581406bbd0 0x7f581406e090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f582c001eb0 tx=0x7f582c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.090+0000 7f58267fc640 1 -- 172.21.15.82:0/3068378785 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5820059230 con 0x7f58300aede0 2024-03-14T12:14:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.261+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f583015e210 con 0x7f58300aede0 2024-03-14T12:14:31.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.262+0000 7f58267fc640 1 -- 172.21.15.82:0/3068378785 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f5820058b30 con 0x7f58300aede0 2024-03-14T12:14:31.261 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:31.261 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:13:52.570910+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f581406bbd0 msgr2=0x7f581406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f581406bbd0 0x7f581406e090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f582c001eb0 tx=0x7f582c009040 comp rx=0 tx=0).stop 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58300aede0 msgr2=0x7f5830145250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58300aede0 0x7f5830145250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f582000c4c0 tx=0x7f582000c980 comp rx=0 tx=0).stop 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 shutdown_connections 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300b4400 0x7f5830158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58300aede0 0x7f5830145250 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f581406bbd0 0x7f581406e090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 --2- 172.21.15.82:0/3068378785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5830071610 0x7f58300ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 >> 172.21.15.82:0/3068378785 conn(0x7f58300cf760 msgr2=0x7f583006dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 shutdown_connections 2024-03-14T12:14:31.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.264+0000 7f5836fa2640 1 -- 172.21.15.82:0/3068378785 wait complete. 2024-03-14T12:14:31.274 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T12:14:31.274 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:31.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.374+0000 7f75f389a640 1 Processor -- start 2024-03-14T12:14:31.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.374+0000 7f75f389a640 1 -- start start 2024-03-14T12:14:31.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f389a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75ec158750 0x7f75ec0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f389a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f75ec15bba0 0x7f75ec0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f389a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec0b53f0 con 0x7f75ec071bf0 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f389a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ec072040 con 0x7f75ec158750 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f389a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ec158b30 con 0x7f75ec15bba0 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f1636640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75ec158750 0x7f75ec0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f1636640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75ec158750 0x7f75ec0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39748/0 (socket says 172.21.15.82:39748) 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.375+0000 7f75f1636640 1 -- 172.21.15.82:0/1043909015 learned_addr learned my addr 172.21.15.82:0/1043909015 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.376+0000 7f75f1636640 1 -- 172.21.15.82:0/1043909015 >> v2:172.21.15.164:3300/0 conn(0x7f75ec15bba0 msgr2=0x7f75ec0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.376+0000 7f75f0e35640 1 --2- 172.21.15.82:0/1043909015 >> v2:172.21.15.164:3300/0 conn(0x7f75ec15bba0 0x7f75ec0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.376+0000 7f75f1636640 1 --2- 172.21.15.82:0/1043909015 >> v2:172.21.15.164:3300/0 conn(0x7f75ec15bba0 0x7f75ec0716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.376+0000 7f75f1636640 1 -- 172.21.15.82:0/1043909015 >> v1:172.21.15.164:6789/0 conn(0x7f75ec071bf0 legacy=0x7f75ec06dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.376+0000 7f75f1636640 1 -- 172.21.15.82:0/1043909015 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0b4e30 con 0x7f75ec158750 2024-03-14T12:14:31.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.376+0000 7f75f1636640 1 --2- 172.21.15.82:0/1043909015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec158750 0x7f75ec0b47e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f75ec06d5e0 tx=0x7f75e802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6fb6364c8b4e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75e3fff640 1 -- 172.21.15.82:0/1043909015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e802ca80 con 0x7f75ec158750 2024-03-14T12:14:31.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75e3fff640 1 -- 172.21.15.82:0/1043909015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e802cc20 con 0x7f75ec158750 2024-03-14T12:14:31.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75e3fff640 1 -- 172.21.15.82:0/1043909015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80343f0 con 0x7f75ec158750 2024-03-14T12:14:31.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 -- 172.21.15.82:0/1043909015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec158750 msgr2=0x7f75ec0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 --2- 172.21.15.82:0/1043909015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec158750 0x7f75ec0b47e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f75ec06d5e0 tx=0x7f75e802bee0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 -- 172.21.15.82:0/1043909015 shutdown_connections 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 --2- 172.21.15.82:0/1043909015 >> v2:172.21.15.164:3300/0 conn(0x7f75ec15bba0 0x7f75ec0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 --2- 172.21.15.82:0/1043909015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec158750 0x7f75ec0b47e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 -- 172.21.15.82:0/1043909015 >> 172.21.15.82:0/1043909015 conn(0x7f75ec0d0a20 msgr2=0x7f75ec0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 -- 172.21.15.82:0/1043909015 shutdown_connections 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.377+0000 7f75f389a640 1 -- 172.21.15.82:0/1043909015 wait complete. 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.378+0000 7f75f389a640 1 Processor -- start 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.378+0000 7f75f389a640 1 -- start start 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f389a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec071bf0 0x7f75ec156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f389a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75ec158750 0x7f75ec166760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f389a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75ec15bba0 0x7f75ec166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f389a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec06dab0 con 0x7f75ec071bf0 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f389a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec06d7d0 con 0x7f75ec15bba0 2024-03-14T12:14:31.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f389a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec15c030 con 0x7f75ec158750 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1636640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec071bf0 0x7f75ec156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1636640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec071bf0 0x7f75ec156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39758/0 (socket says 172.21.15.82:39758) 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1636640 1 -- 172.21.15.82:0/761409243 learned_addr learned my addr 172.21.15.82:0/761409243 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f0e35640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75ec158750 0x7f75ec166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1e37640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75ec15bba0 0x7f75ec166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1636640 1 -- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75ec158750 msgr2=0x7f75ec166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1636640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75ec158750 0x7f75ec166760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1636640 1 -- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75ec15bba0 msgr2=0x7f75ec166ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.379+0000 7f75f1636640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75ec15bba0 0x7f75ec166ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.380+0000 7f75f1636640 1 -- 172.21.15.82:0/761409243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec13f630 con 0x7f75ec071bf0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.380+0000 7f75f1636640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec071bf0 0x7f75ec156b10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f75e8036040 tx=0x7f75e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.380+0000 7f75e1ffb640 1 -- 172.21.15.82:0/761409243 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8043020 con 0x7f75ec071bf0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.380+0000 7f75e1ffb640 1 -- 172.21.15.82:0/761409243 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e803e450 con 0x7f75ec071bf0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.380+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec13f910 con 0x7f75ec071bf0 2024-03-14T12:14:31.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.380+0000 7f75e1ffb640 1 -- 172.21.15.82:0/761409243 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e803e820 con 0x7f75ec071bf0 2024-03-14T12:14:31.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.380+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec167ee0 con 0x7f75ec071bf0 2024-03-14T12:14:31.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.381+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec13fae0 con 0x7f75ec071bf0 2024-03-14T12:14:31.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.381+0000 7f75e1ffb640 1 -- 172.21.15.82:0/761409243 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f75e8005d10 con 0x7f75ec071bf0 2024-03-14T12:14:31.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.382+0000 7f75e1ffb640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f75d006bbd0 0x7f75d006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.382+0000 7f75e1ffb640 1 -- 172.21.15.82:0/761409243 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75e80ad700 con 0x7f75ec071bf0 2024-03-14T12:14:31.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.382+0000 7f75f0e35640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f75d006bbd0 0x7f75d006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.382+0000 7f75f0e35640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f75d006bbd0 0x7f75d006e090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75dc006fd0 tx=0x7f75dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.386+0000 7f75e1ffb640 1 -- 172.21.15.82:0/761409243 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f75e807c1f0 con 0x7f75ec071bf0 2024-03-14T12:14:31.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.600+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75ec04f050 con 0x7f75ec071bf0 2024-03-14T12:14:31.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.600+0000 7f75e1ffb640 1 -- 172.21.15.82:0/761409243 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f75e807ce20 con 0x7f75ec071bf0 2024-03-14T12:14:31.599 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:31.600 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:14:31.600 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:14:31.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f75d006bbd0 msgr2=0x7f75d006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f75d006bbd0 0x7f75d006e090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75dc006fd0 tx=0x7f75dc008040 comp rx=0 tx=0).stop 2024-03-14T12:14:31.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec071bf0 msgr2=0x7f75ec156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec071bf0 0x7f75ec156b10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f75e8036040 tx=0x7f75e8004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 shutdown_connections 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75ec15bba0 0x7f75ec166ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f75d006bbd0 0x7f75d006e090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75ec158750 0x7f75ec166760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 --2- 172.21.15.82:0/761409243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75ec071bf0 0x7f75ec156b10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 >> 172.21.15.82:0/761409243 conn(0x7f75ec0d0a20 msgr2=0x7f75ec0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 shutdown_connections 2024-03-14T12:14:31.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.602+0000 7f75f389a640 1 -- 172.21.15.82:0/761409243 wait complete. 2024-03-14T12:14:31.612 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:31.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.714+0000 7fac9fe73640 1 Processor -- start 2024-03-14T12:14:31.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.715+0000 7fac9fe73640 1 -- start start 2024-03-14T12:14:31.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.715+0000 7fac9fe73640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac980d0760 0x7fac980d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.715+0000 7fac9fe73640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fac980cf370 0x7fac980d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.715+0000 7fac9fe73640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980d1080 con 0x7fac980d1da0 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.715+0000 7fac9fe73640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980cf8c0 con 0x7fac980d0760 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.715+0000 7fac9fe73640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980cfba0 con 0x7fac980cf370 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.715+0000 7fac9dc0f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac980d0760 0x7fac980d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.716+0000 7fac9dc0f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac980d0760 0x7fac980d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60162/0 (socket says 172.21.15.82:60162) 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.716+0000 7fac9dc0f640 1 -- 172.21.15.82:0/2382310877 learned_addr learned my addr 172.21.15.82:0/2382310877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.716+0000 7fac9d40e640 1 --2- 172.21.15.82:0/2382310877 >> v2:172.21.15.164:3300/0 conn(0x7fac980cf370 0x7fac980d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.716+0000 7fac9dc0f640 1 -- 172.21.15.82:0/2382310877 >> v2:172.21.15.164:3300/0 conn(0x7fac980cf370 msgr2=0x7fac980d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.716+0000 7fac9dc0f640 1 --2- 172.21.15.82:0/2382310877 >> v2:172.21.15.164:3300/0 conn(0x7fac980cf370 0x7fac980d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.716+0000 7fac9dc0f640 1 -- 172.21.15.82:0/2382310877 >> v1:172.21.15.164:6789/0 conn(0x7fac980d1da0 legacy=0x7fac980d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.716+0000 7fac9dc0f640 1 -- 172.21.15.82:0/2382310877 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980b6c40 con 0x7fac980d0760 2024-03-14T12:14:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.717+0000 7fac9dc0f640 1 --2- 172.21.15.82:0/2382310877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980d0760 0x7fac980d0b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac9808cbb0 tx=0x7fac9402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7b437ab92cf6a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.720+0000 7fac9cc0d640 1 -- 172.21.15.82:0/2382310877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94034070 con 0x7fac980d0760 2024-03-14T12:14:31.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.720+0000 7fac9cc0d640 1 -- 172.21.15.82:0/2382310877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94030860 con 0x7fac980d0760 2024-03-14T12:14:31.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.721+0000 7fac9cc0d640 1 -- 172.21.15.82:0/2382310877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94030c30 con 0x7fac980d0760 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.721+0000 7fac9fe73640 1 -- 172.21.15.82:0/2382310877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980d0760 msgr2=0x7fac980d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.721+0000 7fac9fe73640 1 --2- 172.21.15.82:0/2382310877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980d0760 0x7fac980d0b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac9808cbb0 tx=0x7fac9402b9a0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.722+0000 7fac9fe73640 1 -- 172.21.15.82:0/2382310877 shutdown_connections 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.722+0000 7fac9fe73640 1 --2- 172.21.15.82:0/2382310877 >> v2:172.21.15.164:3300/0 conn(0x7fac980cf370 0x7fac980d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.722+0000 7fac9fe73640 1 --2- 172.21.15.82:0/2382310877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980d0760 0x7fac980d0b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.722+0000 7fac9fe73640 1 -- 172.21.15.82:0/2382310877 >> 172.21.15.82:0/2382310877 conn(0x7fac9809d030 msgr2=0x7fac980d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.722+0000 7fac9fe73640 1 -- 172.21.15.82:0/2382310877 shutdown_connections 2024-03-14T12:14:31.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.722+0000 7fac9fe73640 1 -- 172.21.15.82:0/2382310877 wait complete. 2024-03-14T12:14:31.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.723+0000 7fac9fe73640 1 Processor -- start 2024-03-14T12:14:31.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.723+0000 7fac9fe73640 1 -- start start 2024-03-14T12:14:31.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9fe73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980cf370 0x7fac98154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9fe73640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac980d0760 0x7fac980c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9fe73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac980d1da0 0x7fac980c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9fe73640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980d4ba0 con 0x7fac980cf370 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9fe73640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980b7200 con 0x7fac980d1da0 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9fe73640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac981617a0 con 0x7fac980d0760 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980cf370 0x7fac98154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980cf370 0x7fac98154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60166/0 (socket says 172.21.15.82:60166) 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 -- 172.21.15.82:0/1491710900 learned_addr learned my addr 172.21.15.82:0/1491710900 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9d40e640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac980d0760 0x7fac980c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9e410640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac980d1da0 0x7fac980c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 -- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac980d0760 msgr2=0x7fac980c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac980d0760 0x7fac980c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 -- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac980d1da0 msgr2=0x7fac980c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac980d1da0 0x7fac980c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.724+0000 7fac9dc0f640 1 -- 172.21.15.82:0/1491710900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980c8240 con 0x7fac980cf370 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.725+0000 7fac9dc0f640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980cf370 0x7fac98154160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fac94000c00 tx=0x7fac94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.725+0000 7fac8e7fc640 1 -- 172.21.15.82:0/1491710900 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94048070 con 0x7fac980cf370 2024-03-14T12:14:31.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.725+0000 7fac8e7fc640 1 -- 172.21.15.82:0/1491710900 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94030660 con 0x7fac980cf370 2024-03-14T12:14:31.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.725+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac980c8520 con 0x7fac980cf370 2024-03-14T12:14:31.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.725+0000 7fac8e7fc640 1 -- 172.21.15.82:0/1491710900 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94034040 con 0x7fac980cf370 2024-03-14T12:14:31.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.725+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980c8ae0 con 0x7fac980cf370 2024-03-14T12:14:31.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.726+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9804f050 con 0x7fac980cf370 2024-03-14T12:14:31.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.726+0000 7fac8e7fc640 1 -- 172.21.15.82:0/1491710900 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fac9404b070 con 0x7fac980cf370 2024-03-14T12:14:31.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.726+0000 7fac8e7fc640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fac7c06bbd0 0x7fac7c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:31.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.727+0000 7fac8e7fc640 1 -- 172.21.15.82:0/1491710900 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fac940acb80 con 0x7fac980cf370 2024-03-14T12:14:31.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.727+0000 7fac9d40e640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fac7c06bbd0 0x7fac7c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:31.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.727+0000 7fac9d40e640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fac7c06bbd0 0x7fac7c06e090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fac880079a0 tx=0x7fac88008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:31.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.731+0000 7fac8e7fc640 1 -- 172.21.15.82:0/1491710900 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fac9407b140 con 0x7fac980cf370 2024-03-14T12:14:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.938+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac980d0e60 con 0x7fac980cf370 2024-03-14T12:14:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.939+0000 7fac8e7fc640 1 -- 172.21.15.82:0/1491710900 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fac9407f1e0 con 0x7fac980cf370 2024-03-14T12:14:31.938 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:31.938 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4059473489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4059473489},{"type":"v1","addr":"172.21.15.82:6836","nonce":4059473489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3671777494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3671777494},{"type":"v1","addr":"172.21.15.82:6837","nonce":3671777494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3224798741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3224798741},{"type":"v1","addr":"172.21.15.164:6835","nonce":3224798741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:13:50.902698+0000","modified":"2024-03-14T12:13:53.462023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/3117714765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3117714765},{"type":"v1","addr":"172.21.15.164:6834","nonce":3117714765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T12:14:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fac7c06bbd0 msgr2=0x7fac7c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fac7c06bbd0 0x7fac7c06e090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fac880079a0 tx=0x7fac88008040 comp rx=0 tx=0).stop 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980cf370 msgr2=0x7fac98154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980cf370 0x7fac98154160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fac94000c00 tx=0x7fac94004730 comp rx=0 tx=0).stop 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 shutdown_connections 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac980d1da0 0x7fac980c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fac7c06bbd0 0x7fac7c06e090 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac980d0760 0x7fac980c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 --2- 172.21.15.82:0/1491710900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac980cf370 0x7fac98154160 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 >> 172.21.15.82:0/1491710900 conn(0x7fac9809d030 msgr2=0x7fac9815ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 shutdown_connections 2024-03-14T12:14:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:31.941+0000 7fac9fe73640 1 -- 172.21.15.82:0/1491710900 wait complete. 2024-03-14T12:14:31.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:32.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.051+0000 7f0847675640 1 Processor -- start 2024-03-14T12:14:32.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.051+0000 7f0847675640 1 -- start start 2024-03-14T12:14:32.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0847675640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 0x7f08400cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0847675640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08400b3da0 con 0x7f084015dde0 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0847675640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08400b4080 con 0x7f08400b4b70 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0847675640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0840070d40 con 0x7f0840151230 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0845c12640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f084015dde0 0x7f084006cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35414/0 (socket says 172.21.15.82:35414) 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0845c12640 1 -- 172.21.15.82:0/3172596361 learned_addr learned my addr 172.21.15.82:0/3172596361 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0845411640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 0x7f08400cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432407849 0 0) 0x7f08400b3da0 con 0x7f084015dde0 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.052+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f082c003440 con 0x7f084015dde0 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3535425909 0 0) 0x7f082c003440 con 0x7f084015dde0 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 msgr2=0x7f08400cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 0x7f08400cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 >> v1:172.21.15.164:6789/0 conn(0x7f08400b4b70 legacy=0x7f0840070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400b4360 con 0x7f084015dde0 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f08380019f0 con 0x7f084015dde0 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 >> v1:172.21.15.82:6789/0 conn(0x7f084015dde0 legacy=0x7f084006cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f082c003ad0 0x7f082c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 0x7f082c00a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c00afe0 0x7f082c00d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0840070d40 con 0x7f082c003ad0 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c00dcf0 con 0x7f082c00afe0 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c00def0 con 0x7f0840151230 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0845411640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f082c003ad0 0x7f082c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0845c12640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c00afe0 0x7f082c00d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0844c10640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 0x7f082c00a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0845411640 1 -- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 msgr2=0x7f082c00a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0845411640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0840151230 0x7f082c00a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0845411640 1 -- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c00afe0 msgr2=0x7f082c00d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0845411640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c00afe0 0x7f082c00d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.053+0000 7f0845411640 1 -- 172.21.15.82:0/3172596361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c003440 con 0x7f082c003ad0 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0845411640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f082c003ad0 0x7f082c003ef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f083c009ac0 tx=0x7f082c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2aabb21834ca61 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c039070 con 0x7f082c003ad0 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c02c6b0 con 0x7f082c003ad0 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0837fff640 1 -- 172.21.15.82:0/3172596361 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c009690 con 0x7f082c003ad0 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0847675640 1 -- 172.21.15.82:0/3172596361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f082c003ad0 msgr2=0x7f082c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0847675640 1 --2- 172.21.15.82:0/3172596361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f082c003ad0 0x7f082c003ef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f083c009ac0 tx=0x7f082c003730 comp rx=0 tx=0).stop 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0845c12640 1 -- 172.21.15.82:0/3172596361 reap_dead start 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0847675640 1 -- 172.21.15.82:0/3172596361 shutdown_connections 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.054+0000 7f0847675640 1 -- 172.21.15.82:0/3172596361 >> 172.21.15.82:0/3172596361 conn(0x7f08400b1c70 msgr2=0x7f08400cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.055+0000 7f0847675640 1 -- 172.21.15.82:0/3172596361 shutdown_connections 2024-03-14T12:14:32.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.055+0000 7f0847675640 1 -- 172.21.15.82:0/3172596361 wait complete. 2024-03-14T12:14:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 Processor -- start 2024-03-14T12:14:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 -- start start 2024-03-14T12:14:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c003ad0 0x7f0840137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0840134cf0 0x7f0840149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08401eecb0 0x7f08401ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084006c690 con 0x7f0840134cf0 2024-03-14T12:14:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084006c3b0 con 0x7f082c003ad0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0847675640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084006d230 con 0x7f08401eecb0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.056+0000 7f0844c10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0840134cf0 0x7f0840149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0840134cf0 0x7f0840149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60184/0 (socket says 172.21.15.82:60184) 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 -- 172.21.15.82:0/2226906847 learned_addr learned my addr 172.21.15.82:0/2226906847 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 -- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08401eecb0 msgr2=0x7f08401ef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0845c12640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08401eecb0 0x7f08401ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0845411640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c003ad0 0x7f0840137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08401eecb0 0x7f08401ef130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 -- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c003ad0 msgr2=0x7f0840137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c003ad0 0x7f0840137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 -- 172.21.15.82:0/2226906847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0840135150 con 0x7f0840134cf0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0844c10640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0840134cf0 0x7f0840149630 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f083000e580 tx=0x7f083000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0835ffb640 1 -- 172.21.15.82:0/2226906847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300052e0 con 0x7f0840134cf0 2024-03-14T12:14:32.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0835ffb640 1 -- 172.21.15.82:0/2226906847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830018070 con 0x7f0840134cf0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084014c2d0 con 0x7f0840134cf0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0835ffb640 1 -- 172.21.15.82:0/2226906847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830003810 con 0x7f0840134cf0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.057+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084014c890 con 0x7f0840134cf0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.058+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084004f050 con 0x7f0840134cf0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.058+0000 7f0835ffb640 1 -- 172.21.15.82:0/2226906847 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f08300039b0 con 0x7f0840134cf0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.059+0000 7f0835ffb640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f082406bc00 0x7f082406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.059+0000 7f0845411640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f082406bc00 0x7f082406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.059+0000 7f0835ffb640 1 -- 172.21.15.82:0/2226906847 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f083008a6a0 con 0x7f0840134cf0 2024-03-14T12:14:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.059+0000 7f0845411640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f082406bc00 0x7f082406e0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f083c036000 tx=0x7f083c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.063+0000 7f0835ffb640 1 -- 172.21.15.82:0/2226906847 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0840135150 con 0x7f0840134cf0 2024-03-14T12:14:32.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.238+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08400b2f40 con 0x7f0840134cf0 2024-03-14T12:14:32.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.239+0000 7f0835ffb640 1 -- 172.21.15.82:0/2226906847 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f0830059110 con 0x7f0840134cf0 2024-03-14T12:14:32.238 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:32.239 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":22,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:31.706031+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:13:49.489722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T12:13:49.874257+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f082406bc00 msgr2=0x7f082406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f082406bc00 0x7f082406e0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f083c036000 tx=0x7f083c030000 comp rx=0 tx=0).stop 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0840134cf0 msgr2=0x7f0840149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0840134cf0 0x7f0840149630 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f083000e580 tx=0x7f083000ea40 comp rx=0 tx=0).stop 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 shutdown_connections 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f082406bc00 0x7f082406e0c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08401eecb0 0x7f08401ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0840134cf0 0x7f0840149630 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 --2- 172.21.15.82:0/2226906847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f082c003ad0 0x7f0840137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 >> 172.21.15.82:0/2226906847 conn(0x7f08400b1c70 msgr2=0x7f084014f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 shutdown_connections 2024-03-14T12:14:32.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.241+0000 7f0847675640 1 -- 172.21.15.82:0/2226906847 wait complete. 2024-03-14T12:14:32.252 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T12:14:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.354+0000 7f3ea799d640 1 Processor -- start 2024-03-14T12:14:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.354+0000 7f3ea799d640 1 -- start start 2024-03-14T12:14:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea799d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3ea00cdd60 0x7f3ea00ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea799d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea0144ac0 0x7f3ea0144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea799d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00ce8b0 con 0x7f3ea015f7a0 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea799d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00cf150 con 0x7f3ea00cdd60 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea799d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00ceb90 con 0x7f3ea0144ac0 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea4f38640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3ea00cdd60 0x7f3ea00ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea4f38640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3ea00cdd60 0x7f3ea00ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60192/0 (socket says 172.21.15.82:60192) 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea4f38640 1 -- 172.21.15.82:0/720902342 learned_addr learned my addr 172.21.15.82:0/720902342 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea5f3a640 1 --2- 172.21.15.82:0/720902342 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea0144ac0 0x7f3ea0144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea4f38640 1 -- 172.21.15.82:0/720902342 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea0144ac0 msgr2=0x7f3ea0144ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea4f38640 1 --2- 172.21.15.82:0/720902342 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea0144ac0 0x7f3ea0144ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea4f38640 1 -- 172.21.15.82:0/720902342 >> v1:172.21.15.164:6789/0 conn(0x7f3ea015f7a0 legacy=0x7f3ea015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.355+0000 7f3ea4f38640 1 -- 172.21.15.82:0/720902342 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00cee70 con 0x7f3ea00cdd60 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.356+0000 7f3ea4f38640 1 --2- 172.21.15.82:0/720902342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea00cdd60 0x7f3ea00ce210 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926584ed446056dd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.356+0000 7f3e97fff640 1 -- 172.21.15.82:0/720902342 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90035020 con 0x7f3ea00cdd60 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.356+0000 7f3e97fff640 1 -- 172.21.15.82:0/720902342 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e90031640 con 0x7f3ea00cdd60 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.356+0000 7f3e97fff640 1 -- 172.21.15.82:0/720902342 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9002d040 con 0x7f3ea00cdd60 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 -- 172.21.15.82:0/720902342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea00cdd60 msgr2=0x7f3ea00ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 --2- 172.21.15.82:0/720902342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea00cdd60 0x7f3ea00ce210 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e9002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 -- 172.21.15.82:0/720902342 shutdown_connections 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 --2- 172.21.15.82:0/720902342 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea0144ac0 0x7f3ea0144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 --2- 172.21.15.82:0/720902342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea00cdd60 0x7f3ea00ce210 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 -- 172.21.15.82:0/720902342 >> 172.21.15.82:0/720902342 conn(0x7f3ea00b1c70 msgr2=0x7f3ea00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 -- 172.21.15.82:0/720902342 shutdown_connections 2024-03-14T12:14:32.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.357+0000 7f3ea799d640 1 -- 172.21.15.82:0/720902342 wait complete. 2024-03-14T12:14:32.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.358+0000 7f3ea799d640 1 Processor -- start 2024-03-14T12:14:32.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.358+0000 7f3ea799d640 1 -- start start 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea799d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea00cdd60 0x7f3ea0135a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea799d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea0144ac0 0x7f3ea00c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea799d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea015f7a0 0x7f3ea00cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea799d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cc670 con 0x7f3ea015f7a0 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea799d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cc4f0 con 0x7f3ea0144ac0 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea799d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0070630 con 0x7f3ea00cdd60 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea015f7a0 0x7f3ea00cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea015f7a0 0x7f3ea00cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60194/0 (socket says 172.21.15.82:60194) 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 -- 172.21.15.82:0/3382705186 learned_addr learned my addr 172.21.15.82:0/3382705186 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5739640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea00cdd60 0x7f3ea0135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea4f38640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea0144ac0 0x7f3ea00c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 -- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea00cdd60 msgr2=0x7f3ea0135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea00cdd60 0x7f3ea0135a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 -- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea0144ac0 msgr2=0x7f3ea00c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea0144ac0 0x7f3ea00c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 -- 172.21.15.82:0/3382705186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0132660 con 0x7f3ea015f7a0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5739640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea00cdd60 0x7f3ea0135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea4f38640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea0144ac0 0x7f3ea00c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea5f3a640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea015f7a0 0x7f3ea00cb900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e7f0 tx=0x7f3e9800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3e95ffb640 1 -- 172.21.15.82:0/3382705186 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98017070 con 0x7f3ea015f7a0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3e95ffb640 1 -- 172.21.15.82:0/3382705186 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9800ee40 con 0x7f3ea015f7a0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea0132940 con 0x7f3ea015f7a0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.359+0000 7f3e95ffb640 1 -- 172.21.15.82:0/3382705186 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800cb80 con 0x7f3ea015f7a0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.360+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea0132f00 con 0x7f3ea015f7a0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.361+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00ccae0 con 0x7f3ea015f7a0 2024-03-14T12:14:32.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.361+0000 7f3e95ffb640 1 -- 172.21.15.82:0/3382705186 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3e9801a070 con 0x7f3ea015f7a0 2024-03-14T12:14:32.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.362+0000 7f3e95ffb640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e8406bbd0 0x7f3e8406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.362+0000 7f3ea5739640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e8406bbd0 0x7f3e8406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.362+0000 7f3e95ffb640 1 -- 172.21.15.82:0/3382705186 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3e9808aca0 con 0x7f3ea015f7a0 2024-03-14T12:14:32.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.362+0000 7f3ea5739640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e8406bbd0 0x7f3e8406e090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0012e0 tx=0x7f3e9c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.366+0000 7f3e95ffb640 1 -- 172.21.15.82:0/3382705186 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3ea00ccae0 con 0x7f3ea015f7a0 2024-03-14T12:14:32.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.572+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3ea004f050 con 0x7f3ea015f7a0 2024-03-14T12:14:32.572 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-14T12:14:32.572+0000 7f8354c85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T12:14:32.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.575+0000 7f3e95ffb640 1 -- 172.21.15.82:0/3382705186 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3e98059790 con 0x7f3ea015f7a0 2024-03-14T12:14:32.575 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T12:14:32.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e8406bbd0 msgr2=0x7f3e8406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e8406bbd0 0x7f3e8406e090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0012e0 tx=0x7f3e9c009040 comp rx=0 tx=0).stop 2024-03-14T12:14:32.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea015f7a0 msgr2=0x7f3ea00cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea015f7a0 0x7f3ea00cb900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e7f0 tx=0x7f3e9800ebf0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 shutdown_connections 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ea015f7a0 0x7f3ea00cb900 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea0144ac0 0x7f3ea00c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3e8406bbd0 0x7f3e8406e090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 --2- 172.21.15.82:0/3382705186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3ea00cdd60 0x7f3ea0135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 >> 172.21.15.82:0/3382705186 conn(0x7f3ea00b1c70 msgr2=0x7f3ea0159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.579+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 shutdown_connections 2024-03-14T12:14:32.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.580+0000 7f3ea799d640 1 -- 172.21.15.82:0/3382705186 wait complete. 2024-03-14T12:14:32.585 INFO:tasks.ceph.mds.b.smithi164.stderr: -6811> 2024-03-14T12:13:56.078+0000 7fe469e74640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:14:32.591 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T12:14:32.616 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-03-14T12:14:32.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.696+0000 7f4a0521f640 1 Processor -- start 2024-03-14T12:14:32.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.696+0000 7f4a0521f640 1 -- start start 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f4a0521f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 0x7f4a000d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f4a0521f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4a0015e2a0 0x7f4a000d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f4a0521f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000d1420 con 0x7f4a0015dec0 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f4a0521f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000d1700 con 0x7f4a0015e2a0 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f4a0521f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c80a0 con 0x7f4a000ca540 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f49fed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4a0015dec0 0x7f4a000d2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35420/0 (socket says 172.21.15.82:35420) 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f49fed9d640 1 -- 172.21.15.82:0/3357695891 learned_addr learned my addr 172.21.15.82:0/3357695891 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887617283 0 0) 0x7f4a000d1420 con 0x7f4a0015dec0 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e8003520 con 0x7f4a0015dec0 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f49ff59e640 1 --2- 172.21.15.82:0/3357695891 >> v2:172.21.15.164:3300/0 conn(0x7f4a0015e2a0 0x7f4a000d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.697+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 0x7f4a000d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967154260 0 0) 0x7f49e8003520 con 0x7f4a0015dec0 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 msgr2=0x7f4a000d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 0x7f4a000d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 >> v2:172.21.15.164:3300/0 conn(0x7f4a0015e2a0 msgr2=0x7f4a000d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 --2- 172.21.15.82:0/3357695891 >> v2:172.21.15.164:3300/0 conn(0x7f4a0015e2a0 0x7f4a000d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000c7a50 con 0x7f4a0015dec0 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f49f00019f0 con 0x7f4a0015dec0 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 >> v1:172.21.15.82:6789/0 conn(0x7f4a0015dec0 legacy=0x7f4a000d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 0x7f49e80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49e8003f30 0x7f49e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e8004d40 0x7f49e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000d1700 con 0x7f49e8003f30 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c80a0 con 0x7f49e8004d40 2024-03-14T12:14:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e800ba10 con 0x7f4a000ca540 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49e8003f30 0x7f49e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fed9d640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 0x7f49e80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49ff59e640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e8004d40 0x7f49e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fe59c640 1 -- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 msgr2=0x7f49e80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a000ca540 0x7f49e80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fe59c640 1 -- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e8004d40 msgr2=0x7f49e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e8004d40 0x7f49e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49fe59c640 1 -- 172.21.15.82:0/3357695891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e8003520 con 0x7f49e8003f30 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.698+0000 7f49ff59e640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e8004d40 0x7f49e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.699+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49e8003f30 0x7f49e8004800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ac0 tx=0x7f49e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c48fb9aa5d97ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.699+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4039070 con 0x7f49e8003f30 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.699+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f402c680 con 0x7f49e8003f30 2024-03-14T12:14:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.699+0000 7f49fdd9b640 1 -- 172.21.15.82:0/3357695891 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4009690 con 0x7f49e8003f30 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.699+0000 7f4a0521f640 1 -- 172.21.15.82:0/3357695891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49e8003f30 msgr2=0x7f49e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.699+0000 7f4a0521f640 1 --2- 172.21.15.82:0/3357695891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49e8003f30 0x7f49e8004800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ac0 tx=0x7f49e8005830 comp rx=0 tx=0).stop 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.699+0000 7f49ff59e640 1 -- 172.21.15.82:0/3357695891 reap_dead start 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.700+0000 7f4a0521f640 1 -- 172.21.15.82:0/3357695891 shutdown_connections 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.700+0000 7f4a0521f640 1 -- 172.21.15.82:0/3357695891 >> 172.21.15.82:0/3357695891 conn(0x7f4a000d3dd0 msgr2=0x7f4a000d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.700+0000 7f4a0521f640 1 -- 172.21.15.82:0/3357695891 shutdown_connections 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.700+0000 7f4a0521f640 1 -- 172.21.15.82:0/3357695891 wait complete. 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.701+0000 7f4a0521f640 1 Processor -- start 2024-03-14T12:14:32.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.701+0000 7f4a0521f640 1 -- start start 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f4a0521f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49e8003f30 0x7f4a0015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f4a0521f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0015fb00 0x7f4a000cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f4a0521f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a000cc3f0 0x7f4a00160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f4a0521f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000d0f50 con 0x7f4a0015fb00 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f4a0521f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000d0dd0 con 0x7f4a000cc3f0 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f4a0521f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000d10d0 con 0x7f49e8003f30 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0015fb00 0x7f4a000cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0015fb00 0x7f4a000cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60212/0 (socket says 172.21.15.82:60212) 2024-03-14T12:14:32.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 -- 172.21.15.82:0/3140284694 learned_addr learned my addr 172.21.15.82:0/3140284694 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fed9d640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49e8003f30 0x7f4a0015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49ff59e640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a000cc3f0 0x7f4a00160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 -- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49e8003f30 msgr2=0x7f4a0015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49e8003f30 0x7f4a0015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 -- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a000cc3f0 msgr2=0x7f4a00160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a000cc3f0 0x7f4a00160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 -- 172.21.15.82:0/3140284694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000cb0f0 con 0x7f4a0015fb00 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49ff59e640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a000cc3f0 0x7f4a00160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49fe59c640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0015fb00 0x7f4a000cbeb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f49f4037040 tx=0x7f49f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.702+0000 7f49e77fe640 1 -- 172.21.15.82:0/3140284694 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4039050 con 0x7f4a0015fb00 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.703+0000 7f49e77fe640 1 -- 172.21.15.82:0/3140284694 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4045070 con 0x7f4a0015fb00 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.703+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a000cb3d0 con 0x7f4a0015fb00 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.703+0000 7f49e77fe640 1 -- 172.21.15.82:0/3140284694 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40044e0 con 0x7f4a0015fb00 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.703+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a000cb990 con 0x7f4a0015fb00 2024-03-14T12:14:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.704+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a000cae80 con 0x7f4a0015fb00 2024-03-14T12:14:32.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.704+0000 7f49e77fe640 1 -- 172.21.15.82:0/3140284694 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f49f4003850 con 0x7f4a0015fb00 2024-03-14T12:14:32.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.704+0000 7f49e77fe640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f49ec06bc00 0x7f49ec06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:32.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.704+0000 7f49e77fe640 1 -- 172.21.15.82:0/3140284694 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f49f40ac5b0 con 0x7f4a0015fb00 2024-03-14T12:14:32.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.704+0000 7f49fed9d640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f49ec06bc00 0x7f49ec06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:32.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.705+0000 7f49fed9d640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f49ec06bc00 0x7f49ec06e0c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a001615f0 tx=0x7f49f0006040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:32.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.709+0000 7f49e77fe640 1 -- 172.21.15.82:0/3140284694 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f49f404e020 con 0x7f4a0015fb00 2024-03-14T12:14:32.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:32.917+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4a0004f050 con 0x7f4a0015fb00 2024-03-14T12:14:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.576+0000 7f49e77fe640 1 -- 172.21.15.82:0/3140284694 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f49f407aa80 con 0x7f4a0015fb00 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.578+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f49ec06bc00 msgr2=0x7f49ec06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.578+0000 7f4a0521f640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f49ec06bc00 0x7f49ec06e0c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a001615f0 tx=0x7f49f0006040 comp rx=0 tx=0).stop 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.578+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0015fb00 msgr2=0x7f4a000cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.578+0000 7f4a0521f640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0015fb00 0x7f4a000cbeb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f49f4037040 tx=0x7f49f4004730 comp rx=0 tx=0).stop 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.578+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 shutdown_connections 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.579+0000 7f4a0521f640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a000cc3f0 0x7f4a00160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.579+0000 7f4a0521f640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0015fb00 0x7f4a000cbeb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.579+0000 7f4a0521f640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f49ec06bc00 0x7f49ec06e0c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.579+0000 7f4a0521f640 1 --2- 172.21.15.82:0/3140284694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49e8003f30 0x7f4a0015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.579+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 >> 172.21.15.82:0/3140284694 conn(0x7f4a000d3dd0 msgr2=0x7f4a000d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.579+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 shutdown_connections 2024-03-14T12:14:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.579+0000 7f4a0521f640 1 -- 172.21.15.82:0/3140284694 wait complete. 2024-03-14T12:14:33.591 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T12:14:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.697+0000 7fe597a43640 1 Processor -- start 2024-03-14T12:14:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.697+0000 7fe597a43640 1 -- start start 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe597a43640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe5900b2860 0x7fe5900cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe597a43640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe59006e670 0x7fe5900cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe597a43640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5900b46a0 con 0x7fe5900cfd10 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe597a43640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5900cced0 con 0x7fe59006e670 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe597a43640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5900b3b20 con 0x7fe5900b2860 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe595fe0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe59006e670 0x7fe5900cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe595fe0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe59006e670 0x7fe5900cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60214/0 (socket says 172.21.15.82:60214) 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe595fe0640 1 -- 172.21.15.82:0/839692819 learned_addr learned my addr 172.21.15.82:0/839692819 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:33.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe594fde640 1 --2- 172.21.15.82:0/839692819 >> v2:172.21.15.164:3300/0 conn(0x7fe5900b2860 0x7fe5900cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe595fe0640 1 -- 172.21.15.82:0/839692819 >> v2:172.21.15.164:3300/0 conn(0x7fe5900b2860 msgr2=0x7fe5900cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe595fe0640 1 --2- 172.21.15.82:0/839692819 >> v2:172.21.15.164:3300/0 conn(0x7fe5900b2860 0x7fe5900cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe595fe0640 1 -- 172.21.15.82:0/839692819 >> v1:172.21.15.164:6789/0 conn(0x7fe5900cfd10 legacy=0x7fe5900d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.698+0000 7fe595fe0640 1 -- 172.21.15.82:0/839692819 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900b3e00 con 0x7fe59006e670 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.699+0000 7fe595fe0640 1 --2- 172.21.15.82:0/839692819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe59006e670 0x7fe5900cd8f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5900d0950 tx=0x7fe58802e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2df22e883b7dc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.699+0000 7fe587fff640 1 -- 172.21.15.82:0/839692819 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58802ea00 con 0x7fe59006e670 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.699+0000 7fe587fff640 1 -- 172.21.15.82:0/839692819 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58802eba0 con 0x7fe59006e670 2024-03-14T12:14:33.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.699+0000 7fe587fff640 1 -- 172.21.15.82:0/839692819 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588009600 con 0x7fe59006e670 2024-03-14T12:14:33.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 -- 172.21.15.82:0/839692819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe59006e670 msgr2=0x7fe5900cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:33.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 --2- 172.21.15.82:0/839692819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe59006e670 0x7fe5900cd8f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5900d0950 tx=0x7fe58802e060 comp rx=0 tx=0).stop 2024-03-14T12:14:33.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 -- 172.21.15.82:0/839692819 shutdown_connections 2024-03-14T12:14:33.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 --2- 172.21.15.82:0/839692819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe59006e670 0x7fe5900cd8f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 --2- 172.21.15.82:0/839692819 >> v2:172.21.15.164:3300/0 conn(0x7fe5900b2860 0x7fe5900cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 -- 172.21.15.82:0/839692819 >> 172.21.15.82:0/839692819 conn(0x7fe5900b19b0 msgr2=0x7fe5900b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:33.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 -- 172.21.15.82:0/839692819 shutdown_connections 2024-03-14T12:14:33.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.700+0000 7fe597a43640 1 -- 172.21.15.82:0/839692819 wait complete. 2024-03-14T12:14:33.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.701+0000 7fe597a43640 1 Processor -- start 2024-03-14T12:14:33.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.701+0000 7fe597a43640 1 -- start start 2024-03-14T12:14:33.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe597a43640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe59006e670 0x7fe5901452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:33.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe597a43640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5900b2860 0x7fe5900ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:33.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe597a43640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5900cfd10 0x7fe590147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:33.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe597a43640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900b3720 con 0x7fe5900cfd10 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe597a43640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900b43c0 con 0x7fe5900b2860 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe597a43640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900b38a0 con 0x7fe59006e670 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5900cfd10 0x7fe590147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5900cfd10 0x7fe590147c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60226/0 (socket says 172.21.15.82:60226) 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 -- 172.21.15.82:0/1147057075 learned_addr learned my addr 172.21.15.82:0/1147057075 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe594fde640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5900b2860 0x7fe5900ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe5957df640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe59006e670 0x7fe5901452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 -- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe59006e670 msgr2=0x7fe5901452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe59006e670 0x7fe5901452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 -- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5900b2860 msgr2=0x7fe5900ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5900b2860 0x7fe5900ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 -- 172.21.15.82:0/1147057075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590158910 con 0x7fe5900cfd10 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe594fde640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5900b2860 0x7fe5900ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe595fe0640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5900cfd10 0x7fe590147c20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe588038040 tx=0x7fe588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe585ffb640 1 -- 172.21.15.82:0/1147057075 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588045020 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe585ffb640 1 -- 172.21.15.82:0/1147057075 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588040660 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590158bf0 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.702+0000 7fe585ffb640 1 -- 172.21.15.82:0/1147057075 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58802ea00 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.703+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5901591b0 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.704+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59004f050 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.704+0000 7fe585ffb640 1 -- 172.21.15.82:0/1147057075 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe588048020 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.704+0000 7fe585ffb640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe57406bbd0 0x7fe57406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.704+0000 7fe585ffb640 1 -- 172.21.15.82:0/1147057075 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fe5880af660 con 0x7fe5900cfd10 2024-03-14T12:14:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.704+0000 7fe5957df640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe57406bbd0 0x7fe57406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.705+0000 7fe5957df640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe57406bbd0 0x7fe57406e090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe58c004ea0 tx=0x7fe58c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:33.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.709+0000 7fe585ffb640 1 -- 172.21.15.82:0/1147057075 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe58804f030 con 0x7fe5900cfd10 2024-03-14T12:14:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:33.880+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5900cf290 con 0x7fe5900cfd10 2024-03-14T12:14:34.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.724+0000 7fe585ffb640 1 -- 172.21.15.82:0/1147057075 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe58807e070 con 0x7fe5900cfd10 2024-03-14T12:14:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-03-14T12:14:34.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe57406bbd0 msgr2=0x7fe57406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:34.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe57406bbd0 0x7fe57406e090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe58c004ea0 tx=0x7fe58c009040 comp rx=0 tx=0).stop 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5900cfd10 msgr2=0x7fe590147c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5900cfd10 0x7fe590147c20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe588038040 tx=0x7fe588004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 shutdown_connections 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5900cfd10 0x7fe590147c20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5900b2860 0x7fe5900ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe57406bbd0 0x7fe57406e090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 --2- 172.21.15.82:0/1147057075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe59006e670 0x7fe5901452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 >> 172.21.15.82:0/1147057075 conn(0x7fe5900b19b0 msgr2=0x7fe5900b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 shutdown_connections 2024-03-14T12:14:34.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.727+0000 7fe597a43640 1 -- 172.21.15.82:0/1147057075 wait complete. 2024-03-14T12:14:34.740 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T12:14:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.846+0000 7f6aa076f640 1 Processor -- start 2024-03-14T12:14:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.846+0000 7f6aa076f640 1 -- start start 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.847+0000 7f6aa076f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 0x7f6a98162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.847+0000 7f6aa076f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a980d08a0 con 0x7f6a98161a90 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.847+0000 7f6aa076f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a980d0b80 con 0x7f6a9806c400 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.847+0000 7f6aa076f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980d0e60 con 0x7f6a980b0bb0 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.847+0000 7f6a9e50b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a98161a90 0x7f6a98161e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35424/0 (socket says 172.21.15.82:35424) 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.847+0000 7f6a9e50b640 1 -- 172.21.15.82:0/2139993434 learned_addr learned my addr 172.21.15.82:0/2139993434 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972248061 0 0) 0x7f6a980d08a0 con 0x7f6a98161a90 2024-03-14T12:14:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a84003520 con 0x7f6a98161a90 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9dd0a640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 0x7f6a98162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2425760175 0 0) 0x7f6a84003520 con 0x7f6a98161a90 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 msgr2=0x7f6a98162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 0x7f6a98162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 >> v1:172.21.15.164:6789/0 conn(0x7f6a9806c400 legacy=0x7f6a9806c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9806bc00 con 0x7f6a98161a90 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6a940019f0 con 0x7f6a98161a90 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 >> v1:172.21.15.82:6789/0 conn(0x7f6a98161a90 legacy=0x7f6a98161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a840039e0 0x7f6a84003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84004800 0x7f6a84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 0x7f6a84005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980d0e60 con 0x7f6a840039e0 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8400bb20 con 0x7f6a84004800 2024-03-14T12:14:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8400bd20 con 0x7f6a980b0bb0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9e50b640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a840039e0 0x7f6a84003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9dd0a640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84004800 0x7f6a84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9ed0c640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 0x7f6a84005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9e50b640 1 -- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 msgr2=0x7f6a84005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9e50b640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 0x7f6a84005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9e50b640 1 -- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84004800 msgr2=0x7f6a84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9e50b640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84004800 0x7f6a84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9e50b640 1 -- 172.21.15.82:0/2139993434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a84003520 con 0x7f6a840039e0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.848+0000 7f6a9ed0c640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a980b0bb0 0x7f6a84005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.849+0000 7f6a9e50b640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a840039e0 0x7f6a84003e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400a850 tx=0x7f6a84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e760edd87bbe498e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.849+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400a4a0 con 0x7f6a840039e0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.849+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a94034650 con 0x7f6a840039e0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.849+0000 7f6a9d509640 1 -- 172.21.15.82:0/2139993434 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94034a40 con 0x7f6a840039e0 2024-03-14T12:14:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.850+0000 7f6aa076f640 1 -- 172.21.15.82:0/2139993434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a840039e0 msgr2=0x7f6a84003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.850+0000 7f6aa076f640 1 --2- 172.21.15.82:0/2139993434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a840039e0 0x7f6a84003e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400a850 tx=0x7f6a84005830 comp rx=0 tx=0).stop 2024-03-14T12:14:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.850+0000 7f6a9ed0c640 1 -- 172.21.15.82:0/2139993434 reap_dead start 2024-03-14T12:14:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.850+0000 7f6aa076f640 1 -- 172.21.15.82:0/2139993434 shutdown_connections 2024-03-14T12:14:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.850+0000 7f6aa076f640 1 -- 172.21.15.82:0/2139993434 >> 172.21.15.82:0/2139993434 conn(0x7f6a980b36a0 msgr2=0x7f6a980b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.850+0000 7f6aa076f640 1 -- 172.21.15.82:0/2139993434 shutdown_connections 2024-03-14T12:14:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.850+0000 7f6aa076f640 1 -- 172.21.15.82:0/2139993434 wait complete. 2024-03-14T12:14:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.851+0000 7f6aa076f640 1 Processor -- start 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.851+0000 7f6aa076f640 1 -- start start 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a840039e0 0x7f6a981f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a980c5790 0x7f6a9806d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a980c5c10 0x7f6a9806d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98162580 con 0x7f6a980c5c10 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980cfb40 con 0x7f6a980c5790 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98162700 con 0x7f6a840039e0 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a980c5c10 0x7f6a9806d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a980c5c10 0x7f6a9806d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60240/0 (socket says 172.21.15.82:60240) 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 -- 172.21.15.82:0/4039775440 learned_addr learned my addr 172.21.15.82:0/4039775440 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9e50b640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a840039e0 0x7f6a981f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9dd0a640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a980c5790 0x7f6a9806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 -- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a840039e0 msgr2=0x7f6a981f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a840039e0 0x7f6a981f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 -- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a980c5790 msgr2=0x7f6a9806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a980c5790 0x7f6a9806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 -- 172.21.15.82:0/4039775440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980c7580 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9dd0a640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a980c5790 0x7f6a9806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a9ed0c640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a980c5c10 0x7f6a9806d880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a84004780 tx=0x7f6a9000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a8effd640 1 -- 172.21.15.82:0/4039775440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90019070 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a8effd640 1 -- 172.21.15.82:0/4039775440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a900052e0 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a980c7860 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6a8effd640 1 -- 172.21.15.82:0/4039775440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90005490 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.852+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a980c7e20 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.853+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a980d29f0 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.854+0000 7f6a8effd640 1 -- 172.21.15.82:0/4039775440 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6a9001c070 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.854+0000 7f6a8effd640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a7c06bbd0 0x7f6a7c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.854+0000 7f6a8effd640 1 -- 172.21.15.82:0/4039775440 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f6a90089cf0 con 0x7f6a980c5c10 2024-03-14T12:14:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.854+0000 7f6a9e50b640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a7c06bbd0 0x7f6a7c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.855+0000 7f6a9e50b640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a7c06bbd0 0x7f6a7c06e090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a94034000 tx=0x7f6a94036040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:34.858+0000 7f6a8effd640 1 -- 172.21.15.82:0/4039775440 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6a980d29f0 con 0x7f6a980c5c10 2024-03-14T12:14:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.029+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a9804f050 con 0x7f6a980c5c10 2024-03-14T12:14:35.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.728+0000 7f6a8effd640 1 -- 172.21.15.82:0/4039775440 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6a9008e070 con 0x7f6a980c5c10 2024-03-14T12:14:35.728 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-03-14T12:14:35.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a7c06bbd0 msgr2=0x7f6a7c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a7c06bbd0 0x7f6a7c06e090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a94034000 tx=0x7f6a94036040 comp rx=0 tx=0).stop 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a980c5c10 msgr2=0x7f6a9806d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a980c5c10 0x7f6a9806d880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a84004780 tx=0x7f6a9000e820 comp rx=0 tx=0).stop 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 shutdown_connections 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a980c5c10 0x7f6a9806d880 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a980c5790 0x7f6a9806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a7c06bbd0 0x7f6a7c06e090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 --2- 172.21.15.82:0/4039775440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a840039e0 0x7f6a981f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.731+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 >> 172.21.15.82:0/4039775440 conn(0x7f6a980b36a0 msgr2=0x7f6a980b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.732+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 shutdown_connections 2024-03-14T12:14:35.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:35.732+0000 7f6aa076f640 1 -- 172.21.15.82:0/4039775440 wait complete. 2024-03-14T12:14:35.746 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T12:14:35.746 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-14T12:14:35.746 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T12:14:35.746 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:14:35.746 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T12:14:35.747 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-14T12:14:35.747 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T12:14:35.747 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:14:35.747 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T12:14:35.747 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-14T12:14:35.747 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T12:14:35.748 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:14:35.748 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T12:14:35.748 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-14T12:14:35.748 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T12:14:35.748 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:14:35.758 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T12:14:35.758 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T12:14:35.761 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T12:14:35.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T12:14:35.762 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T12:14:35.762 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T12:14:35.800 INFO:tasks.ceph.mds.c:Started 2024-03-14T12:14:35.801 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:35.804 INFO:tasks.ceph.mds.d:Started 2024-03-14T12:14:35.804 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:35.804 INFO:tasks.ceph.mds.b:Started 2024-03-14T12:14:35.805 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:35.883 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:14:35.893 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-03-14T12:14:35.893 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-03-14T12:14:41.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-03-14T12:14:41.758 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T12:14:41.758 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T12:14:41.761 INFO:tasks.ceph.mds.a:Started 2024-03-14T12:14:41.762 DEBUG:teuthology.parallel:result is None 2024-03-14T12:14:41.762 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-14T12:14:41.848 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-14T12:14:41.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.875+0000 7f3f2932b640 1 Processor -- start 2024-03-14T12:14:41.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.875+0000 7f3f2932b640 1 -- start start 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f2932b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f240cf370 0x7f3f240d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f2932b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f240cf8c0 con 0x7f3f240d0760 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f2932b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f240cfba0 con 0x7f3f240b7720 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f2932b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f240b6c40 con 0x7f3f240cf370 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f227fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f240cf370 0x7f3f240d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f227fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f240cf370 0x7f3f240d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37448/0 (socket says 172.21.15.82:37448) 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f22ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3f240d0760 0x7f3f240d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38544/0 (socket says 172.21.15.82:38544) 2024-03-14T12:14:41.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.876+0000 7f3f227fc640 1 -- 172.21.15.82:0/4213622600 learned_addr learned my addr 172.21.15.82:0/4213622600 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:41.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f227fc640 1 -- 172.21.15.82:0/4213622600 >> v1:172.21.15.164:6789/0 conn(0x7f3f240b7720 legacy=0x7f3f240b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:41.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f227fc640 1 -- 172.21.15.82:0/4213622600 >> v1:172.21.15.82:6789/0 conn(0x7f3f240d0760 legacy=0x7f3f240d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:41.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f227fc640 1 -- 172.21.15.82:0/4213622600 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240b6f20 con 0x7f3f240cf370 2024-03-14T12:14:41.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f21ffb640 1 -- 172.21.15.82:0/4213622600 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53408436 0 0) 0x7f3f240cf8c0 con 0x7f3f240d0760 2024-03-14T12:14:41.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f227fc640 1 --2- 172.21.15.82:0/4213622600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240cf370 0x7f3f240d4660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f1402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c324f542523ecf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:41.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f21ffb640 1 -- 172.21.15.82:0/4213622600 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1402f840 con 0x7f3f240cf370 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f21ffb640 1 -- 172.21.15.82:0/4213622600 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1402f9e0 con 0x7f3f240cf370 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.877+0000 7f3f21ffb640 1 -- 172.21.15.82:0/4213622600 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1402fdb0 con 0x7f3f240cf370 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.878+0000 7f3f2932b640 1 -- 172.21.15.82:0/4213622600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240cf370 msgr2=0x7f3f240d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.878+0000 7f3f2932b640 1 --2- 172.21.15.82:0/4213622600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240cf370 0x7f3f240d4660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f1402ba20 comp rx=0 tx=0).stop 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.878+0000 7f3f2932b640 1 -- 172.21.15.82:0/4213622600 shutdown_connections 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.878+0000 7f3f2932b640 1 --2- 172.21.15.82:0/4213622600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240cf370 0x7f3f240d4660 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.878+0000 7f3f2932b640 1 -- 172.21.15.82:0/4213622600 >> 172.21.15.82:0/4213622600 conn(0x7f3f2409d030 msgr2=0x7f3f240d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.878+0000 7f3f2932b640 1 -- 172.21.15.82:0/4213622600 shutdown_connections 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.878+0000 7f3f2932b640 1 -- 172.21.15.82:0/4213622600 wait complete. 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.879+0000 7f3f2932b640 1 Processor -- start 2024-03-14T12:14:41.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.879+0000 7f3f2932b640 1 -- start start 2024-03-14T12:14:41.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f2932b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240b7720 0x7f3f240cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:41.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f2932b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f240cf370 0x7f3f240cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:41.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f2932b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f240d0760 0x7f3f2415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:41.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f2932b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241617a0 con 0x7f3f240b7720 2024-03-14T12:14:41.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f2932b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240d4ba0 con 0x7f3f240d0760 2024-03-14T12:14:41.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f2932b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24161920 con 0x7f3f240cf370 2024-03-14T12:14:41.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240b7720 0x7f3f240cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240b7720 0x7f3f240cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37462/0 (socket says 172.21.15.82:37462) 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f22ffd640 1 -- 172.21.15.82:0/2178531215 learned_addr learned my addr 172.21.15.82:0/2178531215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.880+0000 7f3f227fc640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f240cf370 0x7f3f240cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f237fe640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f240d0760 0x7f3f2415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f22ffd640 1 -- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f240cf370 msgr2=0x7f3f240cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f22ffd640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f240cf370 0x7f3f240cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f22ffd640 1 -- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f240d0760 msgr2=0x7f3f2415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f22ffd640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f240d0760 0x7f3f2415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f22ffd640 1 -- 172.21.15.82:0/2178531215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240cc770 con 0x7f3f240b7720 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f237fe640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f240d0760 0x7f3f2415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f22ffd640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240b7720 0x7f3f240cd810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c430 tx=0x7f3f1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:41.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f0b7fe640 1 -- 172.21.15.82:0/2178531215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f180074e0 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f0b7fe640 1 -- 172.21.15.82:0/2178531215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f180047e0 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f0b7fe640 1 -- 172.21.15.82:0/2178531215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18004bd0 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f240cca50 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.881+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f240cd010 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.882+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2404f050 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.882+0000 7f3f0b7fe640 1 -- 172.21.15.82:0/2178531215 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3f18011070 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.883+0000 7f3f0b7fe640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f1006bbd0 0x7f3f1006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.883+0000 7f3f0b7fe640 1 -- 172.21.15.82:0/2178531215 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f1808a3f0 con 0x7f3f240b7720 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.883+0000 7f3f227fc640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f1006bbd0 0x7f3f1006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:41.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.883+0000 7f3f227fc640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f1006bbd0 0x7f3f1006e090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f14036000 tx=0x7f3f14008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:41.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:41.888+0000 7f3f0b7fe640 1 -- 172.21.15.82:0/2178531215 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3f180591e0 con 0x7f3f240b7720 2024-03-14T12:14:42.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:42.070+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3f240d0540 con 0x7f3f240b7720 2024-03-14T12:14:43.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.753+0000 7f3f0b7fe640 1 -- 172.21.15.82:0/2178531215 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3f18058ae0 con 0x7f3f240b7720 2024-03-14T12:14:43.753 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f1006bbd0 msgr2=0x7f3f1006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f1006bbd0 0x7f3f1006e090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f14036000 tx=0x7f3f14008030 comp rx=0 tx=0).stop 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240b7720 msgr2=0x7f3f240cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240b7720 0x7f3f240cd810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c430 tx=0x7f3f1800c8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 shutdown_connections 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f240d0760 0x7f3f2415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3f1006bbd0 0x7f3f1006e090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f240cf370 0x7f3f240cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 --2- 172.21.15.82:0/2178531215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f240b7720 0x7f3f240cd810 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 >> 172.21.15.82:0/2178531215 conn(0x7f3f2409d030 msgr2=0x7f3f240d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:43.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 shutdown_connections 2024-03-14T12:14:43.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.756+0000 7f3f2932b640 1 -- 172.21.15.82:0/2178531215 wait complete. 2024-03-14T12:14:43.786 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-14T12:14:43.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.888+0000 7f46eda44640 1 Processor -- start 2024-03-14T12:14:43.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.888+0000 7f46eda44640 1 -- start start 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46eda44640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46e815b9e0 0x7f46e80b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46eda44640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e815c250 con 0x7f46e815bdc0 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46eda44640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e80cd810 con 0x7f46e80cd0d0 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46eda44640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e80cf730 con 0x7f46e815b9e0 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46e6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46e815b9e0 0x7f46e80b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46e67fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f46e815bdc0 0x7f46e80d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38548/0 (socket says 172.21.15.82:38548) 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46e67fc640 1 -- 172.21.15.82:0/3574044916 learned_addr learned my addr 172.21.15.82:0/3574044916 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46e6ffd640 1 -- 172.21.15.82:0/3574044916 >> v1:172.21.15.164:6789/0 conn(0x7f46e80cd0d0 legacy=0x7f46e815f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46e6ffd640 1 -- 172.21.15.82:0/3574044916 >> v1:172.21.15.82:6789/0 conn(0x7f46e815bdc0 legacy=0x7f46e80d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46e6ffd640 1 -- 172.21.15.82:0/3574044916 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80cf980 con 0x7f46e815b9e0 2024-03-14T12:14:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.889+0000 7f46e5ffb640 1 -- 172.21.15.82:0/3574044916 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389279337 0 0) 0x7f46e815c250 con 0x7f46e815bdc0 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.890+0000 7f46e6ffd640 1 --2- 172.21.15.82:0/3574044916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80b4190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f46d80097a0 tx=0x7f46d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a88cfd30dfe386 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.890+0000 7f46e5ffb640 1 -- 172.21.15.82:0/3574044916 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80315a0 con 0x7f46e815b9e0 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.890+0000 7f46e5ffb640 1 -- 172.21.15.82:0/3574044916 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8037070 con 0x7f46e815b9e0 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.890+0000 7f46e5ffb640 1 -- 172.21.15.82:0/3574044916 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8031960 con 0x7f46e815b9e0 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.891+0000 7f46eda44640 1 -- 172.21.15.82:0/3574044916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 msgr2=0x7f46e80b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.891+0000 7f46eda44640 1 --2- 172.21.15.82:0/3574044916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80b4190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f46d80097a0 tx=0x7f46d802b840 comp rx=0 tx=0).stop 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.891+0000 7f46eda44640 1 -- 172.21.15.82:0/3574044916 shutdown_connections 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.891+0000 7f46eda44640 1 --2- 172.21.15.82:0/3574044916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80b4190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.891+0000 7f46eda44640 1 -- 172.21.15.82:0/3574044916 >> 172.21.15.82:0/3574044916 conn(0x7f46e80b3670 msgr2=0x7f46e80b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.891+0000 7f46eda44640 1 -- 172.21.15.82:0/3574044916 shutdown_connections 2024-03-14T12:14:43.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.891+0000 7f46eda44640 1 -- 172.21.15.82:0/3574044916 wait complete. 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.892+0000 7f46eda44640 1 Processor -- start 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.892+0000 7f46eda44640 1 -- start start 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46eda44640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46e80cd0d0 0x7f46e80c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46eda44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46eda44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e815bdc0 0x7f46e80aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46eda44640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80cd520 con 0x7f46e815b9e0 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46eda44640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80cff40 con 0x7f46e815bdc0 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46eda44640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8160050 con 0x7f46e80cd0d0 2024-03-14T12:14:43.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37474/0 (socket says 172.21.15.82:37474) 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 -- 172.21.15.82:0/1535936057 learned_addr learned my addr 172.21.15.82:0/1535936057 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e77fe640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e815bdc0 0x7f46e80aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e6ffd640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46e80cd0d0 0x7f46e80c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 -- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46e80cd0d0 msgr2=0x7f46e80c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46e80cd0d0 0x7f46e80c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 -- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e815bdc0 msgr2=0x7f46e80aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e815bdc0 0x7f46e80aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 -- 172.21.15.82:0/1535936057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e815c250 con 0x7f46e815b9e0 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.893+0000 7f46e67fc640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80afc90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c4d0 tx=0x7f46dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.894+0000 7f46cffff640 1 -- 172.21.15.82:0/1535936057 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc018070 con 0x7f46e815b9e0 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.894+0000 7f46cffff640 1 -- 172.21.15.82:0/1535936057 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc007500 con 0x7f46e815b9e0 2024-03-14T12:14:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.894+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e815c510 con 0x7f46e815b9e0 2024-03-14T12:14:43.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.894+0000 7f46cffff640 1 -- 172.21.15.82:0/1535936057 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc004a20 con 0x7f46e815b9e0 2024-03-14T12:14:43.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.894+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e815c7f0 con 0x7f46e815b9e0 2024-03-14T12:14:43.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.895+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e804f050 con 0x7f46e815b9e0 2024-03-14T12:14:43.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.895+0000 7f46cffff640 1 -- 172.21.15.82:0/1535936057 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f46dc01b070 con 0x7f46e815b9e0 2024-03-14T12:14:43.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.895+0000 7f46cffff640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f46d406bbd0 0x7f46d406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.896+0000 7f46cffff640 1 -- 172.21.15.82:0/1535936057 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f46dc089990 con 0x7f46e815b9e0 2024-03-14T12:14:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.896+0000 7f46e6ffd640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f46d406bbd0 0x7f46d406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.896+0000 7f46e6ffd640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f46d406bbd0 0x7f46d406e090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46d8036000 tx=0x7f46d803d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:43.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:43.900+0000 7f46cffff640 1 -- 172.21.15.82:0/1535936057 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f46dc058860 con 0x7f46e815b9e0 2024-03-14T12:14:44.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.138+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f46e80b7ae0 con 0x7f46e815b9e0 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.138+0000 7f46cffff640 1 -- 172.21.15.82:0/1535936057 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f46dc058160 con 0x7f46e815b9e0 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlAQmRIBAA45IMoIGH+bLaSIro/QUEmQ==", 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.139 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJljqfOLBAAulbwLxzEv+lHJOPUy3p4BQ==", 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.140 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlt7bdJxAAH34bGUjVrlElC4nA11eNIQ==", 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.141 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlmeIuNBAAzNKyG5CdCVCURsWZywpTnw==", 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDJ6fJlKwo/HBAA/Ma3FBjIdO1LF3RySRD0DQ==", 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.142 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDM6fJlot+1GhAAEQqFqku2rdbqZ3rEBfVO2g==", 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.143 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDP6fJlPRIYGRAAUxqEbSYn6AhIE+3lXEqEOg==", 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDS6fJlEYhJGBAApIl3qciLHwIJzDq5J2NkqA==", 2024-03-14T12:14:44.144 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDY6fJlL2cQHxAAeHY/pdO0TvpUciB22xvOUg==", 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.145 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDb6fJlrNxAHRAALwT6cIpCh9fE4NMoqV6PWA==", 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDe6fJlPLi2GxAAeZ/Nk/340P6Y0ZJYojfNMw==", 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.146 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDh6fJlLhW0GhAAtvxrvm93pbGKTJbzzpIRiQ==", 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.147 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg==", 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlUlkBBhAA9QGPbDVg8uWX5WTR2W936g==", 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.148 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlDQeZAhAAJW1K+HBJuwgzu4nVr8Mq8g==", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:14:44.149 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJldFHSCxAAci5DVZh9406uu1BZIAZXiw==", 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.150 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlhoD/AxAA/LRLwtPkAs14z0fLh+eGjA==", 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-mds", 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDn6fJlxjRUFhAALO7jhJYBBFXxVONLOh/iLQ==", 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.151 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-mds" 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-mgr", 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDn6fJlvT1UFhAAqE1TKGqu3geCfWNF47gsHA==", 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-osd", 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDn6fJlrERUFhAAOEU4BdVWOncb1MJhA6vIFA==", 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-osd" 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rbd", 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDn6fJlYEtUFhAAWWhUpAmVM17dT5tHNtZJew==", 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDn6fJlEFJUFhAA8ldOtv4FJvrO3kfH7iWi7g==", 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-14T12:14:44.153 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rgw", 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDn6fJlE1lUFhAAfsXf/tyBA7/sd2Wh1km75Q==", 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJl5qEWHRAAI8LqGdvUwjHG/5pNgcAB1A==", 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.154 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlxtwQGRAArcSgLFwgKjmSqjq0sst8Gw==", 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:44.155 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:14:44.156 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:14:44.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f46d406bbd0 msgr2=0x7f46d406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f46d406bbd0 0x7f46d406e090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46d8036000 tx=0x7f46d803d040 comp rx=0 tx=0).stop 2024-03-14T12:14:44.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 msgr2=0x7f46e80afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80afc90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c4d0 tx=0x7f46dc00c990 comp rx=0 tx=0).stop 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 shutdown_connections 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e815bdc0 0x7f46e80aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e815b9e0 0x7f46e80afc90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f46d406bbd0 0x7f46d406e090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 --2- 172.21.15.82:0/1535936057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46e80cd0d0 0x7f46e80c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 >> 172.21.15.82:0/1535936057 conn(0x7f46e80b3670 msgr2=0x7f46e806f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.140+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 shutdown_connections 2024-03-14T12:14:44.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.141+0000 7f46eda44640 1 -- 172.21.15.82:0/1535936057 wait complete. 2024-03-14T12:14:44.159 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-14T12:14:44.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.260+0000 7f13cc8a7640 1 Processor -- start 2024-03-14T12:14:44.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.260+0000 7f13cc8a7640 1 -- start start 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13cc8a7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f13c80b4b70 0x7f13c8070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13cc8a7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80b3da0 con 0x7f13c8151230 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13cc8a7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80b4080 con 0x7f13c815dde0 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13cc8a7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c806c3b0 con 0x7f13c80b4b70 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c5d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f13c80b4b70 0x7f13c8070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f13c8151230 0x7f13c80cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38558/0 (socket says 172.21.15.82:38558) 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c659c640 1 -- 172.21.15.82:0/4270505680 learned_addr learned my addr 172.21.15.82:0/4270505680 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c559a640 1 -- 172.21.15.82:0/4270505680 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971404588 0 0) 0x7f13c80b3da0 con 0x7f13c8151230 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c559a640 1 -- 172.21.15.82:0/4270505680 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13b0003440 con 0x7f13c8151230 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c5d9b640 1 -- 172.21.15.82:0/4270505680 >> v1:172.21.15.164:6789/0 conn(0x7f13c815dde0 legacy=0x7f13c806cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c5d9b640 1 -- 172.21.15.82:0/4270505680 >> v1:172.21.15.82:6789/0 conn(0x7f13c8151230 legacy=0x7f13c80cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.261+0000 7f13c5d9b640 1 -- 172.21.15.82:0/4270505680 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80b4360 con 0x7f13c80b4b70 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.262+0000 7f13c5d9b640 1 --2- 172.21.15.82:0/4270505680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c8070630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f13c806c530 tx=0x7f13bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f8afc811d8eea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.262+0000 7f13c559a640 1 -- 172.21.15.82:0/4270505680 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc035020 con 0x7f13c80b4b70 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.262+0000 7f13c559a640 1 -- 172.21.15.82:0/4270505680 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc031830 con 0x7f13c80b4b70 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.262+0000 7f13c559a640 1 -- 172.21.15.82:0/4270505680 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc031c00 con 0x7f13c80b4b70 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.263+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4270505680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 msgr2=0x7f13c8070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.263+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4270505680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c8070630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f13c806c530 tx=0x7f13bc02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.263+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4270505680 shutdown_connections 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.263+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4270505680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c8070630 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.263+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4270505680 >> 172.21.15.82:0/4270505680 conn(0x7f13c80b1c70 msgr2=0x7f13c80cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.263+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4270505680 shutdown_connections 2024-03-14T12:14:44.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.263+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4270505680 wait complete. 2024-03-14T12:14:44.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.264+0000 7f13cc8a7640 1 Processor -- start 2024-03-14T12:14:44.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.264+0000 7f13cc8a7640 1 -- start start 2024-03-14T12:14:44.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13cc8a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c80ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13cc8a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13c8151230 0x7f13c81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13cc8a7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13c815dde0 0x7f13c8138e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13cc8a7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8070b70 con 0x7f13c80b4b70 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13cc8a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c806c690 con 0x7f13c8151230 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13cc8a7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8070cf0 con 0x7f13c815dde0 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c80ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c80ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37492/0 (socket says 172.21.15.82:37492) 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 -- 172.21.15.82:0/4258207130 learned_addr learned my addr 172.21.15.82:0/4258207130 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c5d9b640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13c8151230 0x7f13c81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c6d9d640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13c815dde0 0x7f13c8138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 -- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13c815dde0 msgr2=0x7f13c8138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13c815dde0 0x7f13c8138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 -- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13c8151230 msgr2=0x7f13c81502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13c8151230 0x7f13c81502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 -- 172.21.15.82:0/4258207130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80cd840 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.265+0000 7f13c659c640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c80ce8e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f13b800cb40 tx=0x7f13b800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.266+0000 7f13b6ffd640 1 -- 172.21.15.82:0/4258207130 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8017070 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.266+0000 7f13b6ffd640 1 -- 172.21.15.82:0/4258207130 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b800aca0 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.266+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c80cdb20 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.266+0000 7f13b6ffd640 1 -- 172.21.15.82:0/4258207130 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8012620 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.266+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c80ce3c0 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.267+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c80cdcf0 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.267+0000 7f13b6ffd640 1 -- 172.21.15.82:0/4258207130 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f13b801a070 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.267+0000 7f13b6ffd640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f13a806bbd0 0x7f13a806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.267+0000 7f13b6ffd640 1 -- 172.21.15.82:0/4258207130 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f13b808a5d0 con 0x7f13c80b4b70 2024-03-14T12:14:44.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.267+0000 7f13c5d9b640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f13a806bbd0 0x7f13a806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.268+0000 7f13c5d9b640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f13a806bbd0 0x7f13a806e090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13bc036000 tx=0x7f13bc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:44.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.272+0000 7f13b6ffd640 1 -- 172.21.15.82:0/4258207130 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f13b8059410 con 0x7f13c80b4b70 2024-03-14T12:14:44.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.510+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f13c804f050 con 0x7f13c80b4b70 2024-03-14T12:14:44.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.511+0000 7f13b6ffd640 1 -- 172.21.15.82:0/4258207130 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f13b8058d10 con 0x7f13c80b4b70 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f13a806bbd0 msgr2=0x7f13a806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f13a806bbd0 0x7f13a806e090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13bc036000 tx=0x7f13bc008030 comp rx=0 tx=0).stop 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 msgr2=0x7f13c80ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c80ce8e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f13b800cb40 tx=0x7f13b800a060 comp rx=0 tx=0).stop 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 shutdown_connections 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f13a806bbd0 0x7f13a806e090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13c815dde0 0x7f13c8138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13c8151230 0x7f13c81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 --2- 172.21.15.82:0/4258207130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c80b4b70 0x7f13c80ce8e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 >> 172.21.15.82:0/4258207130 conn(0x7f13c80b1c70 msgr2=0x7f13c815a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:44.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 shutdown_connections 2024-03-14T12:14:44.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.514+0000 7f13cc8a7640 1 -- 172.21.15.82:0/4258207130 wait complete. 2024-03-14T12:14:44.525 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-14T12:14:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f034aca7640 1 Processor -- start 2024-03-14T12:14:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f034aca7640 1 -- start start 2024-03-14T12:14:44.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f034aca7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f034415b9e0 0x7f03440b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f034aca7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f034415bdc0 0x7f03440cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f034aca7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034415c250 con 0x7f03440d0440 2024-03-14T12:14:44.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f034aca7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03440cf730 con 0x7f034415b9e0 2024-03-14T12:14:44.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f034aca7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440cd810 con 0x7f034415bdc0 2024-03-14T12:14:44.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f0348a43640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f034415b9e0 0x7f03440b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f0348a43640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f034415b9e0 0x7f03440b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37504/0 (socket says 172.21.15.82:37504) 2024-03-14T12:14:44.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.626+0000 7f0348a43640 1 -- 172.21.15.82:0/123712207 learned_addr learned my addr 172.21.15.82:0/123712207 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:44.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.627+0000 7f0348a43640 1 -- 172.21.15.82:0/123712207 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f034415bdc0 msgr2=0x7f03440cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:44.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.627+0000 7f0348a43640 1 --2- 172.21.15.82:0/123712207 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f034415bdc0 0x7f03440cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.627+0000 7f0348a43640 1 -- 172.21.15.82:0/123712207 >> v1:172.21.15.164:6789/0 conn(0x7f03440d0440 legacy=0x7f034415f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:44.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.627+0000 7f0348a43640 1 -- 172.21.15.82:0/123712207 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440cf980 con 0x7f034415b9e0 2024-03-14T12:14:44.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.627+0000 7f0348a43640 1 --2- 172.21.15.82:0/123712207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f03440b4190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368da9b5c2cbebde server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:44.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f03437fe640 1 -- 172.21.15.82:0/123712207 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c02c9e0 con 0x7f034415b9e0 2024-03-14T12:14:44.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f03437fe640 1 -- 172.21.15.82:0/123712207 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c02cb80 con 0x7f034415b9e0 2024-03-14T12:14:44.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f03437fe640 1 -- 172.21.15.82:0/123712207 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0343f0 con 0x7f034415b9e0 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 -- 172.21.15.82:0/123712207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 msgr2=0x7f03440b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 --2- 172.21.15.82:0/123712207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f03440b4190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c02be40 comp rx=0 tx=0).stop 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 -- 172.21.15.82:0/123712207 shutdown_connections 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 --2- 172.21.15.82:0/123712207 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f034415bdc0 0x7f03440cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 --2- 172.21.15.82:0/123712207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f03440b4190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 -- 172.21.15.82:0/123712207 >> 172.21.15.82:0/123712207 conn(0x7f03440b3670 msgr2=0x7f03440b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 -- 172.21.15.82:0/123712207 shutdown_connections 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.628+0000 7f034aca7640 1 -- 172.21.15.82:0/123712207 wait complete. 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.629+0000 7f034aca7640 1 Processor -- start 2024-03-14T12:14:44.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.629+0000 7f034aca7640 1 -- start start 2024-03-14T12:14:44.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f034aca7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03440d0440 0x7f0344153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f034aca7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f0344152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f034aca7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034415bdc0 0x7f034406be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f034aca7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0344160050 con 0x7f034415b9e0 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f034aca7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440cff40 con 0x7f034415bdc0 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f034aca7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03441601d0 con 0x7f03440d0440 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f0343fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f0344152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f0343fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f0344152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37512/0 (socket says 172.21.15.82:37512) 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f0343fff640 1 -- 172.21.15.82:0/968985719 learned_addr learned my addr 172.21.15.82:0/968985719 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f0348a43640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03440d0440 0x7f0344153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.630+0000 7f0349244640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034415bdc0 0x7f034406be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f0343fff640 1 -- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03440d0440 msgr2=0x7f0344153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f0343fff640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03440d0440 0x7f0344153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f0343fff640 1 -- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034415bdc0 msgr2=0x7f034406be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f0343fff640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034415bdc0 0x7f034406be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f0343fff640 1 -- 172.21.15.82:0/968985719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440c5630 con 0x7f034415b9e0 2024-03-14T12:14:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f0343fff640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f0344152bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b4a0 tx=0x7f032c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f03417fa640 1 -- 172.21.15.82:0/968985719 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c014070 con 0x7f034415b9e0 2024-03-14T12:14:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f03417fa640 1 -- 172.21.15.82:0/968985719 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c010610 con 0x7f034415b9e0 2024-03-14T12:14:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0344153100 con 0x7f034415b9e0 2024-03-14T12:14:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f03417fa640 1 -- 172.21.15.82:0/968985719 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c010a00 con 0x7f034415b9e0 2024-03-14T12:14:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.631+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03440b0b60 con 0x7f034415b9e0 2024-03-14T12:14:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.632+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034404f050 con 0x7f034415b9e0 2024-03-14T12:14:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.633+0000 7f03417fa640 1 -- 172.21.15.82:0/968985719 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f032c017070 con 0x7f034415b9e0 2024-03-14T12:14:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.633+0000 7f03417fa640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f032406bbd0 0x7f032406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.633+0000 7f03417fa640 1 -- 172.21.15.82:0/968985719 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f032c08a130 con 0x7f034415b9e0 2024-03-14T12:14:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.633+0000 7f0348a43640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f032406bbd0 0x7f032406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.633+0000 7f0348a43640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f032406bbd0 0x7f032406e090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f033c035000 tx=0x7f033c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.637+0000 7f03417fa640 1 -- 172.21.15.82:0/968985719 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f032c058890 con 0x7f034415b9e0 2024-03-14T12:14:44.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.875+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f034415ac90 con 0x7f034415b9e0 2024-03-14T12:14:44.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.877+0000 7f03417fa640 1 -- 172.21.15.82:0/968985719 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f032c05c930 con 0x7f034415b9e0 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.878+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f032406bbd0 msgr2=0x7f032406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.878+0000 7f034aca7640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f032406bbd0 0x7f032406e090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f033c035000 tx=0x7f033c030000 comp rx=0 tx=0).stop 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.878+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 msgr2=0x7f0344152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.878+0000 7f034aca7640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f0344152bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b4a0 tx=0x7f032c00b960 comp rx=0 tx=0).stop 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 shutdown_connections 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f034415bdc0 0x7f034406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f034415b9e0 0x7f0344152bc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f032406bbd0 0x7f032406e090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 --2- 172.21.15.82:0/968985719 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03440d0440 0x7f0344153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 >> 172.21.15.82:0/968985719 conn(0x7f03440b3670 msgr2=0x7f0344159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 shutdown_connections 2024-03-14T12:14:44.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.879+0000 7f034aca7640 1 -- 172.21.15.82:0/968985719 wait complete. 2024-03-14T12:14:44.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-14T12:14:44.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.992+0000 7f9d145b2640 1 Processor -- start 2024-03-14T12:14:44.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.992+0000 7f9d145b2640 1 -- start start 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d145b2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9d0c1551d0 0x7f9d0c0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d145b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 0x7f9d0c161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d145b2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0b60f0 con 0x7f9d0c161fc0 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d145b2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c070340 con 0x7f9d0c1551d0 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d145b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b6d00 con 0x7f9d0c0b5bb0 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d12b4f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9d0c161fc0 0x7f9d0c15e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38572/0 (socket says 172.21.15.82:38572) 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d12b4f640 1 -- 172.21.15.82:0/3477277657 learned_addr learned my addr 172.21.15.82:0/3477277657 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901686720 0 0) 0x7f9d0c0b60f0 con 0x7f9d0c161fc0 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf8003520 con 0x7f9d0c161fc0 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1234e640 1 --2- 172.21.15.82:0/3477277657 >> v2:172.21.15.164:3300/0 conn(0x7f9d0c1551d0 0x7f9d0c0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d11b4d640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 0x7f9d0c161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809128444 0 0) 0x7f9cf8003520 con 0x7f9d0c161fc0 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 msgr2=0x7f9d0c161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 0x7f9d0c161a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 >> v2:172.21.15.164:3300/0 conn(0x7f9d0c1551d0 msgr2=0x7f9d0c0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 --2- 172.21.15.82:0/3477277657 >> v2:172.21.15.164:3300/0 conn(0x7f9d0c1551d0 0x7f9d0c0b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0b63d0 con 0x7f9d0c161fc0 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f9d040019f0 con 0x7f9d0c161fc0 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1234e640 1 --2- 172.21.15.82:0/3477277657 >> v2:172.21.15.164:3300/0 conn(0x7f9d0c1551d0 0x7f9d0c0b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 >> v1:172.21.15.82:6789/0 conn(0x7f9d0c161fc0 legacy=0x7f9d0c15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf80039b0 0x7f9cf8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf8004800 0x7f9cf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 0x7f9cf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c070340 con 0x7f9cf8004800 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf800b9c0 con 0x7f9cf80039b0 2024-03-14T12:14:44.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.993+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf800bb40 con 0x7f9d0c0b5bb0 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d11b4d640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf8004800 0x7f9cf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d1234e640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf80039b0 0x7f9cf8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d12b4f640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 0x7f9cf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d11b4d640 1 -- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 msgr2=0x7f9cf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d11b4d640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 0x7f9cf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d11b4d640 1 -- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf80039b0 msgr2=0x7f9cf8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d11b4d640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf80039b0 0x7f9cf8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d11b4d640 1 -- 172.21.15.82:0/3477277657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8003520 con 0x7f9cf8004800 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d12b4f640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d0c0b5bb0 0x7f9cf80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.994+0000 7f9d11b4d640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf8004800 0x7f9cf8004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ac0 tx=0x7f9cf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63712845a652e2d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc039070 con 0x7f9cf8004800 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc02c6b0 con 0x7f9cf8004800 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d1134c640 1 -- 172.21.15.82:0/3477277657 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc009690 con 0x7f9cf8004800 2024-03-14T12:14:44.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d145b2640 1 -- 172.21.15.82:0/3477277657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf8004800 msgr2=0x7f9cf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:44.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d145b2640 1 --2- 172.21.15.82:0/3477277657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf8004800 0x7f9cf8004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ac0 tx=0x7f9cf8005830 comp rx=0 tx=0).stop 2024-03-14T12:14:44.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d12b4f640 1 -- 172.21.15.82:0/3477277657 reap_dead start 2024-03-14T12:14:44.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d145b2640 1 -- 172.21.15.82:0/3477277657 shutdown_connections 2024-03-14T12:14:44.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d145b2640 1 -- 172.21.15.82:0/3477277657 >> 172.21.15.82:0/3477277657 conn(0x7f9d0c072ee0 msgr2=0x7f9d0c09a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:44.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d145b2640 1 -- 172.21.15.82:0/3477277657 shutdown_connections 2024-03-14T12:14:44.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.995+0000 7f9d145b2640 1 -- 172.21.15.82:0/3477277657 wait complete. 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.996+0000 7f9d145b2640 1 Processor -- start 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.996+0000 7f9d145b2640 1 -- start start 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d145b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf8004800 0x7f9d0c153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d145b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d0c1e8290 0x7f9d0c153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d145b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c0ccba0 0x7f9d0c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d145b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c161a80 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d145b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b6990 con 0x7f9d0c1e8290 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d145b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c161c00 con 0x7f9cf8004800 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c0ccba0 0x7f9d0c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c0ccba0 0x7f9d0c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37530/0 (socket says 172.21.15.82:37530) 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 -- 172.21.15.82:0/2340409258 learned_addr learned my addr 172.21.15.82:0/2340409258 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d11b4d640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d0c1e8290 0x7f9d0c153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d1234e640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf8004800 0x7f9d0c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 -- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf8004800 msgr2=0x7f9d0c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf8004800 0x7f9d0c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 -- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d0c1e8290 msgr2=0x7f9d0c153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d0c1e8290 0x7f9d0c153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.997+0000 7f9d12b4f640 1 -- 172.21.15.82:0/2340409258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0cf230 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.998+0000 7f9d12b4f640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c0ccba0 0x7f9d0c0ccff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c162b40 tx=0x7f9d0400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.998+0000 7f9d02ffd640 1 -- 172.21.15.82:0/2340409258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04007a80 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.998+0000 7f9d02ffd640 1 -- 172.21.15.82:0/2340409258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04018070 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.998+0000 7f9d02ffd640 1 -- 172.21.15.82:0/2340409258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04013420 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.998+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c1e8720 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.998+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c1e8ce0 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.999+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c04f050 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:44.999+0000 7f9d02ffd640 1 -- 172.21.15.82:0/2340409258 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9d040135c0 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.000+0000 7f9d02ffd640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9cf006bc00 0x7f9cf006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.000+0000 7f9d02ffd640 1 -- 172.21.15.82:0/2340409258 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d0408ab50 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.000+0000 7f9d1234e640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9cf006bc00 0x7f9cf006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.000+0000 7f9d1234e640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9cf006bc00 0x7f9cf006e0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0b4550 tx=0x7f9d08008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.004+0000 7f9d02ffd640 1 -- 172.21.15.82:0/2340409258 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9d0c0cf230 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.240+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9d0c0b6ff0 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.243+0000 7f9d02ffd640 1 -- 172.21.15.82:0/2340409258 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9d04059990 con 0x7f9d0c0ccba0 2024-03-14T12:14:45.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9cf006bc00 msgr2=0x7f9cf006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9cf006bc00 0x7f9cf006e0c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0b4550 tx=0x7f9d08008040 comp rx=0 tx=0).stop 2024-03-14T12:14:45.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c0ccba0 msgr2=0x7f9d0c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c0ccba0 0x7f9d0c0ccff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c162b40 tx=0x7f9d0400e530 comp rx=0 tx=0).stop 2024-03-14T12:14:45.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 shutdown_connections 2024-03-14T12:14:45.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d0c0ccba0 0x7f9d0c0ccff0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d0c1e8290 0x7f9d0c153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9cf006bc00 0x7f9cf006e0c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 --2- 172.21.15.82:0/2340409258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf8004800 0x7f9d0c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 >> 172.21.15.82:0/2340409258 conn(0x7f9d0c072ee0 msgr2=0x7f9d0c06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:45.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 shutdown_connections 2024-03-14T12:14:45.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.245+0000 7f9d145b2640 1 -- 172.21.15.82:0/2340409258 wait complete. 2024-03-14T12:14:45.256 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-14T12:14:45.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.357+0000 7f94de7fa640 1 Processor -- start 2024-03-14T12:14:45.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.357+0000 7f94de7fa640 1 -- start start 2024-03-14T12:14:45.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94de7fa640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94d80b3fe0 0x7f94d80b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94de7fa640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d80b4930 con 0x7f94d8147880 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94de7fa640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d8147c60 con 0x7f94d8071f90 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94de7fa640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d80adcf0 con 0x7f94d80b3fe0 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d7fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f94d8147880 0x7f94d806ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38584/0 (socket says 172.21.15.82:38584) 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d7fff640 1 -- 172.21.15.82:0/2851133874 learned_addr learned my addr 172.21.15.82:0/2851133874 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94dcd97640 1 --2- 172.21.15.82:0/2851133874 >> v2:172.21.15.164:3300/0 conn(0x7f94d80b3fe0 0x7f94d80b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753884163 0 0) 0x7f94c8001110 con 0x7f94d8147880 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c4003520 con 0x7f94d8147880 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885003013 0 0) 0x7f94d8147c60 con 0x7f94d8071f90 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c8001110 con 0x7f94d8071f90 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2574005271 0 0) 0x7f94c4003520 con 0x7f94d8147880 2024-03-14T12:14:45.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 >> v2:172.21.15.164:3300/0 conn(0x7f94d80b3fe0 msgr2=0x7f94d80b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 --2- 172.21.15.82:0/2851133874 >> v2:172.21.15.164:3300/0 conn(0x7f94d80b3fe0 0x7f94d80b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.358+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 >> v1:172.21.15.164:6789/0 conn(0x7f94d8071f90 legacy=0x7f94d815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8072600 con 0x7f94d8147880 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f94c8001840 con 0x7f94d8147880 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94dcd97640 1 --2- 172.21.15.82:0/2851133874 >> v2:172.21.15.164:3300/0 conn(0x7f94d80b3fe0 0x7f94d80b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 >> v1:172.21.15.82:6789/0 conn(0x7f94d8147880 legacy=0x7f94d806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94c4003b40 0x7f94c4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94c40051c0 0x7f94c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80adcf0 con 0x7f94c4004800 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c400c0f0 con 0x7f94c40051c0 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c400c2f0 con 0x7f94c4003b40 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94cf7fe640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d7fff640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94c4003b40 0x7f94c4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94dcd97640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94c40051c0 0x7f94c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94cf7fe640 1 -- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94c4003b40 msgr2=0x7f94c4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94cf7fe640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94c4003b40 0x7f94c4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94cf7fe640 1 -- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94c40051c0 msgr2=0x7f94c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94cf7fe640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94c40051c0 0x7f94c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94cf7fe640 1 -- 172.21.15.82:0/2851133874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c4003520 con 0x7f94c4004800 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94d7fff640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94c4003b40 0x7f94c4003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94dcd97640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94c40051c0 0x7f94c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.359+0000 7f94dcd97640 1 -- 172.21.15.82:0/2851133874 reap_dead start 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94cf7fe640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94c4004c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94c00018d0 tx=0x7f94c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408859f3ed72e3ad server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c00014f0 con 0x7f94c4004800 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c002f050 con 0x7f94c4004800 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94d77fe640 1 -- 172.21.15.82:0/2851133874 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c00035f0 con 0x7f94c4004800 2024-03-14T12:14:45.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94de7fa640 1 -- 172.21.15.82:0/2851133874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 msgr2=0x7f94c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94de7fa640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94c4004c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94c00018d0 tx=0x7f94c4005830 comp rx=0 tx=0).stop 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94de7fa640 1 -- 172.21.15.82:0/2851133874 shutdown_connections 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94de7fa640 1 --2- 172.21.15.82:0/2851133874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94c4004c80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94de7fa640 1 -- 172.21.15.82:0/2851133874 >> 172.21.15.82:0/2851133874 conn(0x7f94d80b1bd0 msgr2=0x7f94d80b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94de7fa640 1 -- 172.21.15.82:0/2851133874 shutdown_connections 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.360+0000 7f94de7fa640 1 -- 172.21.15.82:0/2851133874 wait complete. 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 Processor -- start 2024-03-14T12:14:45.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 -- start start 2024-03-14T12:14:45.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94d813c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94d815fea0 0x7f94d815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94d8142160 0x7f94d81425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8072bc0 con 0x7f94c4004800 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80ad980 con 0x7f94d815fea0 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94de7fa640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b43f0 con 0x7f94d8142160 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94d813c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94d813c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37544/0 (socket says 172.21.15.82:37544) 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.362+0000 7f94d7fff640 1 -- 172.21.15.82:0/993258389 learned_addr learned my addr 172.21.15.82:0/993258389 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d7fff640 1 -- 172.21.15.82:0/993258389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94d8142160 msgr2=0x7f94d81425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d7fff640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94d8142160 0x7f94d81425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d7fff640 1 -- 172.21.15.82:0/993258389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94d815fea0 msgr2=0x7f94d815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d7fff640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94d815fea0 0x7f94d815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d7fff640 1 -- 172.21.15.82:0/993258389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d813caf0 con 0x7f94c4004800 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d7fff640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94d813c2c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f94c8007300 tx=0x7f94c80077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d57fa640 1 -- 172.21.15.82:0/993258389 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800e8d0 con 0x7f94c4004800 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d57fa640 1 -- 172.21.15.82:0/993258389 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c8018070 con 0x7f94c4004800 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d813cdd0 con 0x7f94c4004800 2024-03-14T12:14:45.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94d57fa640 1 -- 172.21.15.82:0/993258389 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800ec70 con 0x7f94c4004800 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.363+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d813d890 con 0x7f94c4004800 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.364+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d8074690 con 0x7f94c4004800 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.364+0000 7f94d57fa640 1 -- 172.21.15.82:0/993258389 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f94c801d460 con 0x7f94c4004800 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.365+0000 7f94d57fa640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f94b806bbd0 0x7f94b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.365+0000 7f94d57fa640 1 -- 172.21.15.82:0/993258389 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94c808a700 con 0x7f94c4004800 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.365+0000 7f94cf7fe640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f94b806bbd0 0x7f94b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.365+0000 7f94cf7fe640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f94b806bbd0 0x7f94b806e090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94c0034000 tx=0x7f94c0009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.369+0000 7f94d57fa640 1 -- 172.21.15.82:0/993258389 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f94d80b4930 con 0x7f94c4004800 2024-03-14T12:14:45.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.607+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f94d804f050 con 0x7f94c4004800 2024-03-14T12:14:45.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.609+0000 7f94d57fa640 1 -- 172.21.15.82:0/993258389 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f94c8058ef0 con 0x7f94c4004800 2024-03-14T12:14:45.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f94b806bbd0 msgr2=0x7f94b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f94b806bbd0 0x7f94b806e090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94c0034000 tx=0x7f94c0009030 comp rx=0 tx=0).stop 2024-03-14T12:14:45.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 msgr2=0x7f94d813c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94d813c2c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f94c8007300 tx=0x7f94c80077c0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 shutdown_connections 2024-03-14T12:14:45.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f94b806bbd0 0x7f94b806e090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94d8142160 0x7f94d81425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94d815fea0 0x7f94d815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 --2- 172.21.15.82:0/993258389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4004800 0x7f94d813c2c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 >> 172.21.15.82:0/993258389 conn(0x7f94d80b1bd0 msgr2=0x7f94d80b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:45.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 shutdown_connections 2024-03-14T12:14:45.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.611+0000 7f94de7fa640 1 -- 172.21.15.82:0/993258389 wait complete. 2024-03-14T12:14:45.622 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-14T12:14:45.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.723+0000 7fc4059a2640 1 Processor -- start 2024-03-14T12:14:45.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc4059a2640 1 -- start start 2024-03-14T12:14:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc4059a2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc40015dc10 0x7fc4000d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc4059a2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 0x7fc40006b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc4059a2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4000d07a0 con 0x7fc40015dff0 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc4059a2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40006b6c0 con 0x7fc40015dc10 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc4059a2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000d1750 con 0x7fc40006ad10 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc3fffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc40015dff0 0x7fc4000b56c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38594/0 (socket says 172.21.15.82:38594) 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.724+0000 7fc3fffff640 1 -- 172.21.15.82:0/1502486272 learned_addr learned my addr 172.21.15.82:0/1502486272 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746486631 0 0) 0x7fc4000d07a0 con 0x7fc40015dff0 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f0003520 con 0x7fc40015dff0 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4051a1640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 0x7fc40006b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4049a0640 1 --2- 172.21.15.82:0/1502486272 >> v2:172.21.15.164:3300/0 conn(0x7fc40015dc10 0x7fc4000d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3396911008 0 0) 0x7fc3f0003520 con 0x7fc40015dff0 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 msgr2=0x7fc40006b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 0x7fc40006b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 >> v2:172.21.15.164:3300/0 conn(0x7fc40015dc10 msgr2=0x7fc4000d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 --2- 172.21.15.82:0/1502486272 >> v2:172.21.15.164:3300/0 conn(0x7fc40015dc10 0x7fc4000d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4000721b0 con 0x7fc40015dff0 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fc3ec001670 con 0x7fc40015dff0 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 >> v1:172.21.15.82:6789/0 conn(0x7fc40015dff0 legacy=0x7fc4000b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f00039b0 0x7fc3f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3f0004800 0x7fc3f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 0x7fc3f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40006b6c0 con 0x7fc3f00039b0 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000d1750 con 0x7fc3f0004800 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f000b9c0 con 0x7fc40006ad10 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4049a0640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f00039b0 0x7fc3f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc3fffff640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3f0004800 0x7fc3f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4051a1640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 0x7fc3f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4049a0640 1 -- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 msgr2=0x7fc3f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4049a0640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc40006ad10 0x7fc3f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4049a0640 1 -- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3f0004800 msgr2=0x7fc3f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4049a0640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3f0004800 0x7fc3f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.725+0000 7fc4049a0640 1 -- 172.21.15.82:0/1502486272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f0003520 con 0x7fc3f00039b0 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.726+0000 7fc4049a0640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f00039b0 0x7fc3f0003dd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009af0 tx=0x7fc3f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a363a331383422 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.726+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4039070 con 0x7fc3f00039b0 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.726+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f402c710 con 0x7fc3f00039b0 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.726+0000 7fc3ff7fe640 1 -- 172.21.15.82:0/1502486272 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4009690 con 0x7fc3f00039b0 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.727+0000 7fc4059a2640 1 -- 172.21.15.82:0/1502486272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f00039b0 msgr2=0x7fc3f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.727+0000 7fc4059a2640 1 --2- 172.21.15.82:0/1502486272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3f00039b0 0x7fc3f0003dd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009af0 tx=0x7fc3f0005830 comp rx=0 tx=0).stop 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.727+0000 7fc4051a1640 1 -- 172.21.15.82:0/1502486272 reap_dead start 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.727+0000 7fc4059a2640 1 -- 172.21.15.82:0/1502486272 shutdown_connections 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.727+0000 7fc4059a2640 1 -- 172.21.15.82:0/1502486272 >> 172.21.15.82:0/1502486272 conn(0x7fc40006be20 msgr2=0x7fc40006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.727+0000 7fc4059a2640 1 -- 172.21.15.82:0/1502486272 shutdown_connections 2024-03-14T12:14:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.727+0000 7fc4059a2640 1 -- 172.21.15.82:0/1502486272 wait complete. 2024-03-14T12:14:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.728+0000 7fc4059a2640 1 Processor -- start 2024-03-14T12:14:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.728+0000 7fc4059a2640 1 -- start start 2024-03-14T12:14:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4059a2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3f00039b0 0x7fc400159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4059a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400157e40 0x7fc40014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4059a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4001575a0 0x7fc4000c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4059a2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000b5dd0 con 0x7fc400157e40 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4059a2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000d0cf0 con 0x7fc4001575a0 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4059a2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc400072490 con 0x7fc3f00039b0 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400157e40 0x7fc40014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400157e40 0x7fc40014e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37560/0 (socket says 172.21.15.82:37560) 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 -- 172.21.15.82:0/294127278 learned_addr learned my addr 172.21.15.82:0/294127278 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4049a0640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3f00039b0 0x7fc400159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4051a1640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4001575a0 0x7fc4000c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 -- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3f00039b0 msgr2=0x7fc400159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3f00039b0 0x7fc400159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 -- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4001575a0 msgr2=0x7fc4000c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4001575a0 0x7fc4000c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 -- 172.21.15.82:0/294127278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4001583b0 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4051a1640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4001575a0 0x7fc4000c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc4049a0640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3f00039b0 0x7fc400159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fffff640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400157e40 0x7fc40014e2d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec007340 tx=0x7fc3ec007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.729+0000 7fc3fd7fa640 1 -- 172.21.15.82:0/294127278 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00f940 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.730+0000 7fc3fd7fa640 1 -- 172.21.15.82:0/294127278 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec018070 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.730+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc400158690 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.730+0000 7fc3fd7fa640 1 -- 172.21.15.82:0/294127278 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00fd20 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.730+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc400158c50 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.731+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc400075210 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.731+0000 7fc3fd7fa640 1 -- 172.21.15.82:0/294127278 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc3ec01d460 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.732+0000 7fc3fd7fa640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3e406bbd0 0x7fc3e406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.732+0000 7fc4049a0640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3e406bbd0 0x7fc3e406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.732+0000 7fc3fd7fa640 1 -- 172.21.15.82:0/294127278 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3ec08a8a0 con 0x7fc400157e40 2024-03-14T12:14:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.732+0000 7fc4049a0640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3e406bbd0 0x7fc3e406e090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4036000 tx=0x7fc3f4030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:45.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.736+0000 7fc3fd7fa640 1 -- 172.21.15.82:0/294127278 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc3ec059650 con 0x7fc400157e40 2024-03-14T12:14:45.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.973+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc40015cfd0 con 0x7fc400157e40 2024-03-14T12:14:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.975+0000 7fc3fd7fa640 1 -- 172.21.15.82:0/294127278 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc3ec058f50 con 0x7fc400157e40 2024-03-14T12:14:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3e406bbd0 msgr2=0x7fc3e406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3e406bbd0 0x7fc3e406e090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4036000 tx=0x7fc3f4030000 comp rx=0 tx=0).stop 2024-03-14T12:14:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400157e40 msgr2=0x7fc40014e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400157e40 0x7fc40014e2d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec007340 tx=0x7fc3ec007800 comp rx=0 tx=0).stop 2024-03-14T12:14:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 shutdown_connections 2024-03-14T12:14:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4001575a0 0x7fc4000c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400157e40 0x7fc40014e2d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3e406bbd0 0x7fc3e406e090 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 --2- 172.21.15.82:0/294127278 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3f00039b0 0x7fc400159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 >> 172.21.15.82:0/294127278 conn(0x7fc40006be20 msgr2=0x7fc4000b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 shutdown_connections 2024-03-14T12:14:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:45.977+0000 7fc4059a2640 1 -- 172.21.15.82:0/294127278 wait complete. 2024-03-14T12:14:45.988 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-14T12:14:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.088+0000 7f2e276b2640 1 Processor -- start 2024-03-14T12:14:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.088+0000 7f2e276b2640 1 -- start start 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e276b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 0x7f2e2006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e276b2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2e200b39c0 0x7f2e2006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e276b2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e200d1590 con 0x7f2e2006f7f0 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e276b2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e20161b70 con 0x7f2e200b39c0 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e276b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e20161e50 con 0x7f2e2015df30 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e24c4d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2e2006f7f0 0x7f2e20071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38604/0 (socket says 172.21.15.82:38604) 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e24c4d640 1 -- 172.21.15.82:0/811501457 learned_addr learned my addr 172.21.15.82:0/811501457 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714512032 0 0) 0x7f2e200d1590 con 0x7f2e2006f7f0 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e0c003520 con 0x7f2e2006f7f0 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e25c4f640 1 --2- 172.21.15.82:0/811501457 >> v2:172.21.15.164:3300/0 conn(0x7f2e200b39c0 0x7f2e2006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e2544e640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 0x7f2e2006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3054963882 0 0) 0x7f2e0c003520 con 0x7f2e2006f7f0 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 msgr2=0x7f2e2006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 0x7f2e2006ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 >> v2:172.21.15.164:3300/0 conn(0x7f2e200b39c0 msgr2=0x7f2e2006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 --2- 172.21.15.82:0/811501457 >> v2:172.21.15.164:3300/0 conn(0x7f2e200b39c0 0x7f2e2006b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20162130 con 0x7f2e2006f7f0 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f2e10001670 con 0x7f2e2006f7f0 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 >> v1:172.21.15.82:6789/0 conn(0x7f2e2006f7f0 legacy=0x7f2e20071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0039b0 0x7f2e0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c004800 0x7f2e0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 0x7f2e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e20161b70 con 0x7f2e0c0039b0 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e20161e50 con 0x7f2e0c004800 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.089+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c00ba70 con 0x7f2e2015df30 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e2544e640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0039b0 0x7f2e0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e25c4f640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 0x7f2e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e24c4d640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c004800 0x7f2e0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e2544e640 1 -- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 msgr2=0x7f2e0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e2544e640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 0x7f2e0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e2544e640 1 -- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c004800 msgr2=0x7f2e0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e2544e640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c004800 0x7f2e0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e2544e640 1 -- 172.21.15.82:0/811501457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c003520 con 0x7f2e0c0039b0 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e25c4f640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e2015df30 0x7f2e0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e24c4d640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c004800 0x7f2e0c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.090+0000 7f2e2544e640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0039b0 0x7f2e0c003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00ba00 tx=0x7f2e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f070a114a2ffd631 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c036020 con 0x7f2e0c0039b0 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0325f0 con 0x7f2e0c0039b0 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e17fff640 1 -- 172.21.15.82:0/811501457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c00b600 con 0x7f2e0c0039b0 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e276b2640 1 -- 172.21.15.82:0/811501457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0039b0 msgr2=0x7f2e0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e276b2640 1 --2- 172.21.15.82:0/811501457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0039b0 0x7f2e0c003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00ba00 tx=0x7f2e0c005830 comp rx=0 tx=0).stop 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e25c4f640 1 -- 172.21.15.82:0/811501457 reap_dead start 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e276b2640 1 -- 172.21.15.82:0/811501457 shutdown_connections 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e276b2640 1 -- 172.21.15.82:0/811501457 >> 172.21.15.82:0/811501457 conn(0x7f2e200d09d0 msgr2=0x7f2e200d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e276b2640 1 -- 172.21.15.82:0/811501457 shutdown_connections 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.091+0000 7f2e276b2640 1 -- 172.21.15.82:0/811501457 wait complete. 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.092+0000 7f2e276b2640 1 Processor -- start 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.092+0000 7f2e276b2640 1 -- start start 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e276b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c0039b0 0x7f2e200af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e276b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e200afea0 0x7f2e20157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e276b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e20160460 0x7f2e20160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e276b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2006fe40 con 0x7f2e200afea0 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e276b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201628e0 con 0x7f2e0c0039b0 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e276b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2006ffc0 con 0x7f2e20160460 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e200afea0 0x7f2e20157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e200afea0 0x7f2e20157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37582/0 (socket says 172.21.15.82:37582) 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 -- 172.21.15.82:0/3316024151 learned_addr learned my addr 172.21.15.82:0/3316024151 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e2544e640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c0039b0 0x7f2e200af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e25c4f640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e20160460 0x7f2e20160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 -- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e20160460 msgr2=0x7f2e20160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e20160460 0x7f2e20160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 -- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c0039b0 msgr2=0x7f2e200af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c0039b0 0x7f2e200af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e24c4d640 1 -- 172.21.15.82:0/3316024151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20158650 con 0x7f2e200afea0 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.093+0000 7f2e2544e640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c0039b0 0x7f2e200af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.094+0000 7f2e24c4d640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e200afea0 0x7f2e20157c00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000e8f0 tx=0x7f2e1000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.094+0000 7f2e15ffb640 1 -- 172.21.15.82:0/3316024151 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000c8d0 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.094+0000 7f2e15ffb640 1 -- 172.21.15.82:0/3316024151 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e10018070 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.094+0000 7f2e15ffb640 1 -- 172.21.15.82:0/3316024151 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000cc70 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.094+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e201f78e0 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.094+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e201f7ea0 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.095+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2004f050 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.095+0000 7f2e15ffb640 1 -- 172.21.15.82:0/3316024151 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2e1001d460 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.096+0000 7f2e15ffb640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e0406bbd0 0x7f2e0406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.096+0000 7f2e15ffb640 1 -- 172.21.15.82:0/3316024151 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2e1008a640 con 0x7f2e200afea0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.096+0000 7f2e2544e640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e0406bbd0 0x7f2e0406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.096+0000 7f2e2544e640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e0406bbd0 0x7f2e0406e090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0057e0 tx=0x7f2e1c03d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.100+0000 7f2e15ffb640 1 -- 172.21.15.82:0/3316024151 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2e10059480 con 0x7f2e200afea0 2024-03-14T12:14:46.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.337+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f2e200afc80 con 0x7f2e200afea0 2024-03-14T12:14:46.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.339+0000 7f2e15ffb640 1 -- 172.21.15.82:0/3316024151 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2e10058d80 con 0x7f2e200afea0 2024-03-14T12:14:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e0406bbd0 msgr2=0x7f2e0406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e0406bbd0 0x7f2e0406e090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0057e0 tx=0x7f2e1c03d040 comp rx=0 tx=0).stop 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e200afea0 msgr2=0x7f2e20157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e200afea0 0x7f2e20157c00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000e8f0 tx=0x7f2e1000edb0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 shutdown_connections 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e0406bbd0 0x7f2e0406e090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e20160460 0x7f2e20160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e200afea0 0x7f2e20157c00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 --2- 172.21.15.82:0/3316024151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e0c0039b0 0x7f2e200af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 >> 172.21.15.82:0/3316024151 conn(0x7f2e200d09d0 msgr2=0x7f2e20073e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 shutdown_connections 2024-03-14T12:14:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.341+0000 7f2e276b2640 1 -- 172.21.15.82:0/3316024151 wait complete. 2024-03-14T12:14:46.353 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:46.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.453+0000 7f5471fd4640 1 Processor -- start 2024-03-14T12:14:46.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.453+0000 7f5471fd4640 1 -- start start 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f5471fd4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f546c161c40 0x7f546c1620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f5471fd4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546c0d1000 con 0x7f546c0cdae0 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f5471fd4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546c0cdec0 con 0x7f546c0cc0d0 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f5471fd4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f546c1625f0 con 0x7f546c161c40 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f546c0cdae0 0x7f546c06ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38606/0 (socket says 172.21.15.82:38606) 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546b7fe640 1 -- 172.21.15.82:0/2569409080 learned_addr learned my addr 172.21.15.82:0/2569409080 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546affd640 1 --2- 172.21.15.82:0/2569409080 >> v2:172.21.15.164:3300/0 conn(0x7f546c161c40 0x7f546c1620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849707886 0 0) 0x7f545c001110 con 0x7f546c0cdae0 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5454003520 con 0x7f546c0cdae0 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2355602220 0 0) 0x7f5454003520 con 0x7f546c0cdae0 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 >> v2:172.21.15.164:3300/0 conn(0x7f546c161c40 msgr2=0x7f546c1620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 --2- 172.21.15.82:0/2569409080 >> v2:172.21.15.164:3300/0 conn(0x7f546c161c40 0x7f546c1620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 >> v1:172.21.15.164:6789/0 conn(0x7f546c0cc0d0 legacy=0x7f546c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c0cff60 con 0x7f546c0cdae0 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f545c001840 con 0x7f546c0cdae0 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 >> v1:172.21.15.82:6789/0 conn(0x7f546c0cdae0 legacy=0x7f546c06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54540039b0 0x7f5454003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5454004800 0x7f5454004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54540051c0 0x7f545400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c1625f0 con 0x7f54540039b0 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545400c070 con 0x7f5454004800 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.454+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545400c270 con 0x7f54540051c0 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546b7fe640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54540039b0 0x7f5454003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546affd640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5454004800 0x7f5454004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546bfff640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54540051c0 0x7f545400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546b7fe640 1 -- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54540051c0 msgr2=0x7f545400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546b7fe640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54540051c0 0x7f545400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546b7fe640 1 -- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5454004800 msgr2=0x7f5454004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546b7fe640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5454004800 0x7f5454004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546b7fe640 1 -- 172.21.15.82:0/2569409080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5454003520 con 0x7f54540039b0 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.455+0000 7f546bfff640 1 -- 172.21.15.82:0/2569409080 reap_dead start 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f546b7fe640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54540039b0 0x7f5454003dd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c4c0 tx=0x7f5454005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332a4dd8f0114382 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c00c0e0 con 0x7f54540039b0 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c030070 con 0x7f54540039b0 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f546a7fc640 1 -- 172.21.15.82:0/2569409080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0058d0 con 0x7f54540039b0 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f5471fd4640 1 -- 172.21.15.82:0/2569409080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54540039b0 msgr2=0x7f5454003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54540039b0 0x7f5454003dd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c4c0 tx=0x7f5454005830 comp rx=0 tx=0).stop 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f5471fd4640 1 -- 172.21.15.82:0/2569409080 shutdown_connections 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2569409080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54540039b0 0x7f5454003dd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f5471fd4640 1 -- 172.21.15.82:0/2569409080 >> 172.21.15.82:0/2569409080 conn(0x7f546c0b3610 msgr2=0x7f546c0b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f5471fd4640 1 -- 172.21.15.82:0/2569409080 shutdown_connections 2024-03-14T12:14:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.456+0000 7f5471fd4640 1 -- 172.21.15.82:0/2569409080 wait complete. 2024-03-14T12:14:46.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.457+0000 7f5471fd4640 1 Processor -- start 2024-03-14T12:14:46.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f5471fd4640 1 -- start start 2024-03-14T12:14:46.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f5471fd4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54540039b0 0x7f546c1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f5471fd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f546c0cb6c0 0x7f546c0b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f5471fd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f546c155af0 0x7f546c1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f5471fd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0d0520 con 0x7f546c0cb6c0 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f5471fd4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0d0b70 con 0x7f54540039b0 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f5471fd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0d0800 con 0x7f546c155af0 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f546affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f546c0cb6c0 0x7f546c0b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f546affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f546c0cb6c0 0x7f546c0b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37600/0 (socket says 172.21.15.82:37600) 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.458+0000 7f546affd640 1 -- 172.21.15.82:0/2503470720 learned_addr learned my addr 172.21.15.82:0/2503470720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546b7fe640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54540039b0 0x7f546c1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546bfff640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f546c155af0 0x7f546c1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546affd640 1 -- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f546c155af0 msgr2=0x7f546c1594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546affd640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f546c155af0 0x7f546c1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546affd640 1 -- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54540039b0 msgr2=0x7f546c1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546affd640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54540039b0 0x7f546c1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546affd640 1 -- 172.21.15.82:0/2503470720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c155f70 con 0x7f546c0cb6c0 2024-03-14T12:14:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f546affd640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f546c0cb6c0 0x7f546c0b7580 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54600118c0 tx=0x7f5460011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f5453fff640 1 -- 172.21.15.82:0/2503470720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460019070 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f5453fff640 1 -- 172.21.15.82:0/2503470720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460004240 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c0b5670 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f5453fff640 1 -- 172.21.15.82:0/2503470720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000f820 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.459+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c0b5390 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.460+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c04f050 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.460+0000 7f5453fff640 1 -- 172.21.15.82:0/2503470720 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f546001c070 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.461+0000 7f5453fff640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f545806bc00 0x7f545806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.461+0000 7f5453fff640 1 -- 172.21.15.82:0/2503470720 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f546008aa30 con 0x7f546c0cb6c0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.461+0000 7f546b7fe640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f545806bc00 0x7f545806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.461+0000 7f546b7fe640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f545806bc00 0x7f545806e0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c490 tx=0x7f545c009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.465+0000 7f5453fff640 1 -- 172.21.15.82:0/2503470720 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5460059900 con 0x7f546c0cb6c0 2024-03-14T12:14:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.641+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f546c070060 con 0x7f546c0cb6c0 2024-03-14T12:14:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.641+0000 7f5453fff640 1 -- 172.21.15.82:0/2503470720 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f5460059200 con 0x7f546c0cb6c0 2024-03-14T12:14:46.642 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:46.642 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":30,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:43.753159+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f545806bc00 msgr2=0x7f545806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f545806bc00 0x7f545806e0c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c490 tx=0x7f545c009030 comp rx=0 tx=0).stop 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f546c0cb6c0 msgr2=0x7f546c0b7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f546c0cb6c0 0x7f546c0b7580 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54600118c0 tx=0x7f5460011d80 comp rx=0 tx=0).stop 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 shutdown_connections 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f545806bc00 0x7f545806e0c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f546c155af0 0x7f546c1594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f546c0cb6c0 0x7f546c0b7580 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 --2- 172.21.15.82:0/2503470720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54540039b0 0x7f546c1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 >> 172.21.15.82:0/2503470720 conn(0x7f546c0b3610 msgr2=0x7f546c0cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 shutdown_connections 2024-03-14T12:14:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.644+0000 7f5471fd4640 1 -- 172.21.15.82:0/2503470720 wait complete. 2024-03-14T12:14:46.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:14:46.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:14:46.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.757+0000 7f4735b29640 1 Processor -- start 2024-03-14T12:14:46.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f4735b29640 1 -- start start 2024-03-14T12:14:46.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f4735b29640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f473015dff0 0x7f47300d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f4735b29640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47300d12a0 0x7f47300d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f4735b29640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47300d0c40 0x7f47300ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f4735b29640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47300d2350 con 0x7f473015dff0 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f4735b29640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47300d25a0 con 0x7f47300d12a0 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f4735b29640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47300d2880 con 0x7f47300d0c40 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f472f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f473015dff0 0x7f47300d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f472f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f473015dff0 0x7f47300d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37616/0 (socket says 172.21.15.82:37616) 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.758+0000 7f472f7fe640 1 -- 172.21.15.82:0/1316712736 learned_addr learned my addr 172.21.15.82:0/1316712736 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472effd640 1 --2- 172.21.15.82:0/1316712736 >> v2:172.21.15.164:3300/0 conn(0x7f47300d12a0 0x7f47300d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472ffff640 1 --2- 172.21.15.82:0/1316712736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47300d0c40 0x7f47300ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472f7fe640 1 -- 172.21.15.82:0/1316712736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47300d0c40 msgr2=0x7f47300ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472f7fe640 1 --2- 172.21.15.82:0/1316712736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47300d0c40 0x7f47300ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472f7fe640 1 -- 172.21.15.82:0/1316712736 >> v2:172.21.15.164:3300/0 conn(0x7f47300d12a0 msgr2=0x7f47300d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472f7fe640 1 --2- 172.21.15.82:0/1316712736 >> v2:172.21.15.164:3300/0 conn(0x7f47300d12a0 0x7f47300d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472f7fe640 1 -- 172.21.15.82:0/1316712736 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47300d2b60 con 0x7f473015dff0 2024-03-14T12:14:46.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472ffff640 1 --2- 172.21.15.82:0/1316712736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47300d0c40 0x7f47300ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.759+0000 7f472f7fe640 1 --2- 172.21.15.82:0/1316712736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473015dff0 0x7f47300d4480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f47300d3390 tx=0x7f472002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ca4851317ea8f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f472e7fc640 1 -- 172.21.15.82:0/1316712736 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720035020 con 0x7f473015dff0 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f472e7fc640 1 -- 172.21.15.82:0/1316712736 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720031870 con 0x7f473015dff0 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f472e7fc640 1 -- 172.21.15.82:0/1316712736 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720031c40 con 0x7f473015dff0 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 -- 172.21.15.82:0/1316712736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473015dff0 msgr2=0x7f47300d4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 --2- 172.21.15.82:0/1316712736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473015dff0 0x7f47300d4480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f47300d3390 tx=0x7f472002b910 comp rx=0 tx=0).stop 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 -- 172.21.15.82:0/1316712736 shutdown_connections 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 --2- 172.21.15.82:0/1316712736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47300d0c40 0x7f47300ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 --2- 172.21.15.82:0/1316712736 >> v2:172.21.15.164:3300/0 conn(0x7f47300d12a0 0x7f47300d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 --2- 172.21.15.82:0/1316712736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473015dff0 0x7f47300d4480 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 -- 172.21.15.82:0/1316712736 >> 172.21.15.82:0/1316712736 conn(0x7f473009d0d0 msgr2=0x7f47300d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 -- 172.21.15.82:0/1316712736 shutdown_connections 2024-03-14T12:14:46.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.760+0000 7f4735b29640 1 -- 172.21.15.82:0/1316712736 wait complete. 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.761+0000 7f4735b29640 1 Processor -- start 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f4735b29640 1 -- start start 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f4735b29640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47300d0c40 0x7f4730157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f4735b29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47300d12a0 0x7f47300b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f4735b29640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f473015dff0 0x7f47300b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f4735b29640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473015e550 con 0x7f47300d0c40 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f4735b29640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473015e3d0 con 0x7f47300d12a0 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f4735b29640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47300ced80 con 0x7f473015dff0 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f472f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47300d0c40 0x7f4730157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f472f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47300d0c40 0x7f4730157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37626/0 (socket says 172.21.15.82:37626) 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.762+0000 7f472f7fe640 1 -- 172.21.15.82:0/790227618 learned_addr learned my addr 172.21.15.82:0/790227618 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472effd640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47300d12a0 0x7f47300b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472ffff640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f473015dff0 0x7f47300b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472f7fe640 1 -- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f473015dff0 msgr2=0x7f47300b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472f7fe640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f473015dff0 0x7f47300b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472f7fe640 1 -- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47300d12a0 msgr2=0x7f47300b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472f7fe640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47300d12a0 0x7f47300b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472f7fe640 1 -- 172.21.15.82:0/790227618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473006ca40 con 0x7f47300d0c40 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472effd640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47300d12a0 0x7f47300b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f472f7fe640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47300d0c40 0x7f4730157960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f472002b950 tx=0x7f47200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f4717fff640 1 -- 172.21.15.82:0/790227618 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720040070 con 0x7f47300d0c40 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f4717fff640 1 -- 172.21.15.82:0/790227618 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720030d00 con 0x7f47300d0c40 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47300c9260 con 0x7f47300d0c40 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f4717fff640 1 -- 172.21.15.82:0/790227618 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720035040 con 0x7f47300d0c40 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.763+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47300c9820 con 0x7f47300d0c40 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.764+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473004f050 con 0x7f47300d0c40 2024-03-14T12:14:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.765+0000 7f4717fff640 1 -- 172.21.15.82:0/790227618 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f472004b060 con 0x7f47300d0c40 2024-03-14T12:14:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.765+0000 7f4717fff640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f471c06bbd0 0x7f471c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.765+0000 7f4717fff640 1 -- 172.21.15.82:0/790227618 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f47200ac3b0 con 0x7f47300d0c40 2024-03-14T12:14:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.765+0000 7f472effd640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f471c06bbd0 0x7f471c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.766+0000 7f472effd640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f471c06bbd0 0x7f471c06e090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f473015d950 tx=0x7f4724008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:46.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.771+0000 7f4717fff640 1 -- 172.21.15.82:0/790227618 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f472002f320 con 0x7f47300d0c40 2024-03-14T12:14:46.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.942+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f473015bdd0 con 0x7f47300d0c40 2024-03-14T12:14:46.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.943+0000 7f4717fff640 1 -- 172.21.15.82:0/790227618 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f472007b1f0 con 0x7f47300d0c40 2024-03-14T12:14:46.943 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:14:46.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f471c06bbd0 msgr2=0x7f471c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f471c06bbd0 0x7f471c06e090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f473015d950 tx=0x7f4724008040 comp rx=0 tx=0).stop 2024-03-14T12:14:46.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47300d0c40 msgr2=0x7f4730157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:46.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47300d0c40 0x7f4730157960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f472002b950 tx=0x7f47200046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 shutdown_connections 2024-03-14T12:14:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f471c06bbd0 0x7f471c06e090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f473015dff0 0x7f47300b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47300d12a0 0x7f47300b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 --2- 172.21.15.82:0/790227618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47300d0c40 0x7f4730157960 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 >> 172.21.15.82:0/790227618 conn(0x7f473009d0d0 msgr2=0x7f47300cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 shutdown_connections 2024-03-14T12:14:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:46.945+0000 7f4735b29640 1 -- 172.21.15.82:0/790227618 wait complete. 2024-03-14T12:14:46.956 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T12:14:47.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.056+0000 7fce07f7e640 1 Processor -- start 2024-03-14T12:14:47.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.057+0000 7fce07f7e640 1 -- start start 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.057+0000 7fce07f7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 0x7fce00160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.057+0000 7fce07f7e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0006fc70 con 0x7fce000b4190 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce07f7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce000ce610 con 0x7fce000cc770 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce07f7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0015d340 con 0x7fce0006ed90 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce05519640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fce000b4190 0x7fce0006f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38618/0 (socket says 172.21.15.82:38618) 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce05519640 1 -- 172.21.15.82:0/3956041829 learned_addr learned my addr 172.21.15.82:0/3956041829 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110962117 0 0) 0x7fcdf4001110 con 0x7fce000b4190 2024-03-14T12:14:47.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde8003520 con 0x7fce000b4190 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce05d1a640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 0x7fce00160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2001614607 0 0) 0x7fcde8003520 con 0x7fce000b4190 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 msgr2=0x7fce00160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 0x7fce00160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 >> v1:172.21.15.164:6789/0 conn(0x7fce000cc770 legacy=0x7fce000ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce000ce8f0 con 0x7fce000b4190 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fcdf4001840 con 0x7fce000b4190 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 >> v1:172.21.15.82:6789/0 conn(0x7fce000b4190 legacy=0x7fce0006f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde80039e0 0x7fcde8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 0x7fcde8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde8004d40 0x7fcde80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0015d340 con 0x7fcde8004d40 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde800bb50 con 0x7fcde80039e0 2024-03-14T12:14:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde800bd20 con 0x7fce0006ed90 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce0651b640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde8004d40 0x7fcde80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce05519640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 0x7fcde8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.058+0000 7fce05d1a640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde80039e0 0x7fcde8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce0651b640 1 -- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 msgr2=0x7fcde8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce0651b640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0006ed90 0x7fcde8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce0651b640 1 -- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde80039e0 msgr2=0x7fcde8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce0651b640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde80039e0 0x7fcde8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce0651b640 1 -- 172.21.15.82:0/3956041829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde8003520 con 0x7fcde8004d40 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce0651b640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde8004d40 0x7fcde80051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0001780 tx=0x7fcde8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2829fc1d1bc8c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0001370 con 0x7fcde8004d40 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf00052d0 con 0x7fcde8004d40 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.059+0000 7fce04d18640 1 -- 172.21.15.82:0/3956041829 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003830 con 0x7fcde8004d40 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.060+0000 7fce07f7e640 1 -- 172.21.15.82:0/3956041829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde8004d40 msgr2=0x7fcde80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.060+0000 7fce07f7e640 1 --2- 172.21.15.82:0/3956041829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde8004d40 0x7fcde80051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0001780 tx=0x7fcde8005830 comp rx=0 tx=0).stop 2024-03-14T12:14:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.060+0000 7fce0651b640 1 -- 172.21.15.82:0/3956041829 reap_dead start 2024-03-14T12:14:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.060+0000 7fce07f7e640 1 -- 172.21.15.82:0/3956041829 shutdown_connections 2024-03-14T12:14:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.060+0000 7fce07f7e640 1 -- 172.21.15.82:0/3956041829 >> 172.21.15.82:0/3956041829 conn(0x7fce000b3670 msgr2=0x7fce000b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.060+0000 7fce07f7e640 1 -- 172.21.15.82:0/3956041829 shutdown_connections 2024-03-14T12:14:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.060+0000 7fce07f7e640 1 -- 172.21.15.82:0/3956041829 wait complete. 2024-03-14T12:14:47.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.061+0000 7fce07f7e640 1 Processor -- start 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.061+0000 7fce07f7e640 1 -- start start 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce07f7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce00147b10 0x7fce0015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce07f7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce000c5ec0 0x7fce0015ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce07f7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce000b61b0 0x7fce000b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce07f7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce000cf190 con 0x7fce000b61b0 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce07f7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce000ceeb0 con 0x7fce000c5ec0 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce07f7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce00160420 con 0x7fce00147b10 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce000b61b0 0x7fce000b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce000b61b0 0x7fce000b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37634/0 (socket says 172.21.15.82:37634) 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 -- 172.21.15.82:0/1565275907 learned_addr learned my addr 172.21.15.82:0/1565275907 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce05d1a640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce00147b10 0x7fce0015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce05519640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce000c5ec0 0x7fce0015ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 -- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce00147b10 msgr2=0x7fce0015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce00147b10 0x7fce0015e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 -- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce000c5ec0 msgr2=0x7fce0015ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce000c5ec0 0x7fce0015ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 -- 172.21.15.82:0/1565275907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce000b4c00 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce05d1a640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce00147b10 0x7fce0015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce05519640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce000c5ec0 0x7fce0015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.062+0000 7fce0651b640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce000b61b0 0x7fce000b6640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005040 tx=0x7fcdf0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.063+0000 7fcdfe7fc640 1 -- 172.21.15.82:0/1565275907 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0001370 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.063+0000 7fcdfe7fc640 1 -- 172.21.15.82:0/1565275907 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0043070 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.063+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce000b4ee0 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.063+0000 7fcdfe7fc640 1 -- 172.21.15.82:0/1565275907 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0007a30 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.063+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce000b54a0 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.064+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0004f050 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.064+0000 7fcdfe7fc640 1 -- 172.21.15.82:0/1565275907 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcdf003b040 con 0x7fce000b61b0 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.064+0000 7fcdfe7fc640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcdec06bbd0 0x7fcdec06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.065+0000 7fcdfe7fc640 1 -- 172.21.15.82:0/1565275907 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcdf00ab930 con 0x7fce000b61b0 2024-03-14T12:14:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.065+0000 7fce05d1a640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcdec06bbd0 0x7fcdec06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.065+0000 7fce05d1a640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcdec06bbd0 0x7fcdec06e090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8003ac0 tx=0x7fcdf8009760 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.069+0000 7fcdfe7fc640 1 -- 172.21.15.82:0/1565275907 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fcdf007a800 con 0x7fce000b61b0 2024-03-14T12:14:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.242+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fce000c85a0 con 0x7fce000b61b0 2024-03-14T12:14:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.242+0000 7fcdfe7fc640 1 -- 172.21.15.82:0/1565275907 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcdf007a100 con 0x7fce000b61b0 2024-03-14T12:14:47.242 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:47.243 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-03-14T12:14:47.243 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:14:47.243 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-03-14T12:14:47.243 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-03-14T12:14:47.243 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:14:47.243 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcdec06bbd0 msgr2=0x7fcdec06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcdec06bbd0 0x7fcdec06e090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8003ac0 tx=0x7fcdf8009760 comp rx=0 tx=0).stop 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce000b61b0 msgr2=0x7fce000b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce000b61b0 0x7fce000b6640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005040 tx=0x7fcdf0001120 comp rx=0 tx=0).stop 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 shutdown_connections 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce000b61b0 0x7fce000b6640 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce000c5ec0 0x7fce0015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcdec06bbd0 0x7fcdec06e090 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 --2- 172.21.15.82:0/1565275907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce00147b10 0x7fce0015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 >> 172.21.15.82:0/1565275907 conn(0x7fce000b3670 msgr2=0x7fce0015fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 shutdown_connections 2024-03-14T12:14:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.244+0000 7fce07f7e640 1 -- 172.21.15.82:0/1565275907 wait complete. 2024-03-14T12:14:47.255 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T12:14:47.255 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T12:14:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.359+0000 7f4902627640 1 Processor -- start 2024-03-14T12:14:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.359+0000 7f4902627640 1 -- start start 2024-03-14T12:14:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.359+0000 7f4902627640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48fc144a70 0x7f48fc072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f4902627640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48fc145060 0x7f48fc0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f4902627640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc144e50 con 0x7f48fc06ad10 2024-03-14T12:14:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f4902627640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc072660 con 0x7f48fc145060 2024-03-14T12:14:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f4902627640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc0b58d0 con 0x7f48fc144a70 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fb7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48fc145060 0x7f48fc0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fb7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48fc145060 0x7f48fc0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37648/0 (socket says 172.21.15.82:37648) 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1243195923 learned_addr learned my addr 172.21.15.82:0/1243195923 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fbfff640 1 --2- 172.21.15.82:0/1243195923 >> v2:172.21.15.164:3300/0 conn(0x7f48fc144a70 0x7f48fc072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1243195923 >> v2:172.21.15.164:3300/0 conn(0x7f48fc144a70 msgr2=0x7f48fc072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fb7fe640 1 --2- 172.21.15.82:0/1243195923 >> v2:172.21.15.164:3300/0 conn(0x7f48fc144a70 0x7f48fc072120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1243195923 >> v1:172.21.15.164:6789/0 conn(0x7f48fc06ad10 legacy=0x7f48fc0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.360+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1243195923 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0b5bb0 con 0x7f48fc145060 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f48fb7fe640 1 --2- 172.21.15.82:0/1243195923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc145060 0x7f48fc0b39c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0713c0 tx=0x7f48f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb041f7b2b86b86 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f48faffd640 1 -- 172.21.15.82:0/1243195923 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f002d040 con 0x7f48fc145060 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f48faffd640 1 -- 172.21.15.82:0/1243195923 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0004230 con 0x7f48fc145060 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f48faffd640 1 -- 172.21.15.82:0/1243195923 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0003610 con 0x7f48fc145060 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 -- 172.21.15.82:0/1243195923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc145060 msgr2=0x7f48fc0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 --2- 172.21.15.82:0/1243195923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc145060 0x7f48fc0b39c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0713c0 tx=0x7f48f002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 -- 172.21.15.82:0/1243195923 shutdown_connections 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 --2- 172.21.15.82:0/1243195923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc145060 0x7f48fc0b39c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 --2- 172.21.15.82:0/1243195923 >> v2:172.21.15.164:3300/0 conn(0x7f48fc144a70 0x7f48fc072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 -- 172.21.15.82:0/1243195923 >> 172.21.15.82:0/1243195923 conn(0x7f48fc0d0a30 msgr2=0x7f48fc0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 -- 172.21.15.82:0/1243195923 shutdown_connections 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.361+0000 7f4902627640 1 -- 172.21.15.82:0/1243195923 wait complete. 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 Processor -- start 2024-03-14T12:14:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 -- start start 2024-03-14T12:14:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc06ad10 0x7f48fc1e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc144a70 0x7f48fc1dde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48fc145060 0x7f48fc13fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0b5e90 con 0x7f48fc144a70 2024-03-14T12:14:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0cf390 con 0x7f48fc06ad10 2024-03-14T12:14:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.363+0000 7f4902627640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0b3f00 con 0x7f48fc145060 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc144a70 0x7f48fc1dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc144a70 0x7f48fc1dde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37662/0 (socket says 172.21.15.82:37662) 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1322607022 learned_addr learned my addr 172.21.15.82:0/1322607022 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f4900bc4640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48fc145060 0x7f48fc13fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fbfff640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc06ad10 0x7f48fc1e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48fc145060 msgr2=0x7f48fc13fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48fc145060 0x7f48fc13fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc06ad10 msgr2=0x7f48fc1e1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc06ad10 0x7f48fc1e1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 -- 172.21.15.82:0/1322607022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc1e2480 con 0x7f48fc144a70 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48fb7fe640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc144a70 0x7f48fc1dde40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f48f00090b0 tx=0x7f48f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48f8ff9640 1 -- 172.21.15.82:0/1322607022 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0003c50 con 0x7f48fc144a70 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48f8ff9640 1 -- 172.21.15.82:0/1322607022 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0003df0 con 0x7f48fc144a70 2024-03-14T12:14:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc1dfc80 con 0x7f48fc144a70 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f48f8ff9640 1 -- 172.21.15.82:0/1322607022 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0006280 con 0x7f48fc144a70 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.364+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc1e04c0 con 0x7f48fc144a70 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.365+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc04f050 con 0x7f48fc144a70 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.366+0000 7f48f8ff9640 1 -- 172.21.15.82:0/1322607022 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f48f0042020 con 0x7f48fc144a70 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.366+0000 7f48f8ff9640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f48e806bbd0 0x7f48e806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.366+0000 7f48f8ff9640 1 -- 172.21.15.82:0/1322607022 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f48f00ac2b0 con 0x7f48fc144a70 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.366+0000 7f48fbfff640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f48e806bbd0 0x7f48e806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.366+0000 7f48fbfff640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f48e806bbd0 0x7f48e806e090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48fc1f4cb0 tx=0x7f48ec008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.370+0000 7f48f8ff9640 1 -- 172.21.15.82:0/1322607022 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f48fc1dfc80 con 0x7f48fc144a70 2024-03-14T12:14:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.545+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f48fc0d1290 con 0x7f48fc144a70 2024-03-14T12:14:47.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.768+0000 7f48f8ff9640 1 -- 172.21.15.82:0/1322607022 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f48f007b180 con 0x7f48fc144a70 2024-03-14T12:14:47.769 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-03-14T12:14:47.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f48e806bbd0 msgr2=0x7f48e806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f48e806bbd0 0x7f48e806e090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48fc1f4cb0 tx=0x7f48ec008040 comp rx=0 tx=0).stop 2024-03-14T12:14:47.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc144a70 msgr2=0x7f48fc1dde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc144a70 0x7f48fc1dde40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f48f00090b0 tx=0x7f48f00046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 shutdown_connections 2024-03-14T12:14:47.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f48e806bbd0 0x7f48e806e090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48fc145060 0x7f48fc13fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc144a70 0x7f48fc1dde40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 --2- 172.21.15.82:0/1322607022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48fc06ad10 0x7f48fc1e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 >> 172.21.15.82:0/1322607022 conn(0x7f48fc0d0a30 msgr2=0x7f48fc0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:47.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 shutdown_connections 2024-03-14T12:14:47.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.771+0000 7f4902627640 1 -- 172.21.15.82:0/1322607022 wait complete. 2024-03-14T12:14:47.785 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T12:14:47.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.892+0000 7feab9685640 1 Processor -- start 2024-03-14T12:14:47.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.892+0000 7feab9685640 1 -- start start 2024-03-14T12:14:47.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab9685640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feab415de20 0x7feab40d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab9685640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feab415e200 0x7feab40d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab9685640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40d2cf0 con 0x7feab40d28e0 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab9685640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40cfd80 con 0x7feab415de20 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab9685640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40cece0 con 0x7feab415e200 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feab415de20 0x7feab40d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feab415de20 0x7feab40d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37666/0 (socket says 172.21.15.82:37666) 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab2ffd640 1 -- 172.21.15.82:0/2969881000 learned_addr learned my addr 172.21.15.82:0/2969881000 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab2ffd640 1 -- 172.21.15.82:0/2969881000 >> v2:172.21.15.164:3300/0 conn(0x7feab415e200 msgr2=0x7feab40d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab2ffd640 1 --2- 172.21.15.82:0/2969881000 >> v2:172.21.15.164:3300/0 conn(0x7feab415e200 0x7feab40d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab2ffd640 1 -- 172.21.15.82:0/2969881000 >> v1:172.21.15.164:6789/0 conn(0x7feab40d28e0 legacy=0x7feab40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.893+0000 7feab2ffd640 1 -- 172.21.15.82:0/2969881000 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40cefc0 con 0x7feab415de20 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.894+0000 7feab2ffd640 1 --2- 172.21.15.82:0/2969881000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415de20 0x7feab40d0db0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feab408e500 tx=0x7feaa402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0b05fd49423c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.894+0000 7feab27fc640 1 -- 172.21.15.82:0/2969881000 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa402e070 con 0x7feab415de20 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.894+0000 7feab27fc640 1 -- 172.21.15.82:0/2969881000 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa4037070 con 0x7feab415de20 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.894+0000 7feab27fc640 1 -- 172.21.15.82:0/2969881000 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4031a00 con 0x7feab415de20 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 -- 172.21.15.82:0/2969881000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415de20 msgr2=0x7feab40d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 --2- 172.21.15.82:0/2969881000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415de20 0x7feab40d0db0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feab408e500 tx=0x7feaa402b880 comp rx=0 tx=0).stop 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 -- 172.21.15.82:0/2969881000 shutdown_connections 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 --2- 172.21.15.82:0/2969881000 >> v2:172.21.15.164:3300/0 conn(0x7feab415e200 0x7feab40d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 --2- 172.21.15.82:0/2969881000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415de20 0x7feab40d0db0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 -- 172.21.15.82:0/2969881000 >> 172.21.15.82:0/2969881000 conn(0x7feab409d030 msgr2=0x7feab40d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:47.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 -- 172.21.15.82:0/2969881000 shutdown_connections 2024-03-14T12:14:47.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.895+0000 7feab9685640 1 -- 172.21.15.82:0/2969881000 wait complete. 2024-03-14T12:14:47.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.896+0000 7feab9685640 1 Processor -- start 2024-03-14T12:14:47.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.896+0000 7feab9685640 1 -- start start 2024-03-14T12:14:47.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab40d28e0 0x7feab415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab415de20 0x7feab40c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415e200 0x7feab40c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab40d12f0 con 0x7feab415e200 2024-03-14T12:14:47.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab40cf580 con 0x7feab415de20 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab40ce940 con 0x7feab40d28e0 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415e200 0x7feab40c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415e200 0x7feab40c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37670/0 (socket says 172.21.15.82:37670) 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 -- 172.21.15.82:0/1185040850 learned_addr learned my addr 172.21.15.82:0/1185040850 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feaabfff640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab415de20 0x7feab40c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab2ffd640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab40d28e0 0x7feab415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 -- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab40d28e0 msgr2=0x7feab415a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab40d28e0 0x7feab415a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 -- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab415de20 msgr2=0x7feab40c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab415de20 0x7feab40c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 -- 172.21.15.82:0/1185040850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40cbec0 con 0x7feab415e200 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feaabfff640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab415de20 0x7feab40c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab37fe640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415e200 0x7feab40c85f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c780 tx=0x7feaac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feaab7fe640 1 -- 172.21.15.82:0/1185040850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac017070 con 0x7feab415e200 2024-03-14T12:14:47.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feaab7fe640 1 -- 172.21.15.82:0/1185040850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac003da0 con 0x7feab415e200 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab40cc1a0 con 0x7feab415e200 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feaab7fe640 1 -- 172.21.15.82:0/1185040850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac0052b0 con 0x7feab415e200 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.897+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab40cc760 con 0x7feab415e200 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.898+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab404f050 con 0x7feab415e200 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.899+0000 7feaab7fe640 1 -- 172.21.15.82:0/1185040850 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7feaac01a070 con 0x7feab415e200 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.899+0000 7feaab7fe640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fea9406bbd0 0x7fea9406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.899+0000 7feaab7fe640 1 -- 172.21.15.82:0/1185040850 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7feaac089e90 con 0x7feab415e200 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.899+0000 7feab2ffd640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fea9406bbd0 0x7fea9406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:47.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.900+0000 7feab2ffd640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fea9406bbd0 0x7fea9406e090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feaa4036000 tx=0x7feaa403d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:47.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:47.904+0000 7feaab7fe640 1 -- 172.21.15.82:0/1185040850 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7feaac058bf0 con 0x7feab415e200 2024-03-14T12:14:48.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.076+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7feab40b66a0 con 0x7feab415e200 2024-03-14T12:14:48.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-03-14T12:14:48.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.779+0000 7feaab7fe640 1 -- 172.21.15.82:0/1185040850 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7feaac0584f0 con 0x7feab415e200 2024-03-14T12:14:48.779 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-03-14T12:14:48.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fea9406bbd0 msgr2=0x7fea9406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fea9406bbd0 0x7fea9406e090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feaa4036000 tx=0x7feaa403d040 comp rx=0 tx=0).stop 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415e200 msgr2=0x7feab40c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415e200 0x7feab40c85f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c780 tx=0x7feaac00cc40 comp rx=0 tx=0).stop 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 shutdown_connections 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab415e200 0x7feab40c85f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab415de20 0x7feab40c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fea9406bbd0 0x7fea9406e090 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 --2- 172.21.15.82:0/1185040850 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab40d28e0 0x7feab415a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.782+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 >> 172.21.15.82:0/1185040850 conn(0x7feab409d030 msgr2=0x7feab40c92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.783+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 shutdown_connections 2024-03-14T12:14:48.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.784+0000 7feab9685640 1 -- 172.21.15.82:0/1185040850 wait complete. 2024-03-14T12:14:48.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T12:14:48.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.911+0000 7ff1cba12640 1 Processor -- start 2024-03-14T12:14:48.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.912+0000 7ff1cba12640 1 -- start start 2024-03-14T12:14:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.912+0000 7ff1cba12640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1c4140e70 0x7ff1c40b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.912+0000 7ff1cba12640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c4143c60 0x7ff1c415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.912+0000 7ff1cba12640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c40b41c0 con 0x7ff1c41626f0 2024-03-14T12:14:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.912+0000 7ff1cba12640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c415f3f0 con 0x7ff1c4140e70 2024-03-14T12:14:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.912+0000 7ff1cba12640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c4161c10 con 0x7ff1c4143c60 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c97ae640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1c4140e70 0x7ff1c40b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c9faf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c4143c60 0x7ff1c415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c97ae640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1c4140e70 0x7ff1c40b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:46274/0 (socket says 172.21.15.82:46274) 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c97ae640 1 -- 172.21.15.82:0/3561623116 learned_addr learned my addr 172.21.15.82:0/3561623116 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c9faf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c4143c60 0x7ff1c415eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:56040/0 (socket says 172.21.15.82:56040) 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c8fad640 1 --1- 172.21.15.82:0/3561623116 >> v1:172.21.15.164:6789/0 conn(0x7ff1c41626f0 0x7ff1c4143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:41296/0 (socket says 172.21.15.82:41296) 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c97ae640 1 -- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c4143c60 msgr2=0x7ff1c415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c97ae640 1 --2- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c4143c60 0x7ff1c415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c97ae640 1 -- 172.21.15.82:0/3561623116 >> v1:172.21.15.164:6789/0 conn(0x7ff1c41626f0 legacy=0x7ff1c4143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1bbfff640 1 -- 172.21.15.82:0/3561623116 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574859339 0 0) 0x7ff1c40b41c0 con 0x7ff1c41626f0 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c97ae640 1 -- 172.21.15.82:0/3561623116 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c4161ef0 con 0x7ff1c4140e70 2024-03-14T12:14:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.913+0000 7ff1c9faf640 1 --2- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c4143c60 0x7ff1c415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.914+0000 7ff1c97ae640 1 --2- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4140e70 0x7ff1c40b3b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b913fa70b8f102b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.914+0000 7ff1bbfff640 1 -- 172.21.15.82:0/3561623116 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0034070 con 0x7ff1c4140e70 2024-03-14T12:14:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.914+0000 7ff1bbfff640 1 -- 172.21.15.82:0/3561623116 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c0030840 con 0x7ff1c4140e70 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.914+0000 7ff1bbfff640 1 -- 172.21.15.82:0/3561623116 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0030c10 con 0x7ff1c4140e70 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 -- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4140e70 msgr2=0x7ff1c40b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 --2- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4140e70 0x7ff1c40b3b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c002b980 comp rx=0 tx=0).stop 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 -- 172.21.15.82:0/3561623116 shutdown_connections 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 --2- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c4143c60 0x7ff1c415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 --2- 172.21.15.82:0/3561623116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4140e70 0x7ff1c40b3b30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 -- 172.21.15.82:0/3561623116 >> 172.21.15.82:0/3561623116 conn(0x7ff1c40b2830 msgr2=0x7ff1c40b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 -- 172.21.15.82:0/3561623116 shutdown_connections 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.915+0000 7ff1cba12640 1 -- 172.21.15.82:0/3561623116 wait complete. 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 Processor -- start 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 -- start start 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1c4140e70 0x7ff1c40cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4143c60 0x7ff1c41f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c41626f0 0x7ff1c41f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40af9f0 con 0x7ff1c4140e70 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40af870 con 0x7ff1c4143c60 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.916+0000 7ff1cba12640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40afb70 con 0x7ff1c41626f0 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1c4140e70 0x7ff1c40cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1c4140e70 0x7ff1c40cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37674/0 (socket says 172.21.15.82:37674) 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 -- 172.21.15.82:0/123728622 learned_addr learned my addr 172.21.15.82:0/123728622 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c9faf640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c41626f0 0x7ff1c41f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c8fad640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4143c60 0x7ff1c41f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 -- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c41626f0 msgr2=0x7ff1c41f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c41626f0 0x7ff1c41f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 -- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4143c60 msgr2=0x7ff1c41f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4143c60 0x7ff1c41f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 -- 172.21.15.82:0/123728622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c41412f0 con 0x7ff1c4140e70 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1c97ae640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1c4140e70 0x7ff1c40cd510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1b9ffb640 1 -- 172.21.15.82:0/123728622 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0034040 con 0x7ff1c4140e70 2024-03-14T12:14:48.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1b9ffb640 1 -- 172.21.15.82:0/123728622 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c0030640 con 0x7ff1c4140e70 2024-03-14T12:14:48.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1b9ffb640 1 -- 172.21.15.82:0/123728622 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0039d40 con 0x7ff1c4140e70 2024-03-14T12:14:48.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c41f36b0 con 0x7ff1c4140e70 2024-03-14T12:14:48.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.917+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c40cde80 con 0x7ff1c4140e70 2024-03-14T12:14:48.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.918+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c404f050 con 0x7ff1c4140e70 2024-03-14T12:14:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.919+0000 7ff1b9ffb640 1 -- 172.21.15.82:0/123728622 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff1c004b060 con 0x7ff1c4140e70 2024-03-14T12:14:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.919+0000 7ff1b9ffb640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff1a806bbd0 0x7ff1a806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.919+0000 7ff1b9ffb640 1 -- 172.21.15.82:0/123728622 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff1c00ac680 con 0x7ff1c4140e70 2024-03-14T12:14:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.919+0000 7ff1c8fad640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff1a806bbd0 0x7ff1a806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.920+0000 7ff1c8fad640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff1a806bbd0 0x7ff1a806e090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1c406f180 tx=0x7ff1b4007c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:48.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:48.924+0000 7ff1b9ffb640 1 -- 172.21.15.82:0/123728622 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff1c007b260 con 0x7ff1c4140e70 2024-03-14T12:14:49.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.133+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff1c40b3700 con 0x7ff1c4140e70 2024-03-14T12:14:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.790+0000 7ff1b9ffb640 1 -- 172.21.15.82:0/123728622 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff1c0030810 con 0x7ff1c4140e70 2024-03-14T12:14:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 4 and data pool 5 2024-03-14T12:14:49.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.802+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff1a806bbd0 msgr2=0x7ff1a806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:49.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.802+0000 7ff1cba12640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff1a806bbd0 0x7ff1a806e090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1c406f180 tx=0x7ff1b4007c00 comp rx=0 tx=0).stop 2024-03-14T12:14:49.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.802+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1c4140e70 msgr2=0x7ff1c40cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:49.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.802+0000 7ff1cba12640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1c4140e70 0x7ff1c40cd510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c00046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.807+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 shutdown_connections 2024-03-14T12:14:49.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.807+0000 7ff1cba12640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff1a806bbd0 0x7ff1a806e090 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.807+0000 7ff1cba12640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c41626f0 0x7ff1c41f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.807+0000 7ff1cba12640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c4143c60 0x7ff1c41f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.807+0000 7ff1cba12640 1 --2- 172.21.15.82:0/123728622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1c4140e70 0x7ff1c40cd510 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.807+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 >> 172.21.15.82:0/123728622 conn(0x7ff1c40b2830 msgr2=0x7ff1c40b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.807+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 shutdown_connections 2024-03-14T12:14:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.808+0000 7ff1cba12640 1 -- 172.21.15.82:0/123728622 wait complete. 2024-03-14T12:14:49.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.926+0000 7f7049697640 1 Processor -- start 2024-03-14T12:14:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.926+0000 7f7049697640 1 -- start start 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f7049697640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f704415e0e0 0x7f70440d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f7049697640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70440b3e10 0x7f70440b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f7049697640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704415e4c0 con 0x7f70440d4a20 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f7049697640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70440d1640 con 0x7f70440b3e10 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f7049697640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70440cf2f0 con 0x7f704415e0e0 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f70437fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70440b3e10 0x7f70440b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f70437fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70440b3e10 0x7f70440b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37690/0 (socket says 172.21.15.82:37690) 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f70437fe640 1 -- 172.21.15.82:0/1048429902 learned_addr learned my addr 172.21.15.82:0/1048429902 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f7042ffd640 1 --2- 172.21.15.82:0/1048429902 >> v2:172.21.15.164:3300/0 conn(0x7f704415e0e0 0x7f70440d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f70437fe640 1 -- 172.21.15.82:0/1048429902 >> v2:172.21.15.164:3300/0 conn(0x7f704415e0e0 msgr2=0x7f70440d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f70437fe640 1 --2- 172.21.15.82:0/1048429902 >> v2:172.21.15.164:3300/0 conn(0x7f704415e0e0 0x7f70440d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f70437fe640 1 -- 172.21.15.82:0/1048429902 >> v1:172.21.15.164:6789/0 conn(0x7f70440d4a20 legacy=0x7f70440d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.927+0000 7f70437fe640 1 -- 172.21.15.82:0/1048429902 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70440cf5d0 con 0x7f70440b3e10 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.928+0000 7f70437fe640 1 --2- 172.21.15.82:0/1048429902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f70440b4240 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f704408dca0 tx=0x7f703c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7539e66903a615d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.928+0000 7f7041ffb640 1 -- 172.21.15.82:0/1048429902 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c02ca80 con 0x7f70440b3e10 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.928+0000 7f7041ffb640 1 -- 172.21.15.82:0/1048429902 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c02cc20 con 0x7f70440b3e10 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.928+0000 7f7041ffb640 1 -- 172.21.15.82:0/1048429902 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0343f0 con 0x7f70440b3e10 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.928+0000 7f7049697640 1 -- 172.21.15.82:0/1048429902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 msgr2=0x7f70440b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.928+0000 7f7049697640 1 --2- 172.21.15.82:0/1048429902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f70440b4240 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f704408dca0 tx=0x7f703c02bee0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.929+0000 7f7049697640 1 -- 172.21.15.82:0/1048429902 shutdown_connections 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.929+0000 7f7049697640 1 --2- 172.21.15.82:0/1048429902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f70440b4240 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.929+0000 7f7049697640 1 --2- 172.21.15.82:0/1048429902 >> v2:172.21.15.164:3300/0 conn(0x7f704415e0e0 0x7f70440d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.929+0000 7f7049697640 1 -- 172.21.15.82:0/1048429902 >> 172.21.15.82:0/1048429902 conn(0x7f70440d3d40 msgr2=0x7f70440d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.929+0000 7f7049697640 1 -- 172.21.15.82:0/1048429902 shutdown_connections 2024-03-14T12:14:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.929+0000 7f7049697640 1 -- 172.21.15.82:0/1048429902 wait complete. 2024-03-14T12:14:49.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.930+0000 7f7049697640 1 Processor -- start 2024-03-14T12:14:49.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.930+0000 7f7049697640 1 -- start start 2024-03-14T12:14:49.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f7044153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70440d4a20 0x7f70440c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f704415e0e0 0x7f7044160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440b3610 con 0x7f70440b3e10 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440cfb90 con 0x7f70440d4a20 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70440b38f0 con 0x7f704415e0e0 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f7044153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f7044153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37700/0 (socket says 172.21.15.82:37700) 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 -- 172.21.15.82:0/689648485 learned_addr learned my addr 172.21.15.82:0/689648485 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 -- 172.21.15.82:0/689648485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f704415e0e0 msgr2=0x7f7044160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f704415e0e0 0x7f7044160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 -- 172.21.15.82:0/689648485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70440d4a20 msgr2=0x7f70440c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70440d4a20 0x7f70440c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 -- 172.21.15.82:0/689648485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70440c9cf0 con 0x7f70440b3e10 2024-03-14T12:14:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7042ffd640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f7044153580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f703400d830 tx=0x7f703400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:49.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f702b7fe640 1 -- 172.21.15.82:0/689648485 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400b6c0 con 0x7f70440b3e10 2024-03-14T12:14:49.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f702b7fe640 1 -- 172.21.15.82:0/689648485 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034019070 con 0x7f70440b3e10 2024-03-14T12:14:49.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f702b7fe640 1 -- 172.21.15.82:0/689648485 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400ba80 con 0x7f70440b3e10 2024-03-14T12:14:49.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70440c9fc0 con 0x7f70440b3e10 2024-03-14T12:14:49.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.931+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704415e850 con 0x7f70440b3e10 2024-03-14T12:14:49.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.932+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704404f050 con 0x7f70440b3e10 2024-03-14T12:14:49.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.933+0000 7f702b7fe640 1 -- 172.21.15.82:0/689648485 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f703401e460 con 0x7f70440b3e10 2024-03-14T12:14:49.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.933+0000 7f702b7fe640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f703006bbd0 0x7f703006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:49.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.933+0000 7f702b7fe640 1 -- 172.21.15.82:0/689648485 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f703408b150 con 0x7f70440b3e10 2024-03-14T12:14:49.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.933+0000 7f70427fc640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f703006bbd0 0x7f703006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:49.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.934+0000 7f70427fc640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f703006bbd0 0x7f703006e090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f70440b6530 tx=0x7f70380079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:49.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:49.938+0000 7f702b7fe640 1 -- 172.21.15.82:0/689648485 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7034059c40 con 0x7f70440b3e10 2024-03-14T12:14:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.111+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70440b6280 con 0x7f70440b3e10 2024-03-14T12:14:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.112+0000 7f702b7fe640 1 -- 172.21.15.82:0/689648485 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16670 (secure 0 0 0) 0x7f7034059540 con 0x7f70440b3e10 2024-03-14T12:14:50.112 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:50.112 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":33,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:49.135211+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.113+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f703006bbd0 msgr2=0x7f703006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.113+0000 7f7049697640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f703006bbd0 0x7f703006e090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f70440b6530 tx=0x7f70380079b0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 msgr2=0x7f7044153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f7044153580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f703400d830 tx=0x7f703400dcf0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 shutdown_connections 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f703006bbd0 0x7f703006e090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f704415e0e0 0x7f7044160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70440d4a20 0x7f70440c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 --2- 172.21.15.82:0/689648485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70440b3e10 0x7f7044153580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 >> 172.21.15.82:0/689648485 conn(0x7f70440d3d40 msgr2=0x7f70440b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 shutdown_connections 2024-03-14T12:14:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.114+0000 7f7049697640 1 -- 172.21.15.82:0/689648485 wait complete. 2024-03-14T12:14:50.125 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:50.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.225+0000 7fc7017b3640 1 Processor -- start 2024-03-14T12:14:50.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.225+0000 7fc7017b3640 1 -- start start 2024-03-14T12:14:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.225+0000 7fc7017b3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0d4a20 0x7fc6fc0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.225+0000 7fc7017b3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6fc0b3e10 0x7fc6fc0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.225+0000 7fc7017b3640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc15e4c0 con 0x7fc6fc15e0e0 2024-03-14T12:14:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.225+0000 7fc7017b3640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0d1640 con 0x7fc6fc0b3e10 2024-03-14T12:14:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.225+0000 7fc7017b3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0cf2f0 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fa7fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0d4a20 0x7fc6fc0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fb7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6fc0b3e10 0x7fc6fc0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fa7fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0d4a20 0x7fc6fc0d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:56074/0 (socket says 172.21.15.82:56074) 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fb7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6fc0b3e10 0x7fc6fc0b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:46306/0 (socket says 172.21.15.82:46306) 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/1269599200 learned_addr learned my addr 172.21.15.82:0/1269599200 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/1269599200 >> v2:172.21.15.164:3300/0 conn(0x7fc6fc0b3e10 msgr2=0x7fc6fc0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fa7fc640 1 --2- 172.21.15.82:0/1269599200 >> v2:172.21.15.164:3300/0 conn(0x7fc6fc0b3e10 0x7fc6fc0b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/1269599200 >> v1:172.21.15.164:6789/0 conn(0x7fc6fc15e0e0 legacy=0x7fc6fc0d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/1269599200 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0cf5d0 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.226+0000 7fc6fb7fe640 1 --2- 172.21.15.82:0/1269599200 >> v2:172.21.15.164:3300/0 conn(0x7fc6fc0b3e10 0x7fc6fc0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:14:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.227+0000 7fc6fa7fc640 1 --2- 172.21.15.82:0/1269599200 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0d4a20 0x7fc6fc0d2020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009870 tx=0x7fc6f002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77737df320d78403 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.227+0000 7fc6f9ffb640 1 -- 172.21.15.82:0/1269599200 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0034070 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.227+0000 7fc6f9ffb640 1 -- 172.21.15.82:0/1269599200 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0030840 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.227+0000 7fc6f9ffb640 1 -- 172.21.15.82:0/1269599200 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0030c10 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 -- 172.21.15.82:0/1269599200 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0d4a20 msgr2=0x7fc6fc0d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 --2- 172.21.15.82:0/1269599200 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0d4a20 0x7fc6fc0d2020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009870 tx=0x7fc6f002b980 comp rx=0 tx=0).stop 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 -- 172.21.15.82:0/1269599200 shutdown_connections 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 --2- 172.21.15.82:0/1269599200 >> v2:172.21.15.164:3300/0 conn(0x7fc6fc0b3e10 0x7fc6fc0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 --2- 172.21.15.82:0/1269599200 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0d4a20 0x7fc6fc0d2020 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 -- 172.21.15.82:0/1269599200 >> 172.21.15.82:0/1269599200 conn(0x7fc6fc0d3d40 msgr2=0x7fc6fc0d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 -- 172.21.15.82:0/1269599200 shutdown_connections 2024-03-14T12:14:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.228+0000 7fc7017b3640 1 -- 172.21.15.82:0/1269599200 wait complete. 2024-03-14T12:14:50.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.229+0000 7fc7017b3640 1 Processor -- start 2024-03-14T12:14:50.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.229+0000 7fc7017b3640 1 -- start start 2024-03-14T12:14:50.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0b3e10 0x7fc6fc0b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6fc0d4a20 0x7fc6fc1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc15e0e0 0x7fc6fc0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0b3610 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0cfb90 con 0x7fc6fc15e0e0 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0b38f0 con 0x7fc6fc0b3e10 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6fc0d4a20 0x7fc6fc1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6fc0d4a20 0x7fc6fc1f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37714/0 (socket says 172.21.15.82:37714) 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/564568915 learned_addr learned my addr 172.21.15.82:0/564568915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fb7fe640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc15e0e0 0x7fc6fc0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6faffd640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0b3e10 0x7fc6fc0b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0b3e10 msgr2=0x7fc6fc0b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0b3e10 0x7fc6fc0b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc15e0e0 msgr2=0x7fc6fc0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc15e0e0 0x7fc6fc0b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 -- 172.21.15.82:0/564568915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0b9030 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fb7fe640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc15e0e0 0x7fc6fc0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6faffd640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0b3e10 0x7fc6fc0b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6fa7fc640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6fc0d4a20 0x7fc6fc1f4900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0000c00 tx=0x7fc6f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6e37fe640 1 -- 172.21.15.82:0/564568915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0048070 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6e37fe640 1 -- 172.21.15.82:0/564568915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0030640 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc6e37fe640 1 -- 172.21.15.82:0/564568915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0034040 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc0cafb0 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.230+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc0cb570 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.231+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc04f050 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.232+0000 7fc6e37fe640 1 -- 172.21.15.82:0/564568915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc6f004b070 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.232+0000 7fc6e37fe640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6e806bbd0 0x7fc6e806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.232+0000 7fc6e37fe640 1 -- 172.21.15.82:0/564568915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6f00ac880 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.232+0000 7fc6faffd640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6e806bbd0 0x7fc6e806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.233+0000 7fc6faffd640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6e806bbd0 0x7fc6e806e090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec001cd0 tx=0x7fc6ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.236+0000 7fc6e37fe640 1 -- 172.21.15.82:0/564568915 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc6f007b400 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.412+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6fc074690 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.413+0000 7fc6e37fe640 1 -- 172.21.15.82:0/564568915 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16670 (secure 0 0 0) 0x7fc6f007ad00 con 0x7fc6fc0d4a20 2024-03-14T12:14:50.414 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":33,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:49.135211+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6e806bbd0 msgr2=0x7fc6e806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6e806bbd0 0x7fc6e806e090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec001cd0 tx=0x7fc6ec009040 comp rx=0 tx=0).stop 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6fc0d4a20 msgr2=0x7fc6fc1f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6fc0d4a20 0x7fc6fc1f4900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0000c00 tx=0x7fc6f0004730 comp rx=0 tx=0).stop 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 shutdown_connections 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc15e0e0 0x7fc6fc0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6fc0d4a20 0x7fc6fc1f4900 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6e806bbd0 0x7fc6e806e090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 --2- 172.21.15.82:0/564568915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6fc0b3e10 0x7fc6fc0b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 >> 172.21.15.82:0/564568915 conn(0x7fc6fc0d3d40 msgr2=0x7fc6fc0b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:50.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 shutdown_connections 2024-03-14T12:14:50.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.416+0000 7fc7017b3640 1 -- 172.21.15.82:0/564568915 wait complete. 2024-03-14T12:14:50.427 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T12:14:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.530+0000 7f0269681640 1 Processor -- start 2024-03-14T12:14:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.530+0000 7f0269681640 1 -- start start 2024-03-14T12:14:50.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f0269681640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f02640d1070 0x7f02640d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f0269681640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026406ef30 0x7f02640cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f0269681640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026406e7a0 con 0x7f026415f430 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f0269681640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026406eaa0 con 0x7f02640d1070 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f0269681640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026415bb80 con 0x7f026406ef30 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f0262ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f02640d1070 0x7f02640d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f02627fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f026415f430 0x7f026415b2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38624/0 (socket says 172.21.15.82:38624) 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.531+0000 7f02627fc640 1 -- 172.21.15.82:0/2723491149 learned_addr learned my addr 172.21.15.82:0/2723491149 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.532+0000 7f0262ffd640 1 -- 172.21.15.82:0/2723491149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026406ef30 msgr2=0x7f02640cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.532+0000 7f0262ffd640 1 --2- 172.21.15.82:0/2723491149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026406ef30 0x7f02640cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.532+0000 7f0262ffd640 1 -- 172.21.15.82:0/2723491149 >> v1:172.21.15.82:6789/0 conn(0x7f026415f430 legacy=0x7f026415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.532+0000 7f0261ffb640 1 -- 172.21.15.82:0/2723491149 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834816941 0 0) 0x7f026406e7a0 con 0x7f026415f430 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.532+0000 7f0262ffd640 1 -- 172.21.15.82:0/2723491149 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026415be60 con 0x7f02640d1070 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.532+0000 7f0262ffd640 1 --2- 172.21.15.82:0/2723491149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02640d1070 0x7f02640d1450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f02540097a0 tx=0x7f025402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53578cffe4ac750 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0261ffb640 1 -- 172.21.15.82:0/2723491149 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254035020 con 0x7f02640d1070 2024-03-14T12:14:50.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0261ffb640 1 -- 172.21.15.82:0/2723491149 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0254031830 con 0x7f02640d1070 2024-03-14T12:14:50.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0261ffb640 1 -- 172.21.15.82:0/2723491149 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254031c00 con 0x7f02640d1070 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 -- 172.21.15.82:0/2723491149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02640d1070 msgr2=0x7f02640d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 --2- 172.21.15.82:0/2723491149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02640d1070 0x7f02640d1450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f02540097a0 tx=0x7f025402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 -- 172.21.15.82:0/2723491149 shutdown_connections 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 --2- 172.21.15.82:0/2723491149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026406ef30 0x7f02640cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 --2- 172.21.15.82:0/2723491149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02640d1070 0x7f02640d1450 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 -- 172.21.15.82:0/2723491149 >> 172.21.15.82:0/2723491149 conn(0x7f026406c000 msgr2=0x7f026406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 -- 172.21.15.82:0/2723491149 shutdown_connections 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.533+0000 7f0269681640 1 -- 172.21.15.82:0/2723491149 wait complete. 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.534+0000 7f0269681640 1 Processor -- start 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.534+0000 7f0269681640 1 -- start start 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f0269681640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026406ef30 0x7f026412ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f0269681640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02640d1070 0x7f0264159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f0269681640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f026415f430 0x7f0264140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f0269681640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026406f500 con 0x7f026415f430 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f0269681640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026406f380 con 0x7f026406ef30 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f0269681640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02640ccf70 con 0x7f02640d1070 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02637fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f026415f430 0x7f0264140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02637fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f026415f430 0x7f0264140b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37742/0 (socket says 172.21.15.82:37742) 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02637fe640 1 -- 172.21.15.82:0/3439913007 learned_addr learned my addr 172.21.15.82:0/3439913007 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02627fc640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02640d1070 0x7f0264159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f0262ffd640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026406ef30 0x7f026412ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02637fe640 1 -- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02640d1070 msgr2=0x7f0264159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02637fe640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02640d1070 0x7f0264159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02637fe640 1 -- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026406ef30 msgr2=0x7f026412ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.535+0000 7f02637fe640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026406ef30 0x7f026412ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.536+0000 7f02637fe640 1 -- 172.21.15.82:0/3439913007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0264167c40 con 0x7f026415f430 2024-03-14T12:14:50.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.536+0000 7f02637fe640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f026415f430 0x7f0264140b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f025c00b970 tx=0x7f025c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.536+0000 7f024b7fe640 1 -- 172.21.15.82:0/3439913007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c019070 con 0x7f026415f430 2024-03-14T12:14:50.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.536+0000 7f024b7fe640 1 -- 172.21.15.82:0/3439913007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c00cb60 con 0x7f026415f430 2024-03-14T12:14:50.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.536+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02641670c0 con 0x7f026415f430 2024-03-14T12:14:50.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.536+0000 7f024b7fe640 1 -- 172.21.15.82:0/3439913007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c014440 con 0x7f026415f430 2024-03-14T12:14:50.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.536+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0264167960 con 0x7f026415f430 2024-03-14T12:14:50.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.537+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026404f050 con 0x7f026415f430 2024-03-14T12:14:50.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.537+0000 7f024b7fe640 1 -- 172.21.15.82:0/3439913007 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f025c01c070 con 0x7f026415f430 2024-03-14T12:14:50.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.538+0000 7f024b7fe640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025006bbd0 0x7f025006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.538+0000 7f0262ffd640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025006bbd0 0x7f025006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.538+0000 7f024b7fe640 1 -- 172.21.15.82:0/3439913007 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f025c08a7c0 con 0x7f026415f430 2024-03-14T12:14:50.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.538+0000 7f0262ffd640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025006bbd0 0x7f025006e090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0254036000 tx=0x7f025403c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.542+0000 7f024b7fe640 1 -- 172.21.15.82:0/3439913007 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f025c0592b0 con 0x7f026415f430 2024-03-14T12:14:50.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.747+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f026406dfd0 con 0x7f026415f430 2024-03-14T12:14:50.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.795+0000 7f024b7fe640 1 -- 172.21.15.82:0/3439913007 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f025c058bb0 con 0x7f026415f430 2024-03-14T12:14:50.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025006bbd0 msgr2=0x7f025006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025006bbd0 0x7f025006e090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0254036000 tx=0x7f025403c040 comp rx=0 tx=0).stop 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f026415f430 msgr2=0x7f0264140b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f026415f430 0x7f0264140b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f025c00b970 tx=0x7f025c00be30 comp rx=0 tx=0).stop 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 shutdown_connections 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f026415f430 0x7f0264140b40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025006bbd0 0x7f025006e090 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02640d1070 0x7f0264159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 --2- 172.21.15.82:0/3439913007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026406ef30 0x7f026412ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 >> 172.21.15.82:0/3439913007 conn(0x7f026406c000 msgr2=0x7f02640b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 shutdown_connections 2024-03-14T12:14:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.798+0000 7f0269681640 1 -- 172.21.15.82:0/3439913007 wait complete. 2024-03-14T12:14:50.809 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T12:14:50.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T12:14:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.913+0000 7ff129f2b640 1 Processor -- start 2024-03-14T12:14:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.913+0000 7ff129f2b640 1 -- start start 2024-03-14T12:14:50.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff129f2b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1240d0db0 0x7ff124162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff129f2b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff124147360 0x7ff1240b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff129f2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1241478b0 0x7ff1240b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff129f2b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12406e8e0 con 0x7ff1240d0db0 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff129f2b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12406eb80 con 0x7ff124147360 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff129f2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12406f100 con 0x7ff1241478b0 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1240d0db0 0x7ff124162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1240d0db0 0x7ff124162560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37744/0 (socket says 172.21.15.82:37744) 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 -- 172.21.15.82:0/3304186842 learned_addr learned my addr 172.21.15.82:0/3304186842 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 -- 172.21.15.82:0/3304186842 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1241478b0 msgr2=0x7ff1240b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 --2- 172.21.15.82:0/3304186842 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1241478b0 0x7ff1240b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 -- 172.21.15.82:0/3304186842 >> v2:172.21.15.164:3300/0 conn(0x7ff124147360 msgr2=0x7ff1240b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 --2- 172.21.15.82:0/3304186842 >> v2:172.21.15.164:3300/0 conn(0x7ff124147360 0x7ff1240b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.914+0000 7ff1237fe640 1 -- 172.21.15.82:0/3304186842 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12406ee20 con 0x7ff1240d0db0 2024-03-14T12:14:50.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.915+0000 7ff1237fe640 1 --2- 172.21.15.82:0/3304186842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1240d0db0 0x7ff124162560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff124066f00 tx=0x7ff11402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22eba71eaecfeea server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.915+0000 7ff1227fc640 1 -- 172.21.15.82:0/3304186842 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11402e070 con 0x7ff1240d0db0 2024-03-14T12:14:50.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.915+0000 7ff1227fc640 1 -- 172.21.15.82:0/3304186842 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114037070 con 0x7ff1240d0db0 2024-03-14T12:14:50.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.915+0000 7ff1227fc640 1 -- 172.21.15.82:0/3304186842 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114031a00 con 0x7ff1240d0db0 2024-03-14T12:14:50.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 -- 172.21.15.82:0/3304186842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1240d0db0 msgr2=0x7ff124162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 --2- 172.21.15.82:0/3304186842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1240d0db0 0x7ff124162560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff124066f00 tx=0x7ff11402b880 comp rx=0 tx=0).stop 2024-03-14T12:14:50.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 -- 172.21.15.82:0/3304186842 shutdown_connections 2024-03-14T12:14:50.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 --2- 172.21.15.82:0/3304186842 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1241478b0 0x7ff1240b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 --2- 172.21.15.82:0/3304186842 >> v2:172.21.15.164:3300/0 conn(0x7ff124147360 0x7ff1240b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 --2- 172.21.15.82:0/3304186842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1240d0db0 0x7ff124162560 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 -- 172.21.15.82:0/3304186842 >> 172.21.15.82:0/3304186842 conn(0x7ff1240b4fd0 msgr2=0x7ff1240b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 -- 172.21.15.82:0/3304186842 shutdown_connections 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.916+0000 7ff129f2b640 1 -- 172.21.15.82:0/3304186842 wait complete. 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.917+0000 7ff129f2b640 1 Processor -- start 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.917+0000 7ff129f2b640 1 -- start start 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff129f2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1240d0db0 0x7ff1241e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff129f2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff124147360 0x7ff12409b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff129f2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1241478b0 0x7ff12415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff129f2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff124162aa0 con 0x7ff124147360 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff129f2b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff124162040 con 0x7ff1241478b0 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff129f2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12406d330 con 0x7ff1240d0db0 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff124147360 0x7ff12409b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff124147360 0x7ff12409b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37760/0 (socket says 172.21.15.82:37760) 2024-03-14T12:14:50.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 -- 172.21.15.82:0/2164490677 learned_addr learned my addr 172.21.15.82:0/2164490677 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:50.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff1237fe640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1240d0db0 0x7ff1241e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff123fff640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1241478b0 0x7ff12415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 -- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1240d0db0 msgr2=0x7ff1241e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1240d0db0 0x7ff1241e9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 -- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1241478b0 msgr2=0x7ff12415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:50.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1241478b0 0x7ff12415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:50.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff122ffd640 1 -- 172.21.15.82:0/2164490677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1241e7b80 con 0x7ff124147360 2024-03-14T12:14:50.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.918+0000 7ff1237fe640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1240d0db0 0x7ff1241e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:50.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.919+0000 7ff122ffd640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff124147360 0x7ff12409b0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff11800b4a0 tx=0x7ff11800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.919+0000 7ff10bfff640 1 -- 172.21.15.82:0/2164490677 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118014070 con 0x7ff124147360 2024-03-14T12:14:50.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.919+0000 7ff10bfff640 1 -- 172.21.15.82:0/2164490677 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118010610 con 0x7ff124147360 2024-03-14T12:14:50.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.919+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1241e7e60 con 0x7ff124147360 2024-03-14T12:14:50.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.919+0000 7ff10bfff640 1 -- 172.21.15.82:0/2164490677 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118010a00 con 0x7ff124147360 2024-03-14T12:14:50.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.919+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1241e8420 con 0x7ff124147360 2024-03-14T12:14:50.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.920+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12404f050 con 0x7ff124147360 2024-03-14T12:14:50.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.920+0000 7ff10bfff640 1 -- 172.21.15.82:0/2164490677 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff118017070 con 0x7ff124147360 2024-03-14T12:14:50.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.921+0000 7ff10bfff640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff11006bbd0 0x7ff11006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:50.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.921+0000 7ff1237fe640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff11006bbd0 0x7ff11006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:50.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.921+0000 7ff10bfff640 1 -- 172.21.15.82:0/2164490677 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff11808a4f0 con 0x7ff124147360 2024-03-14T12:14:50.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.921+0000 7ff1237fe640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff11006bbd0 0x7ff11006e090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff114036000 tx=0x7ff11403d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:50.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:50.925+0000 7ff10bfff640 1 -- 172.21.15.82:0/2164490677 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff118058fe0 con 0x7ff124147360 2024-03-14T12:14:51.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.131+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff1241e8790 con 0x7ff124147360 2024-03-14T12:14:51.681 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:14:51.684 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:14:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.800+0000 7ff10bfff640 1 -- 172.21.15.82:0/2164490677 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff1180588e0 con 0x7ff124147360 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff11006bbd0 msgr2=0x7ff11006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff11006bbd0 0x7ff11006e090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff114036000 tx=0x7ff11403d040 comp rx=0 tx=0).stop 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff124147360 msgr2=0x7ff12409b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff124147360 0x7ff12409b0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff11800b4a0 tx=0x7ff11800b960 comp rx=0 tx=0).stop 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 shutdown_connections 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1241478b0 0x7ff12415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff124147360 0x7ff12409b0c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff11006bbd0 0x7ff11006e090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 --2- 172.21.15.82:0/2164490677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1240d0db0 0x7ff1241e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 >> 172.21.15.82:0/2164490677 conn(0x7ff1240b4fd0 msgr2=0x7ff12415fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 shutdown_connections 2024-03-14T12:14:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.802+0000 7ff129f2b640 1 -- 172.21.15.82:0/2164490677 wait complete. 2024-03-14T12:14:51.815 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T12:14:51.815 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:51.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.919+0000 7f4b2aeea640 1 Processor -- start 2024-03-14T12:14:51.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.919+0000 7f4b2aeea640 1 -- start start 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b2aeea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b2406b420 0x7f4b240cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b2aeea640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b240ccb40 con 0x7f4b2415be40 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b2aeea640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b240cce20 con 0x7f4b24066b00 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b2aeea640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b240cd750 con 0x7f4b2406b420 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b28c86640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4b2415be40 0x7f4b240d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52594/0 (socket says 172.21.15.82:52594) 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b28c86640 1 -- 172.21.15.82:0/963665921 learned_addr learned my addr 172.21.15.82:0/963665921 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b29487640 1 --2- 172.21.15.82:0/963665921 >> v2:172.21.15.82:3300/0 conn(0x7f4b2406b420 0x7f4b240cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b237fe640 1 -- 172.21.15.82:0/963665921 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659487280 0 0) 0x7f4b240ccb40 con 0x7f4b2415be40 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b237fe640 1 -- 172.21.15.82:0/963665921 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c003520 con 0x7f4b2415be40 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b29487640 1 -- 172.21.15.82:0/963665921 >> v1:172.21.15.164:6789/0 conn(0x7f4b24066b00 legacy=0x7f4b2406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:51.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b29487640 1 -- 172.21.15.82:0/963665921 >> v1:172.21.15.82:6789/0 conn(0x7f4b2415be40 legacy=0x7f4b240d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:51.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.920+0000 7f4b29487640 1 -- 172.21.15.82:0/963665921 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240cd100 con 0x7f4b2406b420 2024-03-14T12:14:51.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.921+0000 7f4b29487640 1 --2- 172.21.15.82:0/963665921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b240cb860 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009d90 tx=0x7f4b1c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43877b67d950ee48 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:51.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.921+0000 7f4b237fe640 1 -- 172.21.15.82:0/963665921 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c039070 con 0x7f4b2406b420 2024-03-14T12:14:51.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.921+0000 7f4b237fe640 1 -- 172.21.15.82:0/963665921 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c02cd00 con 0x7f4b2406b420 2024-03-14T12:14:51.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.921+0000 7f4b237fe640 1 -- 172.21.15.82:0/963665921 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c034660 con 0x7f4b2406b420 2024-03-14T12:14:51.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.922+0000 7f4b2aeea640 1 -- 172.21.15.82:0/963665921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 msgr2=0x7f4b240cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:51.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.922+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/963665921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b240cb860 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009d90 tx=0x7f4b1c02bf50 comp rx=0 tx=0).stop 2024-03-14T12:14:51.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.922+0000 7f4b2aeea640 1 -- 172.21.15.82:0/963665921 shutdown_connections 2024-03-14T12:14:51.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.922+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/963665921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b240cb860 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:51.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.922+0000 7f4b2aeea640 1 -- 172.21.15.82:0/963665921 >> 172.21.15.82:0/963665921 conn(0x7f4b24072310 msgr2=0x7f4b24072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:51.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.922+0000 7f4b2aeea640 1 -- 172.21.15.82:0/963665921 shutdown_connections 2024-03-14T12:14:51.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.922+0000 7f4b2aeea640 1 -- 172.21.15.82:0/963665921 wait complete. 2024-03-14T12:14:51.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.923+0000 7f4b2aeea640 1 Processor -- start 2024-03-14T12:14:51.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.923+0000 7f4b2aeea640 1 -- start start 2024-03-14T12:14:51.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b2aeea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b24066b00 0x7f4b240c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:51.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b2aeea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b2415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:51.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b2aeea640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b2415be40 0x7f4b2415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b2aeea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240b1310 con 0x7f4b2406b420 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b2aeea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240b1190 con 0x7f4b24066b00 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b2aeea640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2415c220 con 0x7f4b2415be40 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b2415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b2415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58712/0 (socket says 172.21.15.82:58712) 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 -- 172.21.15.82:0/2463386936 learned_addr learned my addr 172.21.15.82:0/2463386936 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b28c86640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b24066b00 0x7f4b240c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b29487640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b2415be40 0x7f4b2415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 -- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b2415be40 msgr2=0x7f4b2415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b2415be40 0x7f4b2415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 -- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b24066b00 msgr2=0x7f4b240c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b24066b00 0x7f4b240c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b23fff640 1 -- 172.21.15.82:0/2463386936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2415db70 con 0x7f4b2406b420 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.924+0000 7f4b29487640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b2415be40 0x7f4b2415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:51.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.925+0000 7f4b23fff640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b2415d630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c430 tx=0x7f4b1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.925+0000 7f4b217fa640 1 -- 172.21.15.82:0/2463386936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180052d0 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.925+0000 7f4b217fa640 1 -- 172.21.15.82:0/2463386936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18005490 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.925+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2415de10 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.925+0000 7f4b217fa640 1 -- 172.21.15.82:0/2463386936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18007260 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.925+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2415e3d0 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.926+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2404f050 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.926+0000 7f4b217fa640 1 -- 172.21.15.82:0/2463386936 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4b18007400 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.926+0000 7f4b217fa640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4b1006bbd0 0x7f4b1006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.927+0000 7f4b217fa640 1 -- 172.21.15.82:0/2463386936 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b18089f60 con 0x7f4b2406b420 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.927+0000 7f4b28c86640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4b1006bbd0 0x7f4b1006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:51.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.927+0000 7f4b28c86640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4b1006bbd0 0x7f4b1006e090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b140011e0 tx=0x7f4b14007d30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:51.931+0000 7f4b217fa640 1 -- 172.21.15.82:0/2463386936 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4b18058370 con 0x7f4b2406b420 2024-03-14T12:14:52.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.138+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b240ca750 con 0x7f4b2406b420 2024-03-14T12:14:52.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.139+0000 7f4b217fa640 1 -- 172.21.15.82:0/2463386936 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f4b1805c410 con 0x7f4b2406b420 2024-03-14T12:14:52.139 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:52.139 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2478084830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2478084830},{"type":"v1","addr":"172.21.15.82:6835","nonce":2478084830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4834,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1811632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1811632800},{"type":"v1","addr":"172.21.15.164:6834","nonce":1811632800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4176962755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4176962755},{"type":"v1","addr":"172.21.15.164:6835","nonce":4176962755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:14:49.135548+0000","modified":"2024-03-14T12:14:51.133632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/682556190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":682556190},{"type":"v1","addr":"172.21.15.82:6837","nonce":682556190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T12:14:52.139 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 15 2024-03-14T12:14:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4b1006bbd0 msgr2=0x7f4b1006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4b1006bbd0 0x7f4b1006e090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b140011e0 tx=0x7f4b14007d30 comp rx=0 tx=0).stop 2024-03-14T12:14:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 msgr2=0x7f4b2415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b2415d630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c430 tx=0x7f4b1800c8f0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 shutdown_connections 2024-03-14T12:14:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4b1006bbd0 0x7f4b1006e090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b2415be40 0x7f4b2415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b2406b420 0x7f4b2415d630 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 --2- 172.21.15.82:0/2463386936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b24066b00 0x7f4b240c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 >> 172.21.15.82:0/2463386936 conn(0x7f4b24072310 msgr2=0x7f4b2406dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 shutdown_connections 2024-03-14T12:14:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.141+0000 7f4b2aeea640 1 -- 172.21.15.82:0/2463386936 wait complete. 2024-03-14T12:14:52.152 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:14:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.251+0000 7f9233545640 1 Processor -- start 2024-03-14T12:14:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.251+0000 7f9233545640 1 -- start start 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f9233545640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f922c0b0f60 0x7f922c0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f9233545640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0afff0 0x7f922c0d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f9233545640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c0b1340 con 0x7f922c06ad10 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f9233545640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0d0ee0 con 0x7f922c0b0f60 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f9233545640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0b4ee0 con 0x7f922c0afff0 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f92312e1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f922c0b0f60 0x7f922c0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f92312e1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f922c0b0f60 0x7f922c0cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58722/0 (socket says 172.21.15.82:58722) 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f92312e1640 1 -- 172.21.15.82:0/2250469678 learned_addr learned my addr 172.21.15.82:0/2250469678 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f92312e1640 1 -- 172.21.15.82:0/2250469678 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0afff0 msgr2=0x7f922c0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f92312e1640 1 --2- 172.21.15.82:0/2250469678 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0afff0 0x7f922c0d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f92312e1640 1 -- 172.21.15.82:0/2250469678 >> v1:172.21.15.164:6789/0 conn(0x7f922c06ad10 legacy=0x7f922c06b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.252+0000 7f92312e1640 1 -- 172.21.15.82:0/2250469678 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0cd030 con 0x7f922c0b0f60 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.253+0000 7f92312e1640 1 --2- 172.21.15.82:0/2250469678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c0b0f60 0x7f922c0cc9a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9228009e50 tx=0x7f922802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcf7ec15fdb1e7f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.253+0000 7f9223fff640 1 -- 172.21.15.82:0/2250469678 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922802ca70 con 0x7f922c0b0f60 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.253+0000 7f9223fff640 1 -- 172.21.15.82:0/2250469678 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922802cc10 con 0x7f922c0b0f60 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.253+0000 7f9223fff640 1 -- 172.21.15.82:0/2250469678 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92280343f0 con 0x7f922c0b0f60 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 -- 172.21.15.82:0/2250469678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c0b0f60 msgr2=0x7f922c0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 --2- 172.21.15.82:0/2250469678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c0b0f60 0x7f922c0cc9a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9228009e50 tx=0x7f922802bed0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 -- 172.21.15.82:0/2250469678 shutdown_connections 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 --2- 172.21.15.82:0/2250469678 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0afff0 0x7f922c0d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 --2- 172.21.15.82:0/2250469678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c0b0f60 0x7f922c0cc9a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 -- 172.21.15.82:0/2250469678 >> 172.21.15.82:0/2250469678 conn(0x7f922c072310 msgr2=0x7f922c072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 -- 172.21.15.82:0/2250469678 shutdown_connections 2024-03-14T12:14:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.254+0000 7f9233545640 1 -- 172.21.15.82:0/2250469678 wait complete. 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.255+0000 7f9233545640 1 Processor -- start 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.255+0000 7f9233545640 1 -- start start 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9233545640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c06ad10 0x7f922c099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9233545640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f922c0afff0 0x7f922c09a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9233545640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0b0f60 0x7f922c09a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9233545640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0cd8d0 con 0x7f922c06ad10 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9233545640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0cd5f0 con 0x7f922c0afff0 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9233545640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0b0450 con 0x7f922c0b0f60 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c06ad10 0x7f922c099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c06ad10 0x7f922c099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58728/0 (socket says 172.21.15.82:58728) 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 -- 172.21.15.82:0/1292164003 learned_addr learned my addr 172.21.15.82:0/1292164003 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9231ae2640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0b0f60 0x7f922c09a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9230ae0640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f922c0afff0 0x7f922c09a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 -- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0b0f60 msgr2=0x7f922c09a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0b0f60 0x7f922c09a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 -- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f922c0afff0 msgr2=0x7f922c09a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f922c0afff0 0x7f922c09a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 -- 172.21.15.82:0/1292164003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c09c260 con 0x7f922c06ad10 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f9230ae0640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f922c0afff0 0x7f922c09a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.256+0000 7f92312e1640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c06ad10 0x7f922c099a30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9228009e50 tx=0x7f9228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.257+0000 7f9221ffb640 1 -- 172.21.15.82:0/1292164003 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228043020 con 0x7f922c06ad10 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.257+0000 7f9221ffb640 1 -- 172.21.15.82:0/1292164003 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922803e450 con 0x7f922c06ad10 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.257+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c0cea90 con 0x7f922c06ad10 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.257+0000 7f9221ffb640 1 -- 172.21.15.82:0/1292164003 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922803e840 con 0x7f922c06ad10 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.257+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c0cf050 con 0x7f922c06ad10 2024-03-14T12:14:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.258+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c04f050 con 0x7f922c06ad10 2024-03-14T12:14:52.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.258+0000 7f9221ffb640 1 -- 172.21.15.82:0/1292164003 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9228046020 con 0x7f922c06ad10 2024-03-14T12:14:52.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.259+0000 7f9221ffb640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f921006bbd0 0x7f921006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.259+0000 7f9230ae0640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f921006bbd0 0x7f921006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.259+0000 7f9221ffb640 1 -- 172.21.15.82:0/1292164003 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92280ad410 con 0x7f922c06ad10 2024-03-14T12:14:52.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.259+0000 7f9230ae0640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f921006bbd0 0x7f921006e090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f921c006fd0 tx=0x7f921c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.263+0000 7f9221ffb640 1 -- 172.21.15.82:0/1292164003 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f922804d030 con 0x7f922c06ad10 2024-03-14T12:14:52.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.435+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f922c0b1ff0 con 0x7f922c06ad10 2024-03-14T12:14:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.436+0000 7f9221ffb640 1 -- 172.21.15.82:0/1292164003 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f922807bf00 con 0x7f922c06ad10 2024-03-14T12:14:52.437 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:52.437 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:50.791329+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.438+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f921006bbd0 msgr2=0x7f921006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.438+0000 7f9233545640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f921006bbd0 0x7f921006e090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f921c006fd0 tx=0x7f921c008040 comp rx=0 tx=0).stop 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.438+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c06ad10 msgr2=0x7f922c099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.438+0000 7f9233545640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c06ad10 0x7f922c099a30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9228009e50 tx=0x7f9228004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 shutdown_connections 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f921006bbd0 0x7f921006e090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f922c0b0f60 0x7f922c09a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f922c0afff0 0x7f922c09a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 --2- 172.21.15.82:0/1292164003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f922c06ad10 0x7f922c099a30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 >> 172.21.15.82:0/1292164003 conn(0x7f922c072310 msgr2=0x7f922c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 shutdown_connections 2024-03-14T12:14:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.439+0000 7f9233545640 1 -- 172.21.15.82:0/1292164003 wait complete. 2024-03-14T12:14:52.450 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T12:14:52.450 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:14:52.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.551+0000 7f1ecff62640 1 Processor -- start 2024-03-14T12:14:52.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.551+0000 7f1ecff62640 1 -- start start 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecff62640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ec8071740 0x7f1ec8071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecff62640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1ec8143f60 0x7f1ec815b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecff62640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec80b3050 con 0x7f1ec815e670 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecff62640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec8072060 con 0x7f1ec8071740 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecff62640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec815ebe0 con 0x7f1ec8143f60 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecdcfe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ec8071740 0x7f1ec8071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecd4fd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1ec815e670 0x7f1ec80b2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52598/0 (socket says 172.21.15.82:52598) 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecd4fd640 1 -- 172.21.15.82:0/1252506470 learned_addr learned my addr 172.21.15.82:0/1252506470 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ece4ff640 1 --2- 172.21.15.82:0/1252506470 >> v2:172.21.15.164:3300/0 conn(0x7f1ec8143f60 0x7f1ec815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecdcfe640 1 -- 172.21.15.82:0/1252506470 >> v2:172.21.15.164:3300/0 conn(0x7f1ec8143f60 msgr2=0x7f1ec815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecdcfe640 1 --2- 172.21.15.82:0/1252506470 >> v2:172.21.15.164:3300/0 conn(0x7f1ec8143f60 0x7f1ec815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecdcfe640 1 -- 172.21.15.82:0/1252506470 >> v1:172.21.15.82:6789/0 conn(0x7f1ec815e670 legacy=0x7f1ec80b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecccfc640 1 -- 172.21.15.82:0/1252506470 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724241985 0 0) 0x7f1ec80b3050 con 0x7f1ec815e670 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.552+0000 7f1ecdcfe640 1 -- 172.21.15.82:0/1252506470 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec815eec0 con 0x7f1ec8071740 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.553+0000 7f1ecdcfe640 1 --2- 172.21.15.82:0/1252506470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec8071740 0x7f1ec8071b20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1ec815f790 tx=0x7f1ec402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce540934a503eace server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.553+0000 7f1ecccfc640 1 -- 172.21.15.82:0/1252506470 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4035020 con 0x7f1ec8071740 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.553+0000 7f1ecccfc640 1 -- 172.21.15.82:0/1252506470 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec4031830 con 0x7f1ec8071740 2024-03-14T12:14:52.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.553+0000 7f1ecccfc640 1 -- 172.21.15.82:0/1252506470 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4031c00 con 0x7f1ec8071740 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.553+0000 7f1ecff62640 1 -- 172.21.15.82:0/1252506470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec8071740 msgr2=0x7f1ec8071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.553+0000 7f1ecff62640 1 --2- 172.21.15.82:0/1252506470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec8071740 0x7f1ec8071b20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1ec815f790 tx=0x7f1ec402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.554+0000 7f1ecff62640 1 -- 172.21.15.82:0/1252506470 shutdown_connections 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.554+0000 7f1ecff62640 1 --2- 172.21.15.82:0/1252506470 >> v2:172.21.15.164:3300/0 conn(0x7f1ec8143f60 0x7f1ec815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.554+0000 7f1ecff62640 1 --2- 172.21.15.82:0/1252506470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec8071740 0x7f1ec8071b20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.554+0000 7f1ecff62640 1 -- 172.21.15.82:0/1252506470 >> 172.21.15.82:0/1252506470 conn(0x7f1ec80b1c70 msgr2=0x7f1ec80cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.554+0000 7f1ecff62640 1 -- 172.21.15.82:0/1252506470 shutdown_connections 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.554+0000 7f1ecff62640 1 -- 172.21.15.82:0/1252506470 wait complete. 2024-03-14T12:14:52.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 Processor -- start 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 -- start start 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec8071740 0x7f1ec813c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ec8143f60 0x7f1ec815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec815e670 0x7f1ec815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec8143ce0 con 0x7f1ec815e670 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec815f480 con 0x7f1ec8071740 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ecff62640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec815b080 con 0x7f1ec8143f60 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.555+0000 7f1ece4ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec815e670 0x7f1ec815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec815e670 0x7f1ec815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58738/0 (socket says 172.21.15.82:58738) 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 -- 172.21.15.82:0/2753249671 learned_addr learned my addr 172.21.15.82:0/2753249671 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ecdcfe640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec8071740 0x7f1ec813c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ecd4fd640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ec8143f60 0x7f1ec815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 -- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ec8143f60 msgr2=0x7f1ec815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ec8143f60 0x7f1ec815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 -- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec8071740 msgr2=0x7f1ec813c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec8071740 0x7f1ec813c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 -- 172.21.15.82:0/2753249671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec81496e0 con 0x7f1ec815e670 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ece4ff640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec815e670 0x7f1ec815e070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007e70 tx=0x7f1ec000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ebe7fc640 1 -- 172.21.15.82:0/2753249671 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0014620 con 0x7f1ec815e670 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ebe7fc640 1 -- 172.21.15.82:0/2753249671 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec0019070 con 0x7f1ec815e670 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec81499c0 con 0x7f1ec815e670 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ebe7fc640 1 -- 172.21.15.82:0/2753249671 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0014a20 con 0x7f1ec815e670 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.556+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec8149f80 con 0x7f1ec815e670 2024-03-14T12:14:52.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.557+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec804f050 con 0x7f1ec815e670 2024-03-14T12:14:52.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.558+0000 7f1ebe7fc640 1 -- 172.21.15.82:0/2753249671 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1ec0014bc0 con 0x7f1ec815e670 2024-03-14T12:14:52.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.558+0000 7f1ebe7fc640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1eac06bbd0 0x7f1eac06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.558+0000 7f1ecdcfe640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1eac06bbd0 0x7f1eac06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.558+0000 7f1ebe7fc640 1 -- 172.21.15.82:0/2753249671 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ec008b380 con 0x7f1ec815e670 2024-03-14T12:14:52.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.558+0000 7f1ecdcfe640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1eac06bbd0 0x7f1eac06e090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4036000 tx=0x7f1ec403c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.562+0000 7f1ebe7fc640 1 -- 172.21.15.82:0/2753249671 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1ec0059e70 con 0x7f1ec815e670 2024-03-14T12:14:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.730+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ec814ab40 con 0x7f1eac06bbd0 2024-03-14T12:14:52.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.735+0000 7f1ebe7fc640 1 -- 172.21.15.82:0/2753249671 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239830 (secure 0 0 0) 0x7f1ec403a420 con 0x7f1eac06bbd0 2024-03-14T12:14:52.738 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:14:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1eac06bbd0 msgr2=0x7f1eac06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1eac06bbd0 0x7f1eac06e090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4036000 tx=0x7f1ec403c040 comp rx=0 tx=0).stop 2024-03-14T12:14:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec815e670 msgr2=0x7f1ec815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec815e670 0x7f1ec815e070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007e70 tx=0x7f1ec000e4b0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 shutdown_connections 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ec815e670 0x7f1ec815e070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1eac06bbd0 0x7f1eac06e090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ec8143f60 0x7f1ec815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 --2- 172.21.15.82:0/2753249671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ec8071740 0x7f1ec813c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 >> 172.21.15.82:0/2753249671 conn(0x7f1ec80b1c70 msgr2=0x7f1ec80cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 shutdown_connections 2024-03-14T12:14:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.738+0000 7f1ecff62640 1 -- 172.21.15.82:0/2753249671 wait complete. 2024-03-14T12:14:52.749 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-14T12:14:51.508896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220004,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754716,"statfs":{"total":773094113280,"available":772868829184,"internally_reserved":0,"allocated":3936256,"data_stored":2214229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000656"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779857+0000","last_change":"2024-03-14T12:14:47.779857+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779857+0000","last_undegraded":"2024-03-14T12:14:47.779857+0000","last_fullsized":"2024-03-14T12:14:47.779857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775646+0000","last_change":"2024-03-14T12:14:47.775646+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775646+0000","last_undegraded":"2024-03-14T12:14:47.775646+0000","last_fullsized":"2024-03-14T12:14:47.775646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776925+0000","last_change":"2024-03-14T12:14:47.776925+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776925+0000","last_undegraded":"2024-03-14T12:14:47.776925+0000","last_fullsized":"2024-03-14T12:14:47.776925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779301+0000","last_change":"2024-03-14T12:14:47.779301+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779301+0000","last_undegraded":"2024-03-14T12:14:47.779301+0000","last_fullsized":"2024-03-14T12:14:47.779301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778241+0000","last_change":"2024-03-14T12:14:47.778241+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778241+0000","last_undegraded":"2024-03-14T12:14:47.778241+0000","last_fullsized":"2024-03-14T12:14:47.778241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777372+0000","last_change":"2024-03-14T12:14:47.777372+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777372+0000","last_undegraded":"2024-03-14T12:14:47.777372+0000","last_fullsized":"2024-03-14T12:14:47.777372+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778295+0000","last_change":"2024-03-14T12:14:47.778295+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778295+0000","last_undegraded":"2024-03-14T12:14:47.778295+0000","last_fullsized":"2024-03-14T12:14:47.778295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777214+0000","last_change":"2024-03-14T12:14:47.777214+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777214+0000","last_undegraded":"2024-03-14T12:14:47.777214+0000","last_fullsized":"2024-03-14T12:14:47.777214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776942+0000","last_change":"2024-03-14T12:14:47.776942+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776942+0000","last_undegraded":"2024-03-14T12:14:47.776942+0000","last_fullsized":"2024-03-14T12:14:47.776942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775209+0000","last_change":"2024-03-14T12:14:47.775209+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775209+0000","last_undegraded":"2024-03-14T12:14:47.775209+0000","last_fullsized":"2024-03-14T12:14:47.775209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777138+0000","last_change":"2024-03-14T12:14:47.777138+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777138+0000","last_undegraded":"2024-03-14T12:14:47.777138+0000","last_fullsized":"2024-03-14T12:14:47.777138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777704+0000","last_change":"2024-03-14T12:14:47.777704+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777704+0000","last_undegraded":"2024-03-14T12:14:47.777704+0000","last_fullsized":"2024-03-14T12:14:47.777704+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776875+0000","last_change":"2024-03-14T12:14:47.776875+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776875+0000","last_undegraded":"2024-03-14T12:14:47.776875+0000","last_fullsized":"2024-03-14T12:14:47.776875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777451+0000","last_change":"2024-03-14T12:14:47.777451+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777451+0000","last_undegraded":"2024-03-14T12:14:47.777451+0000","last_fullsized":"2024-03-14T12:14:47.777451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778261+0000","last_change":"2024-03-14T12:14:47.778261+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778261+0000","last_undegraded":"2024-03-14T12:14:47.778261+0000","last_fullsized":"2024-03-14T12:14:47.778261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776920+0000","last_change":"2024-03-14T12:14:47.776920+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776920+0000","last_undegraded":"2024-03-14T12:14:47.776920+0000","last_fullsized":"2024-03-14T12:14:47.776920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777049+0000","last_change":"2024-03-14T12:14:47.777049+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777049+0000","last_undegraded":"2024-03-14T12:14:47.777049+0000","last_fullsized":"2024-03-14T12:14:47.777049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777110+0000","last_change":"2024-03-14T12:14:47.777110+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777110+0000","last_undegraded":"2024-03-14T12:14:47.777110+0000","last_fullsized":"2024-03-14T12:14:47.777110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777282+0000","last_change":"2024-03-14T12:14:47.777282+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777282+0000","last_undegraded":"2024-03-14T12:14:47.777282+0000","last_fullsized":"2024-03-14T12:14:47.777282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774990+0000","last_change":"2024-03-14T12:14:47.774990+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774990+0000","last_undegraded":"2024-03-14T12:14:47.774990+0000","last_fullsized":"2024-03-14T12:14:47.774990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779678+0000","last_change":"2024-03-14T12:14:47.779678+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779678+0000","last_undegraded":"2024-03-14T12:14:47.779678+0000","last_fullsized":"2024-03-14T12:14:47.779678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778547+0000","last_change":"2024-03-14T12:14:47.778547+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778547+0000","last_undegraded":"2024-03-14T12:14:47.778547+0000","last_fullsized":"2024-03-14T12:14:47.778547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776413+0000","last_change":"2024-03-14T12:14:47.776413+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776413+0000","last_undegraded":"2024-03-14T12:14:47.776413+0000","last_fullsized":"2024-03-14T12:14:47.776413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774743+0000","last_change":"2024-03-14T12:14:47.774743+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774743+0000","last_undegraded":"2024-03-14T12:14:47.774743+0000","last_fullsized":"2024-03-14T12:14:47.774743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776396+0000","last_change":"2024-03-14T12:14:47.776396+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776396+0000","last_undegraded":"2024-03-14T12:14:47.776396+0000","last_fullsized":"2024-03-14T12:14:47.776396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774511+0000","last_change":"2024-03-14T12:14:47.774511+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774511+0000","last_undegraded":"2024-03-14T12:14:47.774511+0000","last_fullsized":"2024-03-14T12:14:47.774511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776371+0000","last_change":"2024-03-14T12:14:47.776371+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776371+0000","last_undegraded":"2024-03-14T12:14:47.776371+0000","last_fullsized":"2024-03-14T12:14:47.776371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776952+0000","last_change":"2024-03-14T12:14:47.776952+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776952+0000","last_undegraded":"2024-03-14T12:14:47.776952+0000","last_fullsized":"2024-03-14T12:14:47.776952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777845+0000","last_change":"2024-03-14T12:14:47.777845+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777845+0000","last_undegraded":"2024-03-14T12:14:47.777845+0000","last_fullsized":"2024-03-14T12:14:47.777845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779722+0000","last_change":"2024-03-14T12:14:47.779722+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779722+0000","last_undegraded":"2024-03-14T12:14:47.779722+0000","last_fullsized":"2024-03-14T12:14:47.779722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T12:14:47.781149+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:14:47.781149+0000","last_peered":"2024-03-14T12:14:47.781149+0000","last_clean":"2024-03-14T12:14:47.781149+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:14:47.781149+0000","last_undegraded":"2024-03-14T12:14:47.781149+0000","last_fullsized":"2024-03-14T12:14:47.781149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:14:52.753 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:14:52.761 INFO:tasks.ceph.mds.a.smithi082.stderr:2024-03-14T12:14:52.761+0000 7f6548c77640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:14:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.861+0000 7f4cff0e9640 1 Processor -- start 2024-03-14T12:14:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.861+0000 7f4cff0e9640 1 -- start start 2024-03-14T12:14:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cff0e9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4cf815dc10 0x7f4cf8161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cff0e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 0x7f4cf8161f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cff0e9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80b33f0 con 0x7f4cf81624a0 2024-03-14T12:14:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cff0e9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf80b36d0 con 0x7f4cf815dc10 2024-03-14T12:14:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cff0e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80b39b0 con 0x7f4cf815dff0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cfd686640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4cf81624a0 0x7f4cf8071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52606/0 (socket says 172.21.15.82:52606) 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cfd686640 1 -- 172.21.15.82:0/1315843744 learned_addr learned my addr 172.21.15.82:0/1315843744 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cfce85640 1 --2- 172.21.15.82:0/1315843744 >> v2:172.21.15.164:3300/0 conn(0x7f4cf815dc10 0x7f4cf8161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf7fff640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 0x7f4cf8161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920659205 0 0) 0x7f4cf80b33f0 con 0x7f4cf81624a0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce8003520 con 0x7f4cf81624a0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024745454 0 0) 0x7f4ce8003520 con 0x7f4cf81624a0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 msgr2=0x7f4cf8161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 0x7f4cf8161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 >> v2:172.21.15.164:3300/0 conn(0x7f4cf815dc10 msgr2=0x7f4cf8161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 --2- 172.21.15.82:0/1315843744 >> v2:172.21.15.164:3300/0 conn(0x7f4cf815dc10 0x7f4cf8161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80b3c90 con 0x7f4cf81624a0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f4cf00019f0 con 0x7f4cf81624a0 2024-03-14T12:14:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 >> v1:172.21.15.82:6789/0 conn(0x7f4cf81624a0 legacy=0x7f4cf8071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80039b0 0x7f4ce8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 0x7f4ce8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce8004d40 0x7f4ce80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80b36d0 con 0x7f4ce8004d40 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80b39b0 con 0x7f4ce80039b0 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce800b9c0 con 0x7f4cf815dff0 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.862+0000 7f4cfd686640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce8004d40 0x7f4ce80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cfce85640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80039b0 0x7f4ce8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cf7fff640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 0x7f4ce8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cfd686640 1 -- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 msgr2=0x7f4ce8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cfd686640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf815dff0 0x7f4ce8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cfd686640 1 -- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80039b0 msgr2=0x7f4ce8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cfd686640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce80039b0 0x7f4ce8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cfd686640 1 -- 172.21.15.82:0/1315843744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8003520 con 0x7f4ce8004d40 2024-03-14T12:14:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.863+0000 7f4cfd686640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce8004d40 0x7f4ce80051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000a870 tx=0x7f4ce8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9cfc574e8708a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf000a4c0 con 0x7f4ce8004d40 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0034630 con 0x7f4ce8004d40 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cf77fe640 1 -- 172.21.15.82:0/1315843744 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0034a20 con 0x7f4ce8004d40 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cff0e9640 1 -- 172.21.15.82:0/1315843744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce8004d40 msgr2=0x7f4ce80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cff0e9640 1 --2- 172.21.15.82:0/1315843744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce8004d40 0x7f4ce80051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000a870 tx=0x7f4ce8005830 comp rx=0 tx=0).stop 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cfd686640 1 -- 172.21.15.82:0/1315843744 reap_dead start 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cff0e9640 1 -- 172.21.15.82:0/1315843744 shutdown_connections 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cff0e9640 1 -- 172.21.15.82:0/1315843744 >> 172.21.15.82:0/1315843744 conn(0x7f4cf80b1c70 msgr2=0x7f4cf80cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cff0e9640 1 -- 172.21.15.82:0/1315843744 shutdown_connections 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.864+0000 7f4cff0e9640 1 -- 172.21.15.82:0/1315843744 wait complete. 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.865+0000 7f4cff0e9640 1 Processor -- start 2024-03-14T12:14:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.865+0000 7f4cff0e9640 1 -- start start 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cff0e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf815c480 0x7f4cf8155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cff0e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf815c860 0x7f4cf81557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cff0e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf80ae2a0 0x7f4cf80ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cff0e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf81620e0 con 0x7f4cf815c480 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cff0e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8161f60 con 0x7f4cf815c860 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cff0e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8162260 con 0x7f4cf80ae2a0 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cfce85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf815c480 0x7f4cf8155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cfce85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf815c480 0x7f4cf8155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58758/0 (socket says 172.21.15.82:58758) 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cfce85640 1 -- 172.21.15.82:0/4076405642 learned_addr learned my addr 172.21.15.82:0/4076405642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cfd686640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf80ae2a0 0x7f4cf80ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.866+0000 7f4cf7fff640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf815c860 0x7f4cf81557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cfce85640 1 -- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf80ae2a0 msgr2=0x7f4cf80ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cfce85640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf80ae2a0 0x7f4cf80ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cfce85640 1 -- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf815c860 msgr2=0x7f4cf81557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cfce85640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf815c860 0x7f4cf81557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cfce85640 1 -- 172.21.15.82:0/4076405642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80af010 con 0x7f4cf815c480 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cf7fff640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf815c860 0x7f4cf81557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cfd686640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf80ae2a0 0x7f4cf80ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cfce85640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf815c480 0x7f4cf8155290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011960 tx=0x7f4cec011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cf57fa640 1 -- 172.21.15.82:0/4076405642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00faf0 con 0x7f4cf815c480 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cf57fa640 1 -- 172.21.15.82:0/4076405642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec00fc90 con 0x7f4cf815c480 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf80af2f0 con 0x7f4cf815c480 2024-03-14T12:14:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cf57fa640 1 -- 172.21.15.82:0/4076405642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec014440 con 0x7f4cf815c480 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.867+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf80af8b0 con 0x7f4cf815c480 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.868+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf804f050 con 0x7f4cf815c480 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.868+0000 7f4cf57fa640 1 -- 172.21.15.82:0/4076405642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4cec012040 con 0x7f4cf815c480 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.869+0000 7f4cf57fa640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4cdc06bc00 0x7f4cdc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.869+0000 7f4cf57fa640 1 -- 172.21.15.82:0/4076405642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cec08b0c0 con 0x7f4cf815c480 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.869+0000 7f4cf7fff640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4cdc06bc00 0x7f4cdc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.869+0000 7f4cf7fff640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4cdc06bc00 0x7f4cdc06e0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4cf81f1770 tx=0x7f4ce4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:52.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:52.873+0000 7f4cf57fa640 1 -- 172.21.15.82:0/4076405642 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4cec019030 con 0x7f4cf815c480 2024-03-14T12:14:53.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.039+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4cf8159af0 con 0x7f4cdc06bc00 2024-03-14T12:14:53.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.045+0000 7f4cf57fa640 1 -- 172.21.15.82:0/4076405642 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239830 (secure 0 0 0) 0x7f4cf8159af0 con 0x7f4cdc06bc00 2024-03-14T12:14:53.047 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4cdc06bc00 msgr2=0x7f4cdc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4cdc06bc00 0x7f4cdc06e0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4cf81f1770 tx=0x7f4ce4008040 comp rx=0 tx=0).stop 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf815c480 msgr2=0x7f4cf8155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf815c480 0x7f4cf8155290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4cec011960 tx=0x7f4cec011e20 comp rx=0 tx=0).stop 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 shutdown_connections 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4cdc06bc00 0x7f4cdc06e0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cf80ae2a0 0x7f4cf80ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf815c860 0x7f4cf81557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 --2- 172.21.15.82:0/4076405642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf815c480 0x7f4cf8155290 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 >> 172.21.15.82:0/4076405642 conn(0x7f4cf80b1c70 msgr2=0x7f4cf81608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:53.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 shutdown_connections 2024-03-14T12:14:53.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.048+0000 7f4cff0e9640 1 -- 172.21.15.82:0/4076405642 wait complete. 2024-03-14T12:14:53.059 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-14T12:14:51.508896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220004,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754716,"statfs":{"total":773094113280,"available":772868829184,"internally_reserved":0,"allocated":3936256,"data_stored":2214229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000656"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779857+0000","last_change":"2024-03-14T12:14:47.779857+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779857+0000","last_undegraded":"2024-03-14T12:14:47.779857+0000","last_fullsized":"2024-03-14T12:14:47.779857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775646+0000","last_change":"2024-03-14T12:14:47.775646+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775646+0000","last_undegraded":"2024-03-14T12:14:47.775646+0000","last_fullsized":"2024-03-14T12:14:47.775646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776925+0000","last_change":"2024-03-14T12:14:47.776925+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776925+0000","last_undegraded":"2024-03-14T12:14:47.776925+0000","last_fullsized":"2024-03-14T12:14:47.776925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779301+0000","last_change":"2024-03-14T12:14:47.779301+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779301+0000","last_undegraded":"2024-03-14T12:14:47.779301+0000","last_fullsized":"2024-03-14T12:14:47.779301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778241+0000","last_change":"2024-03-14T12:14:47.778241+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778241+0000","last_undegraded":"2024-03-14T12:14:47.778241+0000","last_fullsized":"2024-03-14T12:14:47.778241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777372+0000","last_change":"2024-03-14T12:14:47.777372+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777372+0000","last_undegraded":"2024-03-14T12:14:47.777372+0000","last_fullsized":"2024-03-14T12:14:47.777372+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778295+0000","last_change":"2024-03-14T12:14:47.778295+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778295+0000","last_undegraded":"2024-03-14T12:14:47.778295+0000","last_fullsized":"2024-03-14T12:14:47.778295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777214+0000","last_change":"2024-03-14T12:14:47.777214+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777214+0000","last_undegraded":"2024-03-14T12:14:47.777214+0000","last_fullsized":"2024-03-14T12:14:47.777214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776942+0000","last_change":"2024-03-14T12:14:47.776942+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776942+0000","last_undegraded":"2024-03-14T12:14:47.776942+0000","last_fullsized":"2024-03-14T12:14:47.776942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775209+0000","last_change":"2024-03-14T12:14:47.775209+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775209+0000","last_undegraded":"2024-03-14T12:14:47.775209+0000","last_fullsized":"2024-03-14T12:14:47.775209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777138+0000","last_change":"2024-03-14T12:14:47.777138+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777138+0000","last_undegraded":"2024-03-14T12:14:47.777138+0000","last_fullsized":"2024-03-14T12:14:47.777138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777704+0000","last_change":"2024-03-14T12:14:47.777704+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777704+0000","last_undegraded":"2024-03-14T12:14:47.777704+0000","last_fullsized":"2024-03-14T12:14:47.777704+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776875+0000","last_change":"2024-03-14T12:14:47.776875+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776875+0000","last_undegraded":"2024-03-14T12:14:47.776875+0000","last_fullsized":"2024-03-14T12:14:47.776875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777451+0000","last_change":"2024-03-14T12:14:47.777451+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777451+0000","last_undegraded":"2024-03-14T12:14:47.777451+0000","last_fullsized":"2024-03-14T12:14:47.777451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778261+0000","last_change":"2024-03-14T12:14:47.778261+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778261+0000","last_undegraded":"2024-03-14T12:14:47.778261+0000","last_fullsized":"2024-03-14T12:14:47.778261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776920+0000","last_change":"2024-03-14T12:14:47.776920+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776920+0000","last_undegraded":"2024-03-14T12:14:47.776920+0000","last_fullsized":"2024-03-14T12:14:47.776920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777049+0000","last_change":"2024-03-14T12:14:47.777049+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777049+0000","last_undegraded":"2024-03-14T12:14:47.777049+0000","last_fullsized":"2024-03-14T12:14:47.777049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777110+0000","last_change":"2024-03-14T12:14:47.777110+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777110+0000","last_undegraded":"2024-03-14T12:14:47.777110+0000","last_fullsized":"2024-03-14T12:14:47.777110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777282+0000","last_change":"2024-03-14T12:14:47.777282+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777282+0000","last_undegraded":"2024-03-14T12:14:47.777282+0000","last_fullsized":"2024-03-14T12:14:47.777282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774990+0000","last_change":"2024-03-14T12:14:47.774990+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774990+0000","last_undegraded":"2024-03-14T12:14:47.774990+0000","last_fullsized":"2024-03-14T12:14:47.774990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779678+0000","last_change":"2024-03-14T12:14:47.779678+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779678+0000","last_undegraded":"2024-03-14T12:14:47.779678+0000","last_fullsized":"2024-03-14T12:14:47.779678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778547+0000","last_change":"2024-03-14T12:14:47.778547+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778547+0000","last_undegraded":"2024-03-14T12:14:47.778547+0000","last_fullsized":"2024-03-14T12:14:47.778547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776413+0000","last_change":"2024-03-14T12:14:47.776413+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776413+0000","last_undegraded":"2024-03-14T12:14:47.776413+0000","last_fullsized":"2024-03-14T12:14:47.776413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774743+0000","last_change":"2024-03-14T12:14:47.774743+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774743+0000","last_undegraded":"2024-03-14T12:14:47.774743+0000","last_fullsized":"2024-03-14T12:14:47.774743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776396+0000","last_change":"2024-03-14T12:14:47.776396+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776396+0000","last_undegraded":"2024-03-14T12:14:47.776396+0000","last_fullsized":"2024-03-14T12:14:47.776396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774511+0000","last_change":"2024-03-14T12:14:47.774511+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774511+0000","last_undegraded":"2024-03-14T12:14:47.774511+0000","last_fullsized":"2024-03-14T12:14:47.774511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776371+0000","last_change":"2024-03-14T12:14:47.776371+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776371+0000","last_undegraded":"2024-03-14T12:14:47.776371+0000","last_fullsized":"2024-03-14T12:14:47.776371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776952+0000","last_change":"2024-03-14T12:14:47.776952+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776952+0000","last_undegraded":"2024-03-14T12:14:47.776952+0000","last_fullsized":"2024-03-14T12:14:47.776952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777845+0000","last_change":"2024-03-14T12:14:47.777845+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777845+0000","last_undegraded":"2024-03-14T12:14:47.777845+0000","last_fullsized":"2024-03-14T12:14:47.777845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779722+0000","last_change":"2024-03-14T12:14:47.779722+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779722+0000","last_undegraded":"2024-03-14T12:14:47.779722+0000","last_fullsized":"2024-03-14T12:14:47.779722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T12:14:47.781149+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:14:47.781149+0000","last_peered":"2024-03-14T12:14:47.781149+0000","last_clean":"2024-03-14T12:14:47.781149+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:14:47.781149+0000","last_undegraded":"2024-03-14T12:14:47.781149+0000","last_fullsized":"2024-03-14T12:14:47.781149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:14:53.062 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:14:53.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.200+0000 7f68e4d3b640 1 Processor -- start 2024-03-14T12:14:53.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.200+0000 7f68e4d3b640 1 -- start start 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68e4d3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 0x7f68e0071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68e4d3b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e00b3050 con 0x7f68e015e670 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68e4d3b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e0072060 con 0x7f68e0143f60 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68e4d3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e015ebe0 con 0x7f68e0071740 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68d7fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f68e015e670 0x7f68e00b2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52618/0 (socket says 172.21.15.82:52618) 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68d7fff640 1 -- 172.21.15.82:0/1595323396 learned_addr learned my addr 172.21.15.82:0/1595323396 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135091252 0 0) 0x7f68e00b3050 con 0x7f68e015e670 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68de59c640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 0x7f68e0071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68cc003520 con 0x7f68e015e670 2024-03-14T12:14:53.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390989349 0 0) 0x7f68cc003520 con 0x7f68e015e670 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 msgr2=0x7f68e0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 0x7f68e0071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 >> v1:172.21.15.164:6789/0 conn(0x7f68e0143f60 legacy=0x7f68e015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e015eec0 con 0x7f68e015e670 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f68c8001670 con 0x7f68e015e670 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 >> v1:172.21.15.82:6789/0 conn(0x7f68e015e670 legacy=0x7f68e00b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc0039e0 0x7f68cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68cc004800 0x7f68cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 0x7f68cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e015ebe0 con 0x7f68cc004800 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc00ba70 con 0x7f68cc0039e0 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc00bc70 con 0x7f68e0071740 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.201+0000 7f68d7fff640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68cc004800 0x7f68cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68de59c640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc0039e0 0x7f68cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68ded9d640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 0x7f68cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68d7fff640 1 -- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 msgr2=0x7f68cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68d7fff640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 0x7f68cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68d7fff640 1 -- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc0039e0 msgr2=0x7f68cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68d7fff640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc0039e0 0x7f68cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68d7fff640 1 -- 172.21.15.82:0/1595323396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc003520 con 0x7f68cc004800 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68ded9d640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e0071740 0x7f68cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.202+0000 7f68d7fff640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68cc004800 0x7f68cc004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c210 tx=0x7f68cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68ff665f0b633f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8007cf0 con 0x7f68cc004800 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c8030070 con 0x7f68cc004800 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68ddd9b640 1 -- 172.21.15.82:0/1595323396 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8005d30 con 0x7f68cc004800 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1595323396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68cc004800 msgr2=0x7f68cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68e4d3b640 1 --2- 172.21.15.82:0/1595323396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68cc004800 0x7f68cc004c80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c210 tx=0x7f68cc005830 comp rx=0 tx=0).stop 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68ded9d640 1 -- 172.21.15.82:0/1595323396 reap_dead start 2024-03-14T12:14:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1595323396 shutdown_connections 2024-03-14T12:14:53.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1595323396 >> 172.21.15.82:0/1595323396 conn(0x7f68e00b1c70 msgr2=0x7f68e00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:53.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1595323396 shutdown_connections 2024-03-14T12:14:53.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.203+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1595323396 wait complete. 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.204+0000 7f68e4d3b640 1 Processor -- start 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.204+0000 7f68e4d3b640 1 -- start start 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68e4d3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc004800 0x7f68e0157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68e4d3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e015c560 0x7f68e015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68e4d3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e00cda00 0x7f68e00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68e4d3b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e0143ce0 con 0x7f68e015c560 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68e4d3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e015f480 con 0x7f68cc004800 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68e4d3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e015b080 con 0x7f68e00cda00 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e015c560 0x7f68e015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e015c560 0x7f68e015c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58790/0 (socket says 172.21.15.82:58790) 2024-03-14T12:14:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68d7fff640 1 -- 172.21.15.82:0/1898910832 learned_addr learned my addr 172.21.15.82:0/1898910832 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68de59c640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc004800 0x7f68e0157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.205+0000 7f68ded9d640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e00cda00 0x7f68e00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d7fff640 1 -- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e00cda00 msgr2=0x7f68e00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d7fff640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e00cda00 0x7f68e00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d7fff640 1 -- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc004800 msgr2=0x7f68e0157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d7fff640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc004800 0x7f68e0157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d7fff640 1 -- 172.21.15.82:0/1898910832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e01584f0 con 0x7f68e015c560 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68de59c640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc004800 0x7f68e0157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68ded9d640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e00cda00 0x7f68e00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d7fff640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e015c560 0x7f68e015c9e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c210 tx=0x7f68e015f090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d6ffd640 1 -- 172.21.15.82:0/1898910832 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8001c30 con 0x7f68e015c560 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d6ffd640 1 -- 172.21.15.82:0/1898910832 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c8004bf0 con 0x7f68e015c560 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e00ae6f0 con 0x7f68e015c560 2024-03-14T12:14:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68d6ffd640 1 -- 172.21.15.82:0/1898910832 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8007070 con 0x7f68e015c560 2024-03-14T12:14:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.206+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e00aecb0 con 0x7f68e015c560 2024-03-14T12:14:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.207+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e015e460 con 0x7f68e015c560 2024-03-14T12:14:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.207+0000 7f68d6ffd640 1 -- 172.21.15.82:0/1898910832 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f68c8007210 con 0x7f68e015c560 2024-03-14T12:14:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.208+0000 7f68d6ffd640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f68c006bc00 0x7f68c006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.208+0000 7f68d6ffd640 1 -- 172.21.15.82:0/1898910832 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68c80ac000 con 0x7f68e015c560 2024-03-14T12:14:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.208+0000 7f68de59c640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f68c006bc00 0x7f68c006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.208+0000 7f68de59c640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f68c006bc00 0x7f68c006e0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009bc0 tx=0x7f68d0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.212+0000 7f68d6ffd640 1 -- 172.21.15.82:0/1898910832 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f68e015e460 con 0x7f68e015c560 2024-03-14T12:14:53.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.380+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68e004f050 con 0x7f68c006bc00 2024-03-14T12:14:53.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.385+0000 7f68d6ffd640 1 -- 172.21.15.82:0/1898910832 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239830 (secure 0 0 0) 0x7f68e004f050 con 0x7f68c006bc00 2024-03-14T12:14:53.388 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:53.389 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f68c006bc00 msgr2=0x7f68c006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f68c006bc00 0x7f68c006e0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009bc0 tx=0x7f68d0008040 comp rx=0 tx=0).stop 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e015c560 msgr2=0x7f68e015c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e015c560 0x7f68e015c9e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c210 tx=0x7f68e015f090 comp rx=0 tx=0).stop 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 shutdown_connections 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f68c006bc00 0x7f68c006e0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68e00cda00 0x7f68e00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e015c560 0x7f68e015c9e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 --2- 172.21.15.82:0/1898910832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68cc004800 0x7f68e0157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 >> 172.21.15.82:0/1898910832 conn(0x7f68e00b1c70 msgr2=0x7f68e00b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 shutdown_connections 2024-03-14T12:14:53.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.388+0000 7f68e4d3b640 1 -- 172.21.15.82:0/1898910832 wait complete. 2024-03-14T12:14:53.399 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-14T12:14:51.508896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220004,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754716,"statfs":{"total":773094113280,"available":772868829184,"internally_reserved":0,"allocated":3936256,"data_stored":2214229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000656"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779857+0000","last_change":"2024-03-14T12:14:47.779857+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779857+0000","last_undegraded":"2024-03-14T12:14:47.779857+0000","last_fullsized":"2024-03-14T12:14:47.779857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775646+0000","last_change":"2024-03-14T12:14:47.775646+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775646+0000","last_undegraded":"2024-03-14T12:14:47.775646+0000","last_fullsized":"2024-03-14T12:14:47.775646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776925+0000","last_change":"2024-03-14T12:14:47.776925+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776925+0000","last_undegraded":"2024-03-14T12:14:47.776925+0000","last_fullsized":"2024-03-14T12:14:47.776925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779301+0000","last_change":"2024-03-14T12:14:47.779301+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779301+0000","last_undegraded":"2024-03-14T12:14:47.779301+0000","last_fullsized":"2024-03-14T12:14:47.779301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778241+0000","last_change":"2024-03-14T12:14:47.778241+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778241+0000","last_undegraded":"2024-03-14T12:14:47.778241+0000","last_fullsized":"2024-03-14T12:14:47.778241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777372+0000","last_change":"2024-03-14T12:14:47.777372+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777372+0000","last_undegraded":"2024-03-14T12:14:47.777372+0000","last_fullsized":"2024-03-14T12:14:47.777372+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778295+0000","last_change":"2024-03-14T12:14:47.778295+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778295+0000","last_undegraded":"2024-03-14T12:14:47.778295+0000","last_fullsized":"2024-03-14T12:14:47.778295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777214+0000","last_change":"2024-03-14T12:14:47.777214+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777214+0000","last_undegraded":"2024-03-14T12:14:47.777214+0000","last_fullsized":"2024-03-14T12:14:47.777214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776942+0000","last_change":"2024-03-14T12:14:47.776942+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776942+0000","last_undegraded":"2024-03-14T12:14:47.776942+0000","last_fullsized":"2024-03-14T12:14:47.776942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775209+0000","last_change":"2024-03-14T12:14:47.775209+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775209+0000","last_undegraded":"2024-03-14T12:14:47.775209+0000","last_fullsized":"2024-03-14T12:14:47.775209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777138+0000","last_change":"2024-03-14T12:14:47.777138+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777138+0000","last_undegraded":"2024-03-14T12:14:47.777138+0000","last_fullsized":"2024-03-14T12:14:47.777138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777704+0000","last_change":"2024-03-14T12:14:47.777704+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777704+0000","last_undegraded":"2024-03-14T12:14:47.777704+0000","last_fullsized":"2024-03-14T12:14:47.777704+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776875+0000","last_change":"2024-03-14T12:14:47.776875+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776875+0000","last_undegraded":"2024-03-14T12:14:47.776875+0000","last_fullsized":"2024-03-14T12:14:47.776875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777451+0000","last_change":"2024-03-14T12:14:47.777451+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777451+0000","last_undegraded":"2024-03-14T12:14:47.777451+0000","last_fullsized":"2024-03-14T12:14:47.777451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778261+0000","last_change":"2024-03-14T12:14:47.778261+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778261+0000","last_undegraded":"2024-03-14T12:14:47.778261+0000","last_fullsized":"2024-03-14T12:14:47.778261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776920+0000","last_change":"2024-03-14T12:14:47.776920+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776920+0000","last_undegraded":"2024-03-14T12:14:47.776920+0000","last_fullsized":"2024-03-14T12:14:47.776920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777049+0000","last_change":"2024-03-14T12:14:47.777049+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777049+0000","last_undegraded":"2024-03-14T12:14:47.777049+0000","last_fullsized":"2024-03-14T12:14:47.777049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777110+0000","last_change":"2024-03-14T12:14:47.777110+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777110+0000","last_undegraded":"2024-03-14T12:14:47.777110+0000","last_fullsized":"2024-03-14T12:14:47.777110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777282+0000","last_change":"2024-03-14T12:14:47.777282+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777282+0000","last_undegraded":"2024-03-14T12:14:47.777282+0000","last_fullsized":"2024-03-14T12:14:47.777282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774990+0000","last_change":"2024-03-14T12:14:47.774990+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774990+0000","last_undegraded":"2024-03-14T12:14:47.774990+0000","last_fullsized":"2024-03-14T12:14:47.774990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779678+0000","last_change":"2024-03-14T12:14:47.779678+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779678+0000","last_undegraded":"2024-03-14T12:14:47.779678+0000","last_fullsized":"2024-03-14T12:14:47.779678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778547+0000","last_change":"2024-03-14T12:14:47.778547+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778547+0000","last_undegraded":"2024-03-14T12:14:47.778547+0000","last_fullsized":"2024-03-14T12:14:47.778547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776413+0000","last_change":"2024-03-14T12:14:47.776413+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776413+0000","last_undegraded":"2024-03-14T12:14:47.776413+0000","last_fullsized":"2024-03-14T12:14:47.776413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774743+0000","last_change":"2024-03-14T12:14:47.774743+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774743+0000","last_undegraded":"2024-03-14T12:14:47.774743+0000","last_fullsized":"2024-03-14T12:14:47.774743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776396+0000","last_change":"2024-03-14T12:14:47.776396+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776396+0000","last_undegraded":"2024-03-14T12:14:47.776396+0000","last_fullsized":"2024-03-14T12:14:47.776396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774511+0000","last_change":"2024-03-14T12:14:47.774511+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774511+0000","last_undegraded":"2024-03-14T12:14:47.774511+0000","last_fullsized":"2024-03-14T12:14:47.774511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776371+0000","last_change":"2024-03-14T12:14:47.776371+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776371+0000","last_undegraded":"2024-03-14T12:14:47.776371+0000","last_fullsized":"2024-03-14T12:14:47.776371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776952+0000","last_change":"2024-03-14T12:14:47.776952+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776952+0000","last_undegraded":"2024-03-14T12:14:47.776952+0000","last_fullsized":"2024-03-14T12:14:47.776952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777845+0000","last_change":"2024-03-14T12:14:47.777845+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777845+0000","last_undegraded":"2024-03-14T12:14:47.777845+0000","last_fullsized":"2024-03-14T12:14:47.777845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779722+0000","last_change":"2024-03-14T12:14:47.779722+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779722+0000","last_undegraded":"2024-03-14T12:14:47.779722+0000","last_fullsized":"2024-03-14T12:14:47.779722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T12:14:47.781149+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:14:47.781149+0000","last_peered":"2024-03-14T12:14:47.781149+0000","last_clean":"2024-03-14T12:14:47.781149+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:14:47.781149+0000","last_undegraded":"2024-03-14T12:14:47.781149+0000","last_fullsized":"2024-03-14T12:14:47.781149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:14:53.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-03-14T12:14:53.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-03-14T12:14:53.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-03-14T12:14:53.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-03-14T12:14:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.779857+0000', 'last_change': '2024-03-14T12:14:47.779857+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.779857+0000', 'last_undegraded': '2024-03-14T12:14:47.779857+0000', 'last_fullsized': '2024-03-14T12:14:47.779857+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.775646+0000', 'last_change': '2024-03-14T12:14:47.775646+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.775646+0000', 'last_undegraded': '2024-03-14T12:14:47.775646+0000', 'last_fullsized': '2024-03-14T12:14:47.775646+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2024-03-14T12:14:53.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776925+0000', 'last_change': '2024-03-14T12:14:47.776925+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776925+0000', 'last_undegraded': '2024-03-14T12:14:47.776925+0000', 'last_fullsized': '2024-03-14T12:14:47.776925+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.779301+0000', 'last_change': '2024-03-14T12:14:47.779301+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.779301+0000', 'last_undegraded': '2024-03-14T12:14:47.779301+0000', 'last_fullsized': '2024-03-14T12:14:47.779301+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.778241+0000', 'last_change': '2024-03-14T12:14:47.778241+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.778241+0000', 'last_undegraded': '2024-03-14T12:14:47.778241+0000', 'last_fullsized': '2024-03-14T12:14:47.778241+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777372+0000', 'last_change': '2024-03-14T12:14:47.777372+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777372+0000', 'last_undegraded': '2024-03-14T12:14:47.777372+0000', 'last_fullsized': '2024-03-14T12:14:47.777372+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.778295+0000', 'last_change': '2024-03-14T12:14:47.778295+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.778295+0000', 'last_undegraded': '2024-03-14T12:14:47.778295+0000', 'last_fullsized': '2024-03-14T12:14:47.778295+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777214+0000', 'last_change': '2024-03-14T12:14:47.777214+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777214+0000', 'last_undegraded': '2024-03-14T12:14:47.777214+0000', 'last_fullsized': '2024-03-14T12:14:47.777214+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-14T12:14:53.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776942+0000', 'last_change': '2024-03-14T12:14:47.776942+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776942+0000', 'last_undegraded': '2024-03-14T12:14:47.776942+0000', 'last_fullsized': '2024-03-14T12:14:47.776942+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.775209+0000', 'last_change': '2024-03-14T12:14:47.775209+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.775209+0000', 'last_undegraded': '2024-03-14T12:14:47.775209+0000', 'last_fullsized': '2024-03-14T12:14:47.775209+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.774286+0000', 'last_change': '2024-03-14T12:14:47.774286+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.774286+0000', 'last_undegraded': '2024-03-14T12:14:47.774286+0000', 'last_fullsized': '2024-03-14T12:14:47.774286+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-14T12:14:53.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777138+0000', 'last_change': '2024-03-14T12:14:47.777138+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777138+0000', 'last_undegraded': '2024-03-14T12:14:47.777138+0000', 'last_fullsized': '2024-03-14T12:14:47.777138+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777704+0000', 'last_change': '2024-03-14T12:14:47.777704+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777704+0000', 'last_undegraded': '2024-03-14T12:14:47.777704+0000', 'last_fullsized': '2024-03-14T12:14:47.777704+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776875+0000', 'last_change': '2024-03-14T12:14:47.776875+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776875+0000', 'last_undegraded': '2024-03-14T12:14:47.776875+0000', 'last_fullsized': '2024-03-14T12:14:47.776875+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T12:14:53.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777451+0000', 'last_change': '2024-03-14T12:14:47.777451+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777451+0000', 'last_undegraded': '2024-03-14T12:14:47.777451+0000', 'last_fullsized': '2024-03-14T12:14:47.777451+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.778261+0000', 'last_change': '2024-03-14T12:14:47.778261+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.778261+0000', 'last_undegraded': '2024-03-14T12:14:47.778261+0000', 'last_fullsized': '2024-03-14T12:14:47.778261+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2024-03-14T12:14:53.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776920+0000', 'last_change': '2024-03-14T12:14:47.776920+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776920+0000', 'last_undegraded': '2024-03-14T12:14:47.776920+0000', 'last_fullsized': '2024-03-14T12:14:47.776920+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777049+0000', 'last_change': '2024-03-14T12:14:47.777049+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777049+0000', 'last_undegraded': '2024-03-14T12:14:47.777049+0000', 'last_fullsized': '2024-03-14T12:14:47.777049+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777110+0000', 'last_change': '2024-03-14T12:14:47.777110+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777110+0000', 'last_undegraded': '2024-03-14T12:14:47.777110+0000', 'last_fullsized': '2024-03-14T12:14:47.777110+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777282+0000', 'last_change': '2024-03-14T12:14:47.777282+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777282+0000', 'last_undegraded': '2024-03-14T12:14:47.777282+0000', 'last_fullsized': '2024-03-14T12:14:47.777282+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.774990+0000', 'last_change': '2024-03-14T12:14:47.774990+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.774990+0000', 'last_undegraded': '2024-03-14T12:14:47.774990+0000', 'last_fullsized': '2024-03-14T12:14:47.774990+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.779678+0000', 'last_change': '2024-03-14T12:14:47.779678+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.779678+0000', 'last_undegraded': '2024-03-14T12:14:47.779678+0000', 'last_fullsized': '2024-03-14T12:14:47.779678+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2024-03-14T12:14:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.778547+0000', 'last_change': '2024-03-14T12:14:47.778547+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.778547+0000', 'last_undegraded': '2024-03-14T12:14:47.778547+0000', 'last_fullsized': '2024-03-14T12:14:47.778547+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.774286+0000', 'last_change': '2024-03-14T12:14:47.774286+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.774286+0000', 'last_undegraded': '2024-03-14T12:14:47.774286+0000', 'last_fullsized': '2024-03-14T12:14:47.774286+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776413+0000', 'last_change': '2024-03-14T12:14:47.776413+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776413+0000', 'last_undegraded': '2024-03-14T12:14:47.776413+0000', 'last_fullsized': '2024-03-14T12:14:47.776413+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-03-14T12:14:53.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.774743+0000', 'last_change': '2024-03-14T12:14:47.774743+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.774743+0000', 'last_undegraded': '2024-03-14T12:14:47.774743+0000', 'last_fullsized': '2024-03-14T12:14:47.774743+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776396+0000', 'last_change': '2024-03-14T12:14:47.776396+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776396+0000', 'last_undegraded': '2024-03-14T12:14:47.776396+0000', 'last_fullsized': '2024-03-14T12:14:47.776396+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.774511+0000', 'last_change': '2024-03-14T12:14:47.774511+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.774511+0000', 'last_undegraded': '2024-03-14T12:14:47.774511+0000', 'last_fullsized': '2024-03-14T12:14:47.774511+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2024-03-14T12:14:53.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776371+0000', 'last_change': '2024-03-14T12:14:47.776371+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776371+0000', 'last_undegraded': '2024-03-14T12:14:47.776371+0000', 'last_fullsized': '2024-03-14T12:14:47.776371+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.776952+0000', 'last_change': '2024-03-14T12:14:47.776952+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.776952+0000', 'last_undegraded': '2024-03-14T12:14:47.776952+0000', 'last_fullsized': '2024-03-14T12:14:47.776952+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.777845+0000', 'last_change': '2024-03-14T12:14:47.777845+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.777845+0000', 'last_undegraded': '2024-03-14T12:14:47.777845+0000', 'last_fullsized': '2024-03-14T12:14:47.777845+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T12:14:48.770586+0000', 'last_change': '2024-03-14T12:14:48.770586+0000', 'last_active': '2024-03-14T12:14:48.770586+0000', 'last_peered': '2024-03-14T12:14:48.770586+0000', 'last_clean': '2024-03-14T12:14:48.770586+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:48.770586+0000', 'last_undegraded': '2024-03-14T12:14:48.770586+0000', 'last_fullsized': '2024-03-14T12:14:48.770586+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:48.770586+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2024-03-14T12:14:53.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-14T12:14:47.779722+0000', 'last_change': '2024-03-14T12:14:47.779722+0000', 'last_active': '2024-03-14T12:14:47.766209+0000', 'last_peered': '2024-03-14T12:14:47.766209+0000', 'last_clean': '2024-03-14T12:14:47.766209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T12:14:47.779722+0000', 'last_undegraded': '2024-03-14T12:14:47.779722+0000', 'last_fullsized': '2024-03-14T12:14:47.779722+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'last_clean_scrub_stamp': '2024-03-14T12:14:47.766209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T12:14:53.420 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-14T12:14:53.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.539+0000 7f1f14bb3640 1 Processor -- start 2024-03-14T12:14:53.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.540+0000 7f1f14bb3640 1 -- start start 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.540+0000 7f1f14bb3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1f100b2950 0x7f1f1006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.540+0000 7f1f14bb3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f1006cb30 0x7f1f100ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.540+0000 7f1f14bb3640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f100b22c0 con 0x7f1f1015dfa0 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.540+0000 7f1f14bb3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1006cf60 con 0x7f1f1006cb30 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.540+0000 7f1f14bb3640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f100cc100 con 0x7f1f100b2950 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f1006cb30 0x7f1f100ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f1006cb30 0x7f1f100ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58792/0 (socket says 172.21.15.82:58792) 2024-03-14T12:14:53.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/3479154137 learned_addr learned my addr 172.21.15.82:0/3479154137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:53.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0dd9b640 1 --2- 172.21.15.82:0/3479154137 >> v2:172.21.15.164:3300/0 conn(0x7f1f100b2950 0x7f1f1006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/3479154137 >> v2:172.21.15.164:3300/0 conn(0x7f1f100b2950 msgr2=0x7f1f1006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:53.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0ed9d640 1 --2- 172.21.15.82:0/3479154137 >> v2:172.21.15.164:3300/0 conn(0x7f1f100b2950 0x7f1f1006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/3479154137 >> v1:172.21.15.164:6789/0 conn(0x7f1f1015dfa0 legacy=0x7f1f100b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:53.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.541+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/3479154137 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100cc3e0 con 0x7f1f1006cb30 2024-03-14T12:14:53.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f0ed9d640 1 --2- 172.21.15.82:0/3479154137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1006cb30 0x7f1f100ccec0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1f10161b60 tx=0x7f1f0002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a6a829f84ee06 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f0d59a640 1 -- 172.21.15.82:0/3479154137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0002e070 con 0x7f1f1006cb30 2024-03-14T12:14:53.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f0d59a640 1 -- 172.21.15.82:0/3479154137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00037070 con 0x7f1f1006cb30 2024-03-14T12:14:53.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f0d59a640 1 -- 172.21.15.82:0/3479154137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00031a00 con 0x7f1f1006cb30 2024-03-14T12:14:53.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 -- 172.21.15.82:0/3479154137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1006cb30 msgr2=0x7f1f100ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/3479154137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1006cb30 0x7f1f100ccec0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1f10161b60 tx=0x7f1f0002b880 comp rx=0 tx=0).stop 2024-03-14T12:14:53.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 -- 172.21.15.82:0/3479154137 shutdown_connections 2024-03-14T12:14:53.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/3479154137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1006cb30 0x7f1f100ccec0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/3479154137 >> v2:172.21.15.164:3300/0 conn(0x7f1f100b2950 0x7f1f1006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 -- 172.21.15.82:0/3479154137 >> 172.21.15.82:0/3479154137 conn(0x7f1f1009a9e0 msgr2=0x7f1f1009adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:53.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 -- 172.21.15.82:0/3479154137 shutdown_connections 2024-03-14T12:14:53.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.542+0000 7f1f14bb3640 1 -- 172.21.15.82:0/3479154137 wait complete. 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 Processor -- start 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 -- start start 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f1006cb30 0x7f1f1014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f100b2950 0x7f1f100b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1015dfa0 0x7f1f100cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100ae7e0 con 0x7f1f1015dfa0 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100cc9a0 con 0x7f1f1006cb30 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.544+0000 7f1f14bb3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1015e380 con 0x7f1f100b2950 2024-03-14T12:14:53.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1015dfa0 0x7f1f100cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1015dfa0 0x7f1f100cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58796/0 (socket says 172.21.15.82:58796) 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/1356450498 learned_addr learned my addr 172.21.15.82:0/1356450498 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0dd9b640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f100b2950 0x7f1f100b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0e59c640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f1006cb30 0x7f1f1014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f100b2950 msgr2=0x7f1f100b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f100b2950 0x7f1f100b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f1006cb30 msgr2=0x7f1f1014e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f1006cb30 0x7f1f1014e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 -- 172.21.15.82:0/1356450498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100ca2b0 con 0x7f1f1015dfa0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f0ed9d640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1015dfa0 0x7f1f100cb350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1f0002b8c0 tx=0x7f1f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1efeffd640 1 -- 172.21.15.82:0/1356450498 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00032880 con 0x7f1f1015dfa0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1efeffd640 1 -- 172.21.15.82:0/1356450498 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00032a40 con 0x7f1f1015dfa0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1efeffd640 1 -- 172.21.15.82:0/1356450498 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00032e30 con 0x7f1f1015dfa0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f100ca590 con 0x7f1f1015dfa0 2024-03-14T12:14:53.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.545+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f100cab50 con 0x7f1f1015dfa0 2024-03-14T12:14:53.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.546+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1004f050 con 0x7f1f1015dfa0 2024-03-14T12:14:53.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.547+0000 7f1efeffd640 1 -- 172.21.15.82:0/1356450498 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1f000453f0 con 0x7f1f1015dfa0 2024-03-14T12:14:53.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.547+0000 7f1efeffd640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ef006bbd0 0x7f1ef006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.547+0000 7f1f0e59c640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ef006bbd0 0x7f1ef006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.547+0000 7f1efeffd640 1 -- 172.21.15.82:0/1356450498 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f000acb00 con 0x7f1f1015dfa0 2024-03-14T12:14:53.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.548+0000 7f1f0e59c640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ef006bbd0 0x7f1ef006e090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f08006fd0 tx=0x7f1f08008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.552+0000 7f1efeffd640 1 -- 172.21.15.82:0/1356450498 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1f00040070 con 0x7f1f1015dfa0 2024-03-14T12:14:53.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.789+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1f100d1e10 con 0x7f1f1015dfa0 2024-03-14T12:14:53.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.790+0000 7f1efeffd640 1 -- 172.21.15.82:0/1356450498 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1656 (secure 0 0 0) 0x7f1f0007b5f0 con 0x7f1f1015dfa0 2024-03-14T12:14:53.790 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:53.790 INFO:teuthology.orchestra.run.smithi082.stdout:{"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1710418418,"num_in_osds":8,"osd_in_since":1710418409,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225284096,"bytes_avail":772868829184,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5061}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-03-14T12:14:39.507160+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-14T12:14:53.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ef006bbd0 msgr2=0x7f1ef006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ef006bbd0 0x7f1ef006e090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f08006fd0 tx=0x7f1f08008040 comp rx=0 tx=0).stop 2024-03-14T12:14:53.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1015dfa0 msgr2=0x7f1f100cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1015dfa0 0x7f1f100cb350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1f0002b8c0 tx=0x7f1f000046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 shutdown_connections 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f1015dfa0 0x7f1f100cb350 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1ef006bbd0 0x7f1ef006e090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f100b2950 0x7f1f100b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 --2- 172.21.15.82:0/1356450498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f1006cb30 0x7f1f1014e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 >> 172.21.15.82:0/1356450498 conn(0x7f1f1009a9e0 msgr2=0x7f1f100d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 shutdown_connections 2024-03-14T12:14:53.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.792+0000 7f1f14bb3640 1 -- 172.21.15.82:0/1356450498 wait complete. 2024-03-14T12:14:53.803 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-14T12:14:53.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:14:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.903+0000 7f5f28b2b640 1 Processor -- start 2024-03-14T12:14:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.903+0000 7f5f28b2b640 1 -- start start 2024-03-14T12:14:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f28b2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 0x7f5f240d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f28b2b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f24066b00 0x7f5f2406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f28b2b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f240ccb40 con 0x7f5f2415c220 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f28b2b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f240cd750 con 0x7f5f24066b00 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f28b2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240cce20 con 0x7f5f2415be40 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f21d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f24066b00 0x7f5f2406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f22d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5f2415c220 0x7f5f240cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52626/0 (socket says 172.21.15.82:52626) 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f22d9d640 1 -- 172.21.15.82:0/2242135432 learned_addr learned my addr 172.21.15.82:0/2242135432 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f21d9b640 1 -- 172.21.15.82:0/2242135432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 msgr2=0x7f5f240d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f21d9b640 1 --2- 172.21.15.82:0/2242135432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 0x7f5f240d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f21d9b640 1 -- 172.21.15.82:0/2242135432 >> v1:172.21.15.82:6789/0 conn(0x7f5f2415c220 legacy=0x7f5f240cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.904+0000 7f5f21d9b640 1 -- 172.21.15.82:0/2242135432 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240cd100 con 0x7f5f24066b00 2024-03-14T12:14:53.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.905+0000 7f5f21d9b640 1 --2- 172.21.15.82:0/2242135432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f24066b00 0x7f5f2406ad10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f180097a0 tx=0x7f5f1802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a3a89ab69664a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.905+0000 7f5f2159a640 1 -- 172.21.15.82:0/2242135432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18037070 con 0x7f5f24066b00 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.905+0000 7f5f2159a640 1 -- 172.21.15.82:0/2242135432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18030710 con 0x7f5f24066b00 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.905+0000 7f5f2159a640 1 -- 172.21.15.82:0/2242135432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18030ae0 con 0x7f5f24066b00 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2242135432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f24066b00 msgr2=0x7f5f2406ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2242135432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f24066b00 0x7f5f2406ad10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f180097a0 tx=0x7f5f1802b710 comp rx=0 tx=0).stop 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2242135432 shutdown_connections 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2242135432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f24066b00 0x7f5f2406ad10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2242135432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 0x7f5f240d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2242135432 >> 172.21.15.82:0/2242135432 conn(0x7f5f24072310 msgr2=0x7f5f24072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2242135432 shutdown_connections 2024-03-14T12:14:53.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.906+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2242135432 wait complete. 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.907+0000 7f5f28b2b640 1 Processor -- start 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.907+0000 7f5f28b2b640 1 -- start start 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f28b2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f24066b00 0x7f5f241e6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f28b2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 0x7f5f240cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f28b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f2415c220 0x7f5f240c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f28b2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2406b250 con 0x7f5f2415c220 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f28b2b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240d0f70 con 0x7f5f24066b00 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f28b2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2406b3d0 con 0x7f5f2415be40 2024-03-14T12:14:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f2415c220 0x7f5f240c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f2415c220 0x7f5f240c5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58810/0 (socket says 172.21.15.82:58810) 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 -- 172.21.15.82:0/2554649712 learned_addr learned my addr 172.21.15.82:0/2554649712 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f21d9b640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 0x7f5f240cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f2259c640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f24066b00 0x7f5f241e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 -- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 msgr2=0x7f5f240cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 0x7f5f240cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 -- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f24066b00 msgr2=0x7f5f241e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f24066b00 0x7f5f241e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 -- 172.21.15.82:0/2554649712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f241e6850 con 0x7f5f2415c220 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.908+0000 7f5f22d9d640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f2415c220 0x7f5f240c5970 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00c8c0 tx=0x7f5f1c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.909+0000 7f5f12ffd640 1 -- 172.21.15.82:0/2554649712 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c018070 con 0x7f5f2415c220 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.909+0000 7f5f12ffd640 1 -- 172.21.15.82:0/2554649712 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00a640 con 0x7f5f2415c220 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.909+0000 7f5f12ffd640 1 -- 172.21.15.82:0/2554649712 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00aa30 con 0x7f5f2415c220 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.909+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f241e7070 con 0x7f5f2415c220 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.909+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f241e7630 con 0x7f5f2415c220 2024-03-14T12:14:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.910+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2404f050 con 0x7f5f2415c220 2024-03-14T12:14:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.910+0000 7f5f12ffd640 1 -- 172.21.15.82:0/2554649712 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5f1c01b070 con 0x7f5f2415c220 2024-03-14T12:14:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.910+0000 7f5f12ffd640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5f0406bbd0 0x7f5f0406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.910+0000 7f5f12ffd640 1 -- 172.21.15.82:0/2554649712 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f1c089f90 con 0x7f5f2415c220 2024-03-14T12:14:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.910+0000 7f5f2259c640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5f0406bbd0 0x7f5f0406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.911+0000 7f5f2259c640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5f0406bbd0 0x7f5f0406e090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f241590e0 tx=0x7f5f14009110 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:53.915+0000 7f5f12ffd640 1 -- 172.21.15.82:0/2554649712 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5f1c0584c0 con 0x7f5f2415c220 2024-03-14T12:14:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.083+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f241e6b30 con 0x7f5f0406bbd0 2024-03-14T12:14:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.088+0000 7f5f12ffd640 1 -- 172.21.15.82:0/2554649712 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239830 (secure 0 0 0) 0x7f5f241e6b30 con 0x7f5f0406bbd0 2024-03-14T12:14:54.091 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:54.092 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5f0406bbd0 msgr2=0x7f5f0406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5f0406bbd0 0x7f5f0406e090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f241590e0 tx=0x7f5f14009110 comp rx=0 tx=0).stop 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f2415c220 msgr2=0x7f5f240c5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f2415c220 0x7f5f240c5970 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00c8c0 tx=0x7f5f1c00ccc0 comp rx=0 tx=0).stop 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 shutdown_connections 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f2415c220 0x7f5f240c5970 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5f0406bbd0 0x7f5f0406e090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f2415be40 0x7f5f240cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 --2- 172.21.15.82:0/2554649712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f24066b00 0x7f5f241e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:54.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 >> 172.21.15.82:0/2554649712 conn(0x7f5f24072310 msgr2=0x7f5f240b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:54.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 shutdown_connections 2024-03-14T12:14:54.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:54.091+0000 7f5f28b2b640 1 -- 172.21.15.82:0/2554649712 wait complete. 2024-03-14T12:14:54.103 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-03-14T12:14:53.509204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220004,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754716,"statfs":{"total":773094113280,"available":772868829184,"internally_reserved":0,"allocated":3936256,"data_stored":2214229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737899"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779857+0000","last_change":"2024-03-14T12:14:47.779857+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779857+0000","last_undegraded":"2024-03-14T12:14:47.779857+0000","last_fullsized":"2024-03-14T12:14:47.779857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775646+0000","last_change":"2024-03-14T12:14:47.775646+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775646+0000","last_undegraded":"2024-03-14T12:14:47.775646+0000","last_fullsized":"2024-03-14T12:14:47.775646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776925+0000","last_change":"2024-03-14T12:14:47.776925+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776925+0000","last_undegraded":"2024-03-14T12:14:47.776925+0000","last_fullsized":"2024-03-14T12:14:47.776925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779301+0000","last_change":"2024-03-14T12:14:47.779301+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779301+0000","last_undegraded":"2024-03-14T12:14:47.779301+0000","last_fullsized":"2024-03-14T12:14:47.779301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778241+0000","last_change":"2024-03-14T12:14:47.778241+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778241+0000","last_undegraded":"2024-03-14T12:14:47.778241+0000","last_fullsized":"2024-03-14T12:14:47.778241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777372+0000","last_change":"2024-03-14T12:14:47.777372+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777372+0000","last_undegraded":"2024-03-14T12:14:47.777372+0000","last_fullsized":"2024-03-14T12:14:47.777372+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778295+0000","last_change":"2024-03-14T12:14:47.778295+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778295+0000","last_undegraded":"2024-03-14T12:14:47.778295+0000","last_fullsized":"2024-03-14T12:14:47.778295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777214+0000","last_change":"2024-03-14T12:14:47.777214+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777214+0000","last_undegraded":"2024-03-14T12:14:47.777214+0000","last_fullsized":"2024-03-14T12:14:47.777214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776942+0000","last_change":"2024-03-14T12:14:47.776942+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776942+0000","last_undegraded":"2024-03-14T12:14:47.776942+0000","last_fullsized":"2024-03-14T12:14:47.776942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.775209+0000","last_change":"2024-03-14T12:14:47.775209+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.775209+0000","last_undegraded":"2024-03-14T12:14:47.775209+0000","last_fullsized":"2024-03-14T12:14:47.775209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777138+0000","last_change":"2024-03-14T12:14:47.777138+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777138+0000","last_undegraded":"2024-03-14T12:14:47.777138+0000","last_fullsized":"2024-03-14T12:14:47.777138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777704+0000","last_change":"2024-03-14T12:14:47.777704+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777704+0000","last_undegraded":"2024-03-14T12:14:47.777704+0000","last_fullsized":"2024-03-14T12:14:47.777704+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776875+0000","last_change":"2024-03-14T12:14:47.776875+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776875+0000","last_undegraded":"2024-03-14T12:14:47.776875+0000","last_fullsized":"2024-03-14T12:14:47.776875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777451+0000","last_change":"2024-03-14T12:14:47.777451+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777451+0000","last_undegraded":"2024-03-14T12:14:47.777451+0000","last_fullsized":"2024-03-14T12:14:47.777451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778261+0000","last_change":"2024-03-14T12:14:47.778261+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778261+0000","last_undegraded":"2024-03-14T12:14:47.778261+0000","last_fullsized":"2024-03-14T12:14:47.778261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776920+0000","last_change":"2024-03-14T12:14:47.776920+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776920+0000","last_undegraded":"2024-03-14T12:14:47.776920+0000","last_fullsized":"2024-03-14T12:14:47.776920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777049+0000","last_change":"2024-03-14T12:14:47.777049+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777049+0000","last_undegraded":"2024-03-14T12:14:47.777049+0000","last_fullsized":"2024-03-14T12:14:47.777049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777110+0000","last_change":"2024-03-14T12:14:47.777110+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777110+0000","last_undegraded":"2024-03-14T12:14:47.777110+0000","last_fullsized":"2024-03-14T12:14:47.777110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777282+0000","last_change":"2024-03-14T12:14:47.777282+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777282+0000","last_undegraded":"2024-03-14T12:14:47.777282+0000","last_fullsized":"2024-03-14T12:14:47.777282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774990+0000","last_change":"2024-03-14T12:14:47.774990+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774990+0000","last_undegraded":"2024-03-14T12:14:47.774990+0000","last_fullsized":"2024-03-14T12:14:47.774990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779678+0000","last_change":"2024-03-14T12:14:47.779678+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779678+0000","last_undegraded":"2024-03-14T12:14:47.779678+0000","last_fullsized":"2024-03-14T12:14:47.779678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.778547+0000","last_change":"2024-03-14T12:14:47.778547+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.778547+0000","last_undegraded":"2024-03-14T12:14:47.778547+0000","last_fullsized":"2024-03-14T12:14:47.778547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774286+0000","last_change":"2024-03-14T12:14:47.774286+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774286+0000","last_undegraded":"2024-03-14T12:14:47.774286+0000","last_fullsized":"2024-03-14T12:14:47.774286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776413+0000","last_change":"2024-03-14T12:14:47.776413+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776413+0000","last_undegraded":"2024-03-14T12:14:47.776413+0000","last_fullsized":"2024-03-14T12:14:47.776413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774743+0000","last_change":"2024-03-14T12:14:47.774743+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774743+0000","last_undegraded":"2024-03-14T12:14:47.774743+0000","last_fullsized":"2024-03-14T12:14:47.774743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776396+0000","last_change":"2024-03-14T12:14:47.776396+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776396+0000","last_undegraded":"2024-03-14T12:14:47.776396+0000","last_fullsized":"2024-03-14T12:14:47.776396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.774511+0000","last_change":"2024-03-14T12:14:47.774511+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.774511+0000","last_undegraded":"2024-03-14T12:14:47.774511+0000","last_fullsized":"2024-03-14T12:14:47.774511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776371+0000","last_change":"2024-03-14T12:14:47.776371+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776371+0000","last_undegraded":"2024-03-14T12:14:47.776371+0000","last_fullsized":"2024-03-14T12:14:47.776371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.776952+0000","last_change":"2024-03-14T12:14:47.776952+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.776952+0000","last_undegraded":"2024-03-14T12:14:47.776952+0000","last_fullsized":"2024-03-14T12:14:47.776952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.777845+0000","last_change":"2024-03-14T12:14:47.777845+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.777845+0000","last_undegraded":"2024-03-14T12:14:47.777845+0000","last_fullsized":"2024-03-14T12:14:47.777845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:14:48.770586+0000","last_change":"2024-03-14T12:14:48.770586+0000","last_active":"2024-03-14T12:14:48.770586+0000","last_peered":"2024-03-14T12:14:48.770586+0000","last_clean":"2024-03-14T12:14:48.770586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:48.770586+0000","last_undegraded":"2024-03-14T12:14:48.770586+0000","last_fullsized":"2024-03-14T12:14:48.770586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-14T12:14:47.779722+0000","last_change":"2024-03-14T12:14:47.779722+0000","last_active":"2024-03-14T12:14:47.766209+0000","last_peered":"2024-03-14T12:14:47.766209+0000","last_clean":"2024-03-14T12:14:47.766209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:14:47.779722+0000","last_undegraded":"2024-03-14T12:14:47.779722+0000","last_fullsized":"2024-03-14T12:14:47.779722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T12:14:47.781149+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:14:47.781149+0000","last_peered":"2024-03-14T12:14:47.781149+0000","last_clean":"2024-03-14T12:14:47.781149+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:14:47.781149+0000","last_undegraded":"2024-03-14T12:14:47.781149+0000","last_fullsized":"2024-03-14T12:14:47.781149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":125551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:14:57.107 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:14:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.219+0000 7fc0c9e9a640 1 Processor -- start 2024-03-14T12:14:57.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.219+0000 7fc0c9e9a640 1 -- start start 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.220+0000 7fc0c9e9a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc0c406b420 0x7fc0c40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.220+0000 7fc0c9e9a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c40b1340 con 0x7fc0c40afff0 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.220+0000 7fc0c9e9a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c40b0480 con 0x7fc0c40b0f60 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.220+0000 7fc0c9e9a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c40b4ee0 con 0x7fc0c406b420 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.220+0000 7fc0c2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc0c406b420 0x7fc0c40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.220+0000 7fc0c3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc0c40afff0 0x7fc0c40b5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52634/0 (socket says 172.21.15.82:52634) 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.220+0000 7fc0c3fff640 1 -- 172.21.15.82:0/928259537 learned_addr learned my addr 172.21.15.82:0/928259537 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.221+0000 7fc0c2ffd640 1 -- 172.21.15.82:0/928259537 >> v1:172.21.15.164:6789/0 conn(0x7fc0c40b0f60 legacy=0x7fc0c406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:57.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.221+0000 7fc0c2ffd640 1 -- 172.21.15.82:0/928259537 >> v1:172.21.15.82:6789/0 conn(0x7fc0c40afff0 legacy=0x7fc0c40b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.221+0000 7fc0c27fc640 1 -- 172.21.15.82:0/928259537 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034631664 0 0) 0x7fc0c40b1340 con 0x7fc0c40afff0 2024-03-14T12:14:57.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.221+0000 7fc0c2ffd640 1 -- 172.21.15.82:0/928259537 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40cd030 con 0x7fc0c406b420 2024-03-14T12:14:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.221+0000 7fc0c2ffd640 1 --2- 172.21.15.82:0/928259537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c406b420 0x7fc0c40cc9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3e986c30190d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.221+0000 7fc0c27fc640 1 -- 172.21.15.82:0/928259537 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8035020 con 0x7fc0c406b420 2024-03-14T12:14:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c27fc640 1 -- 172.21.15.82:0/928259537 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b8031830 con 0x7fc0c406b420 2024-03-14T12:14:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c27fc640 1 -- 172.21.15.82:0/928259537 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8031c00 con 0x7fc0c406b420 2024-03-14T12:14:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/928259537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c406b420 msgr2=0x7fc0c40cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/928259537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c406b420 0x7fc0c40cc9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/928259537 shutdown_connections 2024-03-14T12:14:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/928259537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c406b420 0x7fc0c40cc9a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/928259537 >> 172.21.15.82:0/928259537 conn(0x7fc0c4072310 msgr2=0x7fc0c4072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/928259537 shutdown_connections 2024-03-14T12:14:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.222+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/928259537 wait complete. 2024-03-14T12:14:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.223+0000 7fc0c9e9a640 1 Processor -- start 2024-03-14T12:14:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.223+0000 7fc0c9e9a640 1 -- start start 2024-03-14T12:14:57.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c9e9a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0c406b420 0x7fc0c40999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c9e9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c40afff0 0x7fc0c40c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c9e9a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0c40b0f60 0x7fc0c40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c9e9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40cd8d0 con 0x7fc0c40afff0 2024-03-14T12:14:57.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c9e9a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40cd5f0 con 0x7fc0c406b420 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c9e9a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40d09a0 con 0x7fc0c40b0f60 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c40afff0 0x7fc0c40c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c40afff0 0x7fc0c40c3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58826/0 (socket says 172.21.15.82:58826) 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.224+0000 7fc0c2ffd640 1 -- 172.21.15.82:0/2199638997 learned_addr learned my addr 172.21.15.82:0/2199638997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c2ffd640 1 -- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0c40b0f60 msgr2=0x7fc0c40c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c3fff640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0c40b0f60 0x7fc0c40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c37fe640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0c406b420 0x7fc0c40999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c2ffd640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0c40b0f60 0x7fc0c40c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c2ffd640 1 -- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0c406b420 msgr2=0x7fc0c40999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c2ffd640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0c406b420 0x7fc0c40999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c2ffd640 1 -- 172.21.15.82:0/2199638997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40cfa80 con 0x7fc0c40afff0 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c3fff640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0c40b0f60 0x7fc0c40c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c37fe640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0c406b420 0x7fc0c40999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:57.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c2ffd640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c40afff0 0x7fc0c40c3c50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80098c0 tx=0x7fc0b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0abfff640 1 -- 172.21.15.82:0/2199638997 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8004230 con 0x7fc0c40afff0 2024-03-14T12:14:57.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0abfff640 1 -- 172.21.15.82:0/2199638997 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b80043d0 con 0x7fc0c40afff0 2024-03-14T12:14:57.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c40cfd60 con 0x7fc0c40afff0 2024-03-14T12:14:57.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0abfff640 1 -- 172.21.15.82:0/2199638997 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b803ca50 con 0x7fc0c40afff0 2024-03-14T12:14:57.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.225+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c40d0320 con 0x7fc0c40afff0 2024-03-14T12:14:57.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.226+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c404f050 con 0x7fc0c40afff0 2024-03-14T12:14:57.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.226+0000 7fc0abfff640 1 -- 172.21.15.82:0/2199638997 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc0b8038020 con 0x7fc0c40afff0 2024-03-14T12:14:57.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.227+0000 7fc0abfff640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc0b006bbd0 0x7fc0b006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.227+0000 7fc0abfff640 1 -- 172.21.15.82:0/2199638997 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0b80acce0 con 0x7fc0c40afff0 2024-03-14T12:14:57.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.227+0000 7fc0c37fe640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc0b006bbd0 0x7fc0b006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.227+0000 7fc0c37fe640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc0b006bbd0 0x7fc0b006e090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400a830 tx=0x7fc0b4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.231+0000 7fc0abfff640 1 -- 172.21.15.82:0/2199638997 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc0b8041070 con 0x7fc0c40afff0 2024-03-14T12:14:57.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.400+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0c40d0690 con 0x7fc0b006bbd0 2024-03-14T12:14:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.406+0000 7fc0abfff640 1 -- 172.21.15.82:0/2199638997 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7fc0c40d0690 con 0x7fc0b006bbd0 2024-03-14T12:14:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:14:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc0b006bbd0 msgr2=0x7fc0b006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc0b006bbd0 0x7fc0b006e090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400a830 tx=0x7fc0b4008040 comp rx=0 tx=0).stop 2024-03-14T12:14:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c40afff0 msgr2=0x7fc0c40c3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c40afff0 0x7fc0c40c3c50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80098c0 tx=0x7fc0b80046c0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 shutdown_connections 2024-03-14T12:14:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc0b006bbd0 0x7fc0b006e090 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0c40b0f60 0x7fc0c40c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c40afff0 0x7fc0c40c3c50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 --2- 172.21.15.82:0/2199638997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0c406b420 0x7fc0c40999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 >> 172.21.15.82:0/2199638997 conn(0x7fc0c4072310 msgr2=0x7fc0c40b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 shutdown_connections 2024-03-14T12:14:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.409+0000 7fc0c9e9a640 1 -- 172.21.15.82:0/2199638997 wait complete. 2024-03-14T12:14:57.419 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-03-14T12:14:55.509618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220420,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754300,"statfs":{"total":773094113280,"available":772868403200,"internally_reserved":0,"allocated":4333568,"data_stored":2391236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.728378"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889873+0000","last_change":"2024-03-14T12:14:49.796207+0000","last_active":"2024-03-14T12:14:50.889873+0000","last_peered":"2024-03-14T12:14:50.889873+0000","last_clean":"2024-03-14T12:14:50.889873+0000","last_became_active":"2024-03-14T12:14:49.795954+0000","last_became_peered":"2024-03-14T12:14:49.795954+0000","last_unstale":"2024-03-14T12:14:50.889873+0000","last_undegraded":"2024-03-14T12:14:50.889873+0000","last_fullsized":"2024-03-14T12:14:50.889873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:58:34.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794811+0000","last_change":"2024-03-14T12:14:49.795464+0000","last_active":"2024-03-14T12:14:50.794811+0000","last_peered":"2024-03-14T12:14:50.794811+0000","last_clean":"2024-03-14T12:14:50.794811+0000","last_became_active":"2024-03-14T12:14:49.795088+0000","last_became_peered":"2024-03-14T12:14:49.795088+0000","last_unstale":"2024-03-14T12:14:50.794811+0000","last_undegraded":"2024-03-14T12:14:50.794811+0000","last_fullsized":"2024-03-14T12:14:50.794811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:38.128752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976749+0000","last_change":"2024-03-14T12:14:49.795668+0000","last_active":"2024-03-14T12:14:50.976749+0000","last_peered":"2024-03-14T12:14:50.976749+0000","last_clean":"2024-03-14T12:14:50.976749+0000","last_became_active":"2024-03-14T12:14:49.795028+0000","last_became_peered":"2024-03-14T12:14:49.795028+0000","last_unstale":"2024-03-14T12:14:50.976749+0000","last_undegraded":"2024-03-14T12:14:50.976749+0000","last_fullsized":"2024-03-14T12:14:50.976749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:41:06.594787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.835957+0000","last_change":"2024-03-14T12:14:49.797050+0000","last_active":"2024-03-14T12:14:50.835957+0000","last_peered":"2024-03-14T12:14:50.835957+0000","last_clean":"2024-03-14T12:14:50.835957+0000","last_became_active":"2024-03-14T12:14:49.796792+0000","last_became_peered":"2024-03-14T12:14:49.796792+0000","last_unstale":"2024-03-14T12:14:50.835957+0000","last_undegraded":"2024-03-14T12:14:50.835957+0000","last_fullsized":"2024-03-14T12:14:50.835957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:02:17.484737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976266+0000","last_change":"2024-03-14T12:14:49.806012+0000","last_active":"2024-03-14T12:14:50.976266+0000","last_peered":"2024-03-14T12:14:50.976266+0000","last_clean":"2024-03-14T12:14:50.976266+0000","last_became_active":"2024-03-14T12:14:49.805738+0000","last_became_peered":"2024-03-14T12:14:49.805738+0000","last_unstale":"2024-03-14T12:14:50.976266+0000","last_undegraded":"2024-03-14T12:14:50.976266+0000","last_fullsized":"2024-03-14T12:14:50.976266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:08:33.660552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794375+0000","last_change":"2024-03-14T12:14:49.800940+0000","last_active":"2024-03-14T12:14:50.794375+0000","last_peered":"2024-03-14T12:14:50.794375+0000","last_clean":"2024-03-14T12:14:50.794375+0000","last_became_active":"2024-03-14T12:14:49.800736+0000","last_became_peered":"2024-03-14T12:14:49.800736+0000","last_unstale":"2024-03-14T12:14:50.794375+0000","last_undegraded":"2024-03-14T12:14:50.794375+0000","last_fullsized":"2024-03-14T12:14:50.794375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:58:12.693037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796000+0000","last_change":"2024-03-14T12:14:49.802948+0000","last_active":"2024-03-14T12:14:50.796000+0000","last_peered":"2024-03-14T12:14:50.796000+0000","last_clean":"2024-03-14T12:14:50.796000+0000","last_became_active":"2024-03-14T12:14:49.802517+0000","last_became_peered":"2024-03-14T12:14:49.802517+0000","last_unstale":"2024-03-14T12:14:50.796000+0000","last_undegraded":"2024-03-14T12:14:50.796000+0000","last_fullsized":"2024-03-14T12:14:50.796000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:10:48.711082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795214+0000","last_change":"2024-03-14T12:14:49.800830+0000","last_active":"2024-03-14T12:14:50.795214+0000","last_peered":"2024-03-14T12:14:50.795214+0000","last_clean":"2024-03-14T12:14:50.795214+0000","last_became_active":"2024-03-14T12:14:49.800539+0000","last_became_peered":"2024-03-14T12:14:49.800539+0000","last_unstale":"2024-03-14T12:14:50.795214+0000","last_undegraded":"2024-03-14T12:14:50.795214+0000","last_fullsized":"2024-03-14T12:14:50.795214+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:02:17.484737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796464+0000","last_change":"2024-03-14T12:14:49.801416+0000","last_active":"2024-03-14T12:14:50.796464+0000","last_peered":"2024-03-14T12:14:50.796464+0000","last_clean":"2024-03-14T12:14:50.796464+0000","last_became_active":"2024-03-14T12:14:49.801120+0000","last_became_peered":"2024-03-14T12:14:49.801120+0000","last_unstale":"2024-03-14T12:14:50.796464+0000","last_undegraded":"2024-03-14T12:14:50.796464+0000","last_fullsized":"2024-03-14T12:14:50.796464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:25:41.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.835963+0000","last_change":"2024-03-14T12:14:49.803300+0000","last_active":"2024-03-14T12:14:50.835963+0000","last_peered":"2024-03-14T12:14:50.835963+0000","last_clean":"2024-03-14T12:14:50.835963+0000","last_became_active":"2024-03-14T12:14:49.803001+0000","last_became_peered":"2024-03-14T12:14:49.803001+0000","last_unstale":"2024-03-14T12:14:50.835963+0000","last_undegraded":"2024-03-14T12:14:50.835963+0000","last_fullsized":"2024-03-14T12:14:50.835963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:08:53.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796050+0000","last_change":"2024-03-14T12:14:49.802450+0000","last_active":"2024-03-14T12:14:50.796050+0000","last_peered":"2024-03-14T12:14:50.796050+0000","last_clean":"2024-03-14T12:14:50.796050+0000","last_became_active":"2024-03-14T12:14:49.802183+0000","last_became_peered":"2024-03-14T12:14:49.802183+0000","last_unstale":"2024-03-14T12:14:50.796050+0000","last_undegraded":"2024-03-14T12:14:50.796050+0000","last_fullsized":"2024-03-14T12:14:50.796050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:33:04.523501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976719+0000","last_change":"2024-03-14T12:14:49.806028+0000","last_active":"2024-03-14T12:14:50.976719+0000","last_peered":"2024-03-14T12:14:50.976719+0000","last_clean":"2024-03-14T12:14:50.976719+0000","last_became_active":"2024-03-14T12:14:49.805700+0000","last_became_peered":"2024-03-14T12:14:49.805700+0000","last_unstale":"2024-03-14T12:14:50.976719+0000","last_undegraded":"2024-03-14T12:14:50.976719+0000","last_fullsized":"2024-03-14T12:14:50.976719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:58:12.693037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976235+0000","last_change":"2024-03-14T12:14:49.807895+0000","last_active":"2024-03-14T12:14:50.976235+0000","last_peered":"2024-03-14T12:14:50.976235+0000","last_clean":"2024-03-14T12:14:50.976235+0000","last_became_active":"2024-03-14T12:14:49.807631+0000","last_became_peered":"2024-03-14T12:14:49.807631+0000","last_unstale":"2024-03-14T12:14:50.976235+0000","last_undegraded":"2024-03-14T12:14:50.976235+0000","last_fullsized":"2024-03-14T12:14:50.976235+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:26.813769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794608+0000","last_change":"2024-03-14T12:14:49.805092+0000","last_active":"2024-03-14T12:14:50.794608+0000","last_peered":"2024-03-14T12:14:50.794608+0000","last_clean":"2024-03-14T12:14:50.794608+0000","last_became_active":"2024-03-14T12:14:49.804797+0000","last_became_peered":"2024-03-14T12:14:49.804797+0000","last_unstale":"2024-03-14T12:14:50.794608+0000","last_undegraded":"2024-03-14T12:14:50.794608+0000","last_fullsized":"2024-03-14T12:14:50.794608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:10:48.711082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976460+0000","last_change":"2024-03-14T12:14:49.797864+0000","last_active":"2024-03-14T12:14:50.976460+0000","last_peered":"2024-03-14T12:14:50.976460+0000","last_clean":"2024-03-14T12:14:50.976460+0000","last_became_active":"2024-03-14T12:14:49.797114+0000","last_became_peered":"2024-03-14T12:14:49.797114+0000","last_unstale":"2024-03-14T12:14:50.976460+0000","last_undegraded":"2024-03-14T12:14:50.976460+0000","last_fullsized":"2024-03-14T12:14:50.976460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:08:53.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795648+0000","last_change":"2024-03-14T12:14:49.808306+0000","last_active":"2024-03-14T12:14:50.795648+0000","last_peered":"2024-03-14T12:14:50.795648+0000","last_clean":"2024-03-14T12:14:50.795648+0000","last_became_active":"2024-03-14T12:14:49.807598+0000","last_became_peered":"2024-03-14T12:14:49.807598+0000","last_unstale":"2024-03-14T12:14:50.795648+0000","last_undegraded":"2024-03-14T12:14:50.795648+0000","last_fullsized":"2024-03-14T12:14:50.795648+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:17:27.554080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795404+0000","last_change":"2024-03-14T12:14:49.802170+0000","last_active":"2024-03-14T12:14:50.795404+0000","last_peered":"2024-03-14T12:14:50.795404+0000","last_clean":"2024-03-14T12:14:50.795404+0000","last_became_active":"2024-03-14T12:14:49.801847+0000","last_became_peered":"2024-03-14T12:14:49.801847+0000","last_unstale":"2024-03-14T12:14:50.795404+0000","last_undegraded":"2024-03-14T12:14:50.795404+0000","last_fullsized":"2024-03-14T12:14:50.795404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:39:12.040885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794756+0000","last_change":"2024-03-14T12:14:49.808208+0000","last_active":"2024-03-14T12:14:50.794756+0000","last_peered":"2024-03-14T12:14:50.794756+0000","last_clean":"2024-03-14T12:14:50.794756+0000","last_became_active":"2024-03-14T12:14:49.807782+0000","last_became_peered":"2024-03-14T12:14:49.807782+0000","last_unstale":"2024-03-14T12:14:50.794756+0000","last_undegraded":"2024-03-14T12:14:50.794756+0000","last_fullsized":"2024-03-14T12:14:50.794756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:14:41.571575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794478+0000","last_change":"2024-03-14T12:14:49.806537+0000","last_active":"2024-03-14T12:14:50.794478+0000","last_peered":"2024-03-14T12:14:50.794478+0000","last_clean":"2024-03-14T12:14:50.794478+0000","last_became_active":"2024-03-14T12:14:49.806286+0000","last_became_peered":"2024-03-14T12:14:49.806286+0000","last_unstale":"2024-03-14T12:14:50.794478+0000","last_undegraded":"2024-03-14T12:14:50.794478+0000","last_fullsized":"2024-03-14T12:14:50.794478+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:17:27.554080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976296+0000","last_change":"2024-03-14T12:14:49.801734+0000","last_active":"2024-03-14T12:14:50.976296+0000","last_peered":"2024-03-14T12:14:50.976296+0000","last_clean":"2024-03-14T12:14:50.976296+0000","last_became_active":"2024-03-14T12:14:49.801448+0000","last_became_peered":"2024-03-14T12:14:49.801448+0000","last_unstale":"2024-03-14T12:14:50.976296+0000","last_undegraded":"2024-03-14T12:14:50.976296+0000","last_fullsized":"2024-03-14T12:14:50.976296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:56:06.120611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795692+0000","last_change":"2024-03-14T12:14:49.796280+0000","last_active":"2024-03-14T12:14:50.795692+0000","last_peered":"2024-03-14T12:14:50.795692+0000","last_clean":"2024-03-14T12:14:50.795692+0000","last_became_active":"2024-03-14T12:14:49.795995+0000","last_became_peered":"2024-03-14T12:14:49.795995+0000","last_unstale":"2024-03-14T12:14:50.795692+0000","last_undegraded":"2024-03-14T12:14:50.795692+0000","last_fullsized":"2024-03-14T12:14:50.795692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:51:10.076907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795611+0000","last_change":"2024-03-14T12:14:49.805896+0000","last_active":"2024-03-14T12:14:50.795611+0000","last_peered":"2024-03-14T12:14:50.795611+0000","last_clean":"2024-03-14T12:14:50.795611+0000","last_became_active":"2024-03-14T12:14:49.805611+0000","last_became_peered":"2024-03-14T12:14:49.805611+0000","last_unstale":"2024-03-14T12:14:50.795611+0000","last_undegraded":"2024-03-14T12:14:50.795611+0000","last_fullsized":"2024-03-14T12:14:50.795611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:15:59.525379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.883743+0000","last_change":"2024-03-14T12:14:49.798708+0000","last_active":"2024-03-14T12:14:50.883743+0000","last_peered":"2024-03-14T12:14:50.883743+0000","last_clean":"2024-03-14T12:14:50.883743+0000","last_became_active":"2024-03-14T12:14:49.798430+0000","last_became_peered":"2024-03-14T12:14:49.798430+0000","last_unstale":"2024-03-14T12:14:50.883743+0000","last_undegraded":"2024-03-14T12:14:50.883743+0000","last_fullsized":"2024-03-14T12:14:50.883743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:54:17.524668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976390+0000","last_change":"2024-03-14T12:14:49.799275+0000","last_active":"2024-03-14T12:14:50.976390+0000","last_peered":"2024-03-14T12:14:50.976390+0000","last_clean":"2024-03-14T12:14:50.976390+0000","last_became_active":"2024-03-14T12:14:49.799010+0000","last_became_peered":"2024-03-14T12:14:49.799010+0000","last_unstale":"2024-03-14T12:14:50.976390+0000","last_undegraded":"2024-03-14T12:14:50.976390+0000","last_fullsized":"2024-03-14T12:14:50.976390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:02:17.484737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.836021+0000","last_change":"2024-03-14T12:14:49.802484+0000","last_active":"2024-03-14T12:14:50.836021+0000","last_peered":"2024-03-14T12:14:50.836021+0000","last_clean":"2024-03-14T12:14:50.836021+0000","last_became_active":"2024-03-14T12:14:49.802218+0000","last_became_peered":"2024-03-14T12:14:49.802218+0000","last_unstale":"2024-03-14T12:14:50.836021+0000","last_undegraded":"2024-03-14T12:14:50.836021+0000","last_fullsized":"2024-03-14T12:14:50.836021+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:08:34.373406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794864+0000","last_change":"2024-03-14T12:14:49.797756+0000","last_active":"2024-03-14T12:14:50.794864+0000","last_peered":"2024-03-14T12:14:50.794864+0000","last_clean":"2024-03-14T12:14:50.794864+0000","last_became_active":"2024-03-14T12:14:49.797453+0000","last_became_peered":"2024-03-14T12:14:49.797453+0000","last_unstale":"2024-03-14T12:14:50.794864+0000","last_undegraded":"2024-03-14T12:14:50.794864+0000","last_fullsized":"2024-03-14T12:14:50.794864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:41.131023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.836199+0000","last_change":"2024-03-14T12:14:49.802790+0000","last_active":"2024-03-14T12:14:50.836199+0000","last_peered":"2024-03-14T12:14:50.836199+0000","last_clean":"2024-03-14T12:14:50.836199+0000","last_became_active":"2024-03-14T12:14:49.802469+0000","last_became_peered":"2024-03-14T12:14:49.802469+0000","last_unstale":"2024-03-14T12:14:50.836199+0000","last_undegraded":"2024-03-14T12:14:50.836199+0000","last_fullsized":"2024-03-14T12:14:50.836199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:39:12.040885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976703+0000","last_change":"2024-03-14T12:14:49.796634+0000","last_active":"2024-03-14T12:14:50.976703+0000","last_peered":"2024-03-14T12:14:50.976703+0000","last_clean":"2024-03-14T12:14:50.976703+0000","last_became_active":"2024-03-14T12:14:49.796326+0000","last_became_peered":"2024-03-14T12:14:49.796326+0000","last_unstale":"2024-03-14T12:14:50.976703+0000","last_undegraded":"2024-03-14T12:14:50.976703+0000","last_fullsized":"2024-03-14T12:14:50.976703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:17.283711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795864+0000","last_change":"2024-03-14T12:14:49.797604+0000","last_active":"2024-03-14T12:14:50.795864+0000","last_peered":"2024-03-14T12:14:50.795864+0000","last_clean":"2024-03-14T12:14:50.795864+0000","last_became_active":"2024-03-14T12:14:49.796896+0000","last_became_peered":"2024-03-14T12:14:49.796896+0000","last_unstale":"2024-03-14T12:14:50.795864+0000","last_undegraded":"2024-03-14T12:14:50.795864+0000","last_fullsized":"2024-03-14T12:14:50.795864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:16:26.476198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795585+0000","last_change":"2024-03-14T12:14:49.801841+0000","last_active":"2024-03-14T12:14:50.795585+0000","last_peered":"2024-03-14T12:14:50.795585+0000","last_clean":"2024-03-14T12:14:50.795585+0000","last_became_active":"2024-03-14T12:14:49.801550+0000","last_became_peered":"2024-03-14T12:14:49.801550+0000","last_unstale":"2024-03-14T12:14:50.795585+0000","last_undegraded":"2024-03-14T12:14:50.795585+0000","last_fullsized":"2024-03-14T12:14:50.795585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:41:06.594787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889830+0000","last_change":"2024-03-14T12:14:49.798635+0000","last_active":"2024-03-14T12:14:50.889830+0000","last_peered":"2024-03-14T12:14:50.889830+0000","last_clean":"2024-03-14T12:14:50.889830+0000","last_became_active":"2024-03-14T12:14:49.798123+0000","last_became_peered":"2024-03-14T12:14:49.798123+0000","last_unstale":"2024-03-14T12:14:50.889830+0000","last_undegraded":"2024-03-14T12:14:50.889830+0000","last_fullsized":"2024-03-14T12:14:50.889830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:51:52.328291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795271+0000","last_change":"2024-03-14T12:14:49.806610+0000","last_active":"2024-03-14T12:14:50.795271+0000","last_peered":"2024-03-14T12:14:50.795271+0000","last_clean":"2024-03-14T12:14:50.795271+0000","last_became_active":"2024-03-14T12:14:49.804181+0000","last_became_peered":"2024-03-14T12:14:49.804181+0000","last_unstale":"2024-03-14T12:14:50.795271+0000","last_undegraded":"2024-03-14T12:14:50.795271+0000","last_fullsized":"2024-03-14T12:14:50.795271+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:08:53.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.883930+0000","last_change":"2024-03-14T12:14:49.798063+0000","last_active":"2024-03-14T12:14:50.883930+0000","last_peered":"2024-03-14T12:14:50.883930+0000","last_clean":"2024-03-14T12:14:50.883930+0000","last_became_active":"2024-03-14T12:14:49.797802+0000","last_became_peered":"2024-03-14T12:14:49.797802+0000","last_unstale":"2024-03-14T12:14:50.883930+0000","last_undegraded":"2024-03-14T12:14:50.883930+0000","last_fullsized":"2024-03-14T12:14:50.883930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:16:26.476198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889931+0000","last_change":"2024-03-14T12:14:48.802531+0000","last_active":"2024-03-14T12:14:50.889931+0000","last_peered":"2024-03-14T12:14:50.889931+0000","last_clean":"2024-03-14T12:14:50.889931+0000","last_became_active":"2024-03-14T12:14:48.802272+0000","last_became_peered":"2024-03-14T12:14:48.802272+0000","last_unstale":"2024-03-14T12:14:50.889931+0000","last_undegraded":"2024-03-14T12:14:50.889931+0000","last_fullsized":"2024-03-14T12:14:50.889931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:18:40.626722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889948+0000","last_change":"2024-03-14T12:14:49.802587+0000","last_active":"2024-03-14T12:14:50.889948+0000","last_peered":"2024-03-14T12:14:50.889948+0000","last_clean":"2024-03-14T12:14:50.889948+0000","last_became_active":"2024-03-14T12:14:49.802276+0000","last_became_peered":"2024-03-14T12:14:49.802276+0000","last_unstale":"2024-03-14T12:14:50.889948+0000","last_undegraded":"2024-03-14T12:14:50.889948+0000","last_fullsized":"2024-03-14T12:14:50.889948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:54:17.524668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.883977+0000","last_change":"2024-03-14T12:14:48.801279+0000","last_active":"2024-03-14T12:14:50.883977+0000","last_peered":"2024-03-14T12:14:50.883977+0000","last_clean":"2024-03-14T12:14:50.883977+0000","last_became_active":"2024-03-14T12:14:48.801053+0000","last_became_peered":"2024-03-14T12:14:48.801053+0000","last_unstale":"2024-03-14T12:14:50.883977+0000","last_undegraded":"2024-03-14T12:14:50.883977+0000","last_fullsized":"2024-03-14T12:14:50.883977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:08:32.656175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796545+0000","last_change":"2024-03-14T12:14:49.807314+0000","last_active":"2024-03-14T12:14:50.796545+0000","last_peered":"2024-03-14T12:14:50.796545+0000","last_clean":"2024-03-14T12:14:50.796545+0000","last_became_active":"2024-03-14T12:14:49.807041+0000","last_became_peered":"2024-03-14T12:14:49.807041+0000","last_unstale":"2024-03-14T12:14:50.796545+0000","last_undegraded":"2024-03-14T12:14:50.796545+0000","last_fullsized":"2024-03-14T12:14:50.796545+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:44.554939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794395+0000","last_change":"2024-03-14T12:14:48.791173+0000","last_active":"2024-03-14T12:14:50.794395+0000","last_peered":"2024-03-14T12:14:50.794395+0000","last_clean":"2024-03-14T12:14:50.794395+0000","last_became_active":"2024-03-14T12:14:48.790907+0000","last_became_peered":"2024-03-14T12:14:48.790907+0000","last_unstale":"2024-03-14T12:14:50.794395+0000","last_undegraded":"2024-03-14T12:14:50.794395+0000","last_fullsized":"2024-03-14T12:14:50.794395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:36:20.050890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.836247+0000","last_change":"2024-03-14T12:14:49.795712+0000","last_active":"2024-03-14T12:14:50.836247+0000","last_peered":"2024-03-14T12:14:50.836247+0000","last_clean":"2024-03-14T12:14:50.836247+0000","last_became_active":"2024-03-14T12:14:49.795452+0000","last_became_peered":"2024-03-14T12:14:49.795452+0000","last_unstale":"2024-03-14T12:14:50.836247+0000","last_undegraded":"2024-03-14T12:14:50.836247+0000","last_fullsized":"2024-03-14T12:14:50.836247+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:17.283711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976297+0000","last_change":"2024-03-14T12:14:48.794760+0000","last_active":"2024-03-14T12:14:50.976297+0000","last_peered":"2024-03-14T12:14:50.976297+0000","last_clean":"2024-03-14T12:14:50.976297+0000","last_became_active":"2024-03-14T12:14:48.794494+0000","last_became_peered":"2024-03-14T12:14:48.794494+0000","last_unstale":"2024-03-14T12:14:50.976297+0000","last_undegraded":"2024-03-14T12:14:50.976297+0000","last_fullsized":"2024-03-14T12:14:50.976297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:57:59.913979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795744+0000","last_change":"2024-03-14T12:14:49.803658+0000","last_active":"2024-03-14T12:14:50.795744+0000","last_peered":"2024-03-14T12:14:50.795744+0000","last_clean":"2024-03-14T12:14:50.795744+0000","last_became_active":"2024-03-14T12:14:49.803359+0000","last_became_peered":"2024-03-14T12:14:49.803359+0000","last_unstale":"2024-03-14T12:14:50.795744+0000","last_undegraded":"2024-03-14T12:14:50.795744+0000","last_fullsized":"2024-03-14T12:14:50.795744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:27.088830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795932+0000","last_change":"2024-03-14T12:14:48.802366+0000","last_active":"2024-03-14T12:14:50.795932+0000","last_peered":"2024-03-14T12:14:50.795932+0000","last_clean":"2024-03-14T12:14:50.795932+0000","last_became_active":"2024-03-14T12:14:48.802120+0000","last_became_peered":"2024-03-14T12:14:48.802120+0000","last_unstale":"2024-03-14T12:14:50.795932+0000","last_undegraded":"2024-03-14T12:14:50.795932+0000","last_fullsized":"2024-03-14T12:14:50.795932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:30:24.735629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976579+0000","last_change":"2024-03-14T12:14:49.796712+0000","last_active":"2024-03-14T12:14:50.976579+0000","last_peered":"2024-03-14T12:14:50.976579+0000","last_clean":"2024-03-14T12:14:50.976579+0000","last_became_active":"2024-03-14T12:14:49.796433+0000","last_became_peered":"2024-03-14T12:14:49.796433+0000","last_unstale":"2024-03-14T12:14:50.976579+0000","last_undegraded":"2024-03-14T12:14:50.976579+0000","last_fullsized":"2024-03-14T12:14:50.976579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:08:34.373406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795670+0000","last_change":"2024-03-14T12:14:48.801018+0000","last_active":"2024-03-14T12:14:50.795670+0000","last_peered":"2024-03-14T12:14:50.795670+0000","last_clean":"2024-03-14T12:14:50.795670+0000","last_became_active":"2024-03-14T12:14:48.800707+0000","last_became_peered":"2024-03-14T12:14:48.800707+0000","last_unstale":"2024-03-14T12:14:50.795670+0000","last_undegraded":"2024-03-14T12:14:50.795670+0000","last_fullsized":"2024-03-14T12:14:50.795670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:18:40.626722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795558+0000","last_change":"2024-03-14T12:14:49.798574+0000","last_active":"2024-03-14T12:14:50.795558+0000","last_peered":"2024-03-14T12:14:50.795558+0000","last_clean":"2024-03-14T12:14:50.795558+0000","last_became_active":"2024-03-14T12:14:49.798308+0000","last_became_peered":"2024-03-14T12:14:49.798308+0000","last_unstale":"2024-03-14T12:14:50.795558+0000","last_undegraded":"2024-03-14T12:14:50.795558+0000","last_fullsized":"2024-03-14T12:14:50.795558+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:58:34.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794127+0000","last_change":"2024-03-14T12:14:48.792526+0000","last_active":"2024-03-14T12:14:50.794127+0000","last_peered":"2024-03-14T12:14:50.794127+0000","last_clean":"2024-03-14T12:14:50.794127+0000","last_became_active":"2024-03-14T12:14:48.792272+0000","last_became_peered":"2024-03-14T12:14:48.792272+0000","last_unstale":"2024-03-14T12:14:50.794127+0000","last_undegraded":"2024-03-14T12:14:50.794127+0000","last_fullsized":"2024-03-14T12:14:50.794127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:15.910454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.835674+0000","last_change":"2024-03-14T12:14:49.791341+0000","last_active":"2024-03-14T12:14:50.835674+0000","last_peered":"2024-03-14T12:14:50.835674+0000","last_clean":"2024-03-14T12:14:50.835674+0000","last_became_active":"2024-03-14T12:14:49.791066+0000","last_became_peered":"2024-03-14T12:14:49.791066+0000","last_unstale":"2024-03-14T12:14:50.835674+0000","last_undegraded":"2024-03-14T12:14:50.835674+0000","last_fullsized":"2024-03-14T12:14:50.835674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:05:27.709896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795629+0000","last_change":"2024-03-14T12:14:48.797076+0000","last_active":"2024-03-14T12:14:50.795629+0000","last_peered":"2024-03-14T12:14:50.795629+0000","last_clean":"2024-03-14T12:14:50.795629+0000","last_became_active":"2024-03-14T12:14:48.796648+0000","last_became_peered":"2024-03-14T12:14:48.796648+0000","last_unstale":"2024-03-14T12:14:50.795629+0000","last_undegraded":"2024-03-14T12:14:50.795629+0000","last_fullsized":"2024-03-14T12:14:50.795629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:58.036578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976827+0000","last_change":"2024-03-14T12:14:49.803972+0000","last_active":"2024-03-14T12:14:50.976827+0000","last_peered":"2024-03-14T12:14:50.976827+0000","last_clean":"2024-03-14T12:14:50.976827+0000","last_became_active":"2024-03-14T12:14:49.803681+0000","last_became_peered":"2024-03-14T12:14:49.803681+0000","last_unstale":"2024-03-14T12:14:50.976827+0000","last_undegraded":"2024-03-14T12:14:50.976827+0000","last_fullsized":"2024-03-14T12:14:50.976827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:17:27.554080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794695+0000","last_change":"2024-03-14T12:14:48.795654+0000","last_active":"2024-03-14T12:14:50.794695+0000","last_peered":"2024-03-14T12:14:50.794695+0000","last_clean":"2024-03-14T12:14:50.794695+0000","last_became_active":"2024-03-14T12:14:48.795308+0000","last_became_peered":"2024-03-14T12:14:48.795308+0000","last_unstale":"2024-03-14T12:14:50.794695+0000","last_undegraded":"2024-03-14T12:14:50.794695+0000","last_fullsized":"2024-03-14T12:14:50.794695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:54:16.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794652+0000","last_change":"2024-03-14T12:14:49.806368+0000","last_active":"2024-03-14T12:14:50.794652+0000","last_peered":"2024-03-14T12:14:50.794652+0000","last_clean":"2024-03-14T12:14:50.794652+0000","last_became_active":"2024-03-14T12:14:49.806088+0000","last_became_peered":"2024-03-14T12:14:49.806088+0000","last_unstale":"2024-03-14T12:14:50.794652+0000","last_undegraded":"2024-03-14T12:14:50.794652+0000","last_fullsized":"2024-03-14T12:14:50.794652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:46:03.087839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.884029+0000","last_change":"2024-03-14T12:14:48.793702+0000","last_active":"2024-03-14T12:14:50.884029+0000","last_peered":"2024-03-14T12:14:50.884029+0000","last_clean":"2024-03-14T12:14:50.884029+0000","last_became_active":"2024-03-14T12:14:48.793417+0000","last_became_peered":"2024-03-14T12:14:48.793417+0000","last_unstale":"2024-03-14T12:14:50.884029+0000","last_undegraded":"2024-03-14T12:14:50.884029+0000","last_fullsized":"2024-03-14T12:14:50.884029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:47:04.863848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794475+0000","last_change":"2024-03-14T12:14:49.797719+0000","last_active":"2024-03-14T12:14:50.794475+0000","last_peered":"2024-03-14T12:14:50.794475+0000","last_clean":"2024-03-14T12:14:50.794475+0000","last_became_active":"2024-03-14T12:14:49.797424+0000","last_became_peered":"2024-03-14T12:14:49.797424+0000","last_unstale":"2024-03-14T12:14:50.794475+0000","last_undegraded":"2024-03-14T12:14:50.794475+0000","last_fullsized":"2024-03-14T12:14:50.794475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:41:06.594787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976740+0000","last_change":"2024-03-14T12:14:48.802470+0000","last_active":"2024-03-14T12:14:50.976740+0000","last_peered":"2024-03-14T12:14:50.976740+0000","last_clean":"2024-03-14T12:14:50.976740+0000","last_became_active":"2024-03-14T12:14:48.802188+0000","last_became_peered":"2024-03-14T12:14:48.802188+0000","last_unstale":"2024-03-14T12:14:50.976740+0000","last_undegraded":"2024-03-14T12:14:50.976740+0000","last_fullsized":"2024-03-14T12:14:50.976740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:40:34.224642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976657+0000","last_change":"2024-03-14T12:14:49.795883+0000","last_active":"2024-03-14T12:14:50.976657+0000","last_peered":"2024-03-14T12:14:50.976657+0000","last_clean":"2024-03-14T12:14:50.976657+0000","last_became_active":"2024-03-14T12:14:49.795394+0000","last_became_peered":"2024-03-14T12:14:49.795394+0000","last_unstale":"2024-03-14T12:14:50.976657+0000","last_undegraded":"2024-03-14T12:14:50.976657+0000","last_fullsized":"2024-03-14T12:14:50.976657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:39:12.040885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794410+0000","last_change":"2024-03-14T12:14:48.793418+0000","last_active":"2024-03-14T12:14:50.794410+0000","last_peered":"2024-03-14T12:14:50.794410+0000","last_clean":"2024-03-14T12:14:50.794410+0000","last_became_active":"2024-03-14T12:14:48.793130+0000","last_became_peered":"2024-03-14T12:14:48.793130+0000","last_unstale":"2024-03-14T12:14:50.794410+0000","last_undegraded":"2024-03-14T12:14:50.794410+0000","last_fullsized":"2024-03-14T12:14:50.794410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:40:39.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.835683+0000","last_change":"2024-03-14T12:14:49.793073+0000","last_active":"2024-03-14T12:14:50.835683+0000","last_peered":"2024-03-14T12:14:50.835683+0000","last_clean":"2024-03-14T12:14:50.835683+0000","last_became_active":"2024-03-14T12:14:49.792714+0000","last_became_peered":"2024-03-14T12:14:49.792714+0000","last_unstale":"2024-03-14T12:14:50.835683+0000","last_undegraded":"2024-03-14T12:14:50.835683+0000","last_fullsized":"2024-03-14T12:14:50.835683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:25:41.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796044+0000","last_change":"2024-03-14T12:14:48.798353+0000","last_active":"2024-03-14T12:14:50.796044+0000","last_peered":"2024-03-14T12:14:50.796044+0000","last_clean":"2024-03-14T12:14:50.796044+0000","last_became_active":"2024-03-14T12:14:48.798045+0000","last_became_peered":"2024-03-14T12:14:48.798045+0000","last_unstale":"2024-03-14T12:14:50.796044+0000","last_undegraded":"2024-03-14T12:14:50.796044+0000","last_fullsized":"2024-03-14T12:14:50.796044+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:10:23.765238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794561+0000","last_change":"2024-03-14T12:14:49.799593+0000","last_active":"2024-03-14T12:14:50.794561+0000","last_peered":"2024-03-14T12:14:50.794561+0000","last_clean":"2024-03-14T12:14:50.794561+0000","last_became_active":"2024-03-14T12:14:49.799217+0000","last_became_peered":"2024-03-14T12:14:49.799217+0000","last_unstale":"2024-03-14T12:14:50.794561+0000","last_undegraded":"2024-03-14T12:14:50.794561+0000","last_fullsized":"2024-03-14T12:14:50.794561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:27.088830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.835621+0000","last_change":"2024-03-14T12:14:48.792685+0000","last_active":"2024-03-14T12:14:50.835621+0000","last_peered":"2024-03-14T12:14:50.835621+0000","last_clean":"2024-03-14T12:14:50.835621+0000","last_became_active":"2024-03-14T12:14:48.792380+0000","last_became_peered":"2024-03-14T12:14:48.792380+0000","last_unstale":"2024-03-14T12:14:50.835621+0000","last_undegraded":"2024-03-14T12:14:50.835621+0000","last_fullsized":"2024-03-14T12:14:50.835621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:47:04.863848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795429+0000","last_change":"2024-03-14T12:14:49.804497+0000","last_active":"2024-03-14T12:14:50.795429+0000","last_peered":"2024-03-14T12:14:50.795429+0000","last_clean":"2024-03-14T12:14:50.795429+0000","last_became_active":"2024-03-14T12:14:49.804184+0000","last_became_peered":"2024-03-14T12:14:49.804184+0000","last_unstale":"2024-03-14T12:14:50.795429+0000","last_undegraded":"2024-03-14T12:14:50.795429+0000","last_fullsized":"2024-03-14T12:14:50.795429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:17.283711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794174+0000","last_change":"2024-03-14T12:14:48.795683+0000","last_active":"2024-03-14T12:14:50.794174+0000","last_peered":"2024-03-14T12:14:50.794174+0000","last_clean":"2024-03-14T12:14:50.794174+0000","last_became_active":"2024-03-14T12:14:48.795392+0000","last_became_peered":"2024-03-14T12:14:48.795392+0000","last_unstale":"2024-03-14T12:14:50.794174+0000","last_undegraded":"2024-03-14T12:14:50.794174+0000","last_fullsized":"2024-03-14T12:14:50.794174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:51:51.323914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889746+0000","last_change":"2024-03-14T12:14:49.795454+0000","last_active":"2024-03-14T12:14:50.889746+0000","last_peered":"2024-03-14T12:14:50.889746+0000","last_clean":"2024-03-14T12:14:50.889746+0000","last_became_active":"2024-03-14T12:14:49.795178+0000","last_became_peered":"2024-03-14T12:14:49.795178+0000","last_unstale":"2024-03-14T12:14:50.889746+0000","last_undegraded":"2024-03-14T12:14:50.889746+0000","last_fullsized":"2024-03-14T12:14:50.889746+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:40:40.255526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794144+0000","last_change":"2024-03-14T12:14:48.791161+0000","last_active":"2024-03-14T12:14:50.794144+0000","last_peered":"2024-03-14T12:14:50.794144+0000","last_clean":"2024-03-14T12:14:50.794144+0000","last_became_active":"2024-03-14T12:14:48.790891+0000","last_became_peered":"2024-03-14T12:14:48.790891+0000","last_unstale":"2024-03-14T12:14:50.794144+0000","last_undegraded":"2024-03-14T12:14:50.794144+0000","last_fullsized":"2024-03-14T12:14:50.794144+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:40:34.224642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796305+0000","last_change":"2024-03-14T12:14:49.795604+0000","last_active":"2024-03-14T12:14:50.796305+0000","last_peered":"2024-03-14T12:14:50.796305+0000","last_clean":"2024-03-14T12:14:50.796305+0000","last_became_active":"2024-03-14T12:14:49.795306+0000","last_became_peered":"2024-03-14T12:14:49.795306+0000","last_unstale":"2024-03-14T12:14:50.796305+0000","last_undegraded":"2024-03-14T12:14:50.796305+0000","last_fullsized":"2024-03-14T12:14:50.796305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:05:27.709896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.836008+0000","last_change":"2024-03-14T12:14:48.796621+0000","last_active":"2024-03-14T12:14:50.836008+0000","last_peered":"2024-03-14T12:14:50.836008+0000","last_clean":"2024-03-14T12:14:50.836008+0000","last_became_active":"2024-03-14T12:14:48.796349+0000","last_became_peered":"2024-03-14T12:14:48.796349+0000","last_unstale":"2024-03-14T12:14:50.836008+0000","last_undegraded":"2024-03-14T12:14:50.836008+0000","last_fullsized":"2024-03-14T12:14:50.836008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:25.809392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976836+0000","last_change":"2024-03-14T12:14:49.798317+0000","last_active":"2024-03-14T12:14:50.976836+0000","last_peered":"2024-03-14T12:14:50.976836+0000","last_clean":"2024-03-14T12:14:50.976836+0000","last_became_active":"2024-03-14T12:14:49.797703+0000","last_became_peered":"2024-03-14T12:14:49.797703+0000","last_unstale":"2024-03-14T12:14:50.976836+0000","last_undegraded":"2024-03-14T12:14:50.976836+0000","last_fullsized":"2024-03-14T12:14:50.976836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:27.088830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796188+0000","last_change":"2024-03-14T12:14:48.799353+0000","last_active":"2024-03-14T12:14:50.796188+0000","last_peered":"2024-03-14T12:14:50.796188+0000","last_clean":"2024-03-14T12:14:50.796188+0000","last_became_active":"2024-03-14T12:14:48.799040+0000","last_became_peered":"2024-03-14T12:14:48.799040+0000","last_unstale":"2024-03-14T12:14:50.796188+0000","last_undegraded":"2024-03-14T12:14:50.796188+0000","last_fullsized":"2024-03-14T12:14:50.796188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:25.809392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.835916+0000","last_change":"2024-03-14T12:14:49.796353+0000","last_active":"2024-03-14T12:14:50.835916+0000","last_peered":"2024-03-14T12:14:50.835916+0000","last_clean":"2024-03-14T12:14:50.835916+0000","last_became_active":"2024-03-14T12:14:49.796095+0000","last_became_peered":"2024-03-14T12:14:49.796095+0000","last_unstale":"2024-03-14T12:14:50.835916+0000","last_undegraded":"2024-03-14T12:14:50.835916+0000","last_fullsized":"2024-03-14T12:14:50.835916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:44.554939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796306+0000","last_change":"2024-03-14T12:14:48.798080+0000","last_active":"2024-03-14T12:14:50.796306+0000","last_peered":"2024-03-14T12:14:50.796306+0000","last_clean":"2024-03-14T12:14:50.796306+0000","last_became_active":"2024-03-14T12:14:48.797646+0000","last_became_peered":"2024-03-14T12:14:48.797646+0000","last_unstale":"2024-03-14T12:14:50.796306+0000","last_undegraded":"2024-03-14T12:14:50.796306+0000","last_fullsized":"2024-03-14T12:14:50.796306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:16:42.320110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976228+0000","last_change":"2024-03-14T12:14:49.802931+0000","last_active":"2024-03-14T12:14:50.976228+0000","last_peered":"2024-03-14T12:14:50.976228+0000","last_clean":"2024-03-14T12:14:50.976228+0000","last_became_active":"2024-03-14T12:14:49.802681+0000","last_became_peered":"2024-03-14T12:14:49.802681+0000","last_unstale":"2024-03-14T12:14:50.976228+0000","last_undegraded":"2024-03-14T12:14:50.976228+0000","last_fullsized":"2024-03-14T12:14:50.976228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:22.375857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.796508+0000","last_change":"2024-03-14T12:14:48.800965+0000","last_active":"2024-03-14T12:14:50.796508+0000","last_peered":"2024-03-14T12:14:50.796508+0000","last_clean":"2024-03-14T12:14:50.796508+0000","last_became_active":"2024-03-14T12:14:48.800694+0000","last_became_peered":"2024-03-14T12:14:48.800694+0000","last_unstale":"2024-03-14T12:14:50.796508+0000","last_undegraded":"2024-03-14T12:14:50.796508+0000","last_fullsized":"2024-03-14T12:14:50.796508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:21.371480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976333+0000","last_change":"2024-03-14T12:14:49.802374+0000","last_active":"2024-03-14T12:14:50.976333+0000","last_peered":"2024-03-14T12:14:50.976333+0000","last_clean":"2024-03-14T12:14:50.976333+0000","last_became_active":"2024-03-14T12:14:49.802104+0000","last_became_peered":"2024-03-14T12:14:49.802104+0000","last_unstale":"2024-03-14T12:14:50.976333+0000","last_undegraded":"2024-03-14T12:14:50.976333+0000","last_fullsized":"2024-03-14T12:14:50.976333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:47:05.868225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.883829+0000","last_change":"2024-03-14T12:14:48.794448+0000","last_active":"2024-03-14T12:14:50.883829+0000","last_peered":"2024-03-14T12:14:50.883829+0000","last_clean":"2024-03-14T12:14:50.883829+0000","last_became_active":"2024-03-14T12:14:48.794183+0000","last_became_peered":"2024-03-14T12:14:48.794183+0000","last_unstale":"2024-03-14T12:14:50.883829+0000","last_undegraded":"2024-03-14T12:14:50.883829+0000","last_fullsized":"2024-03-14T12:14:50.883829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:25.809392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976344+0000","last_change":"2024-03-14T12:14:49.799784+0000","last_active":"2024-03-14T12:14:50.976344+0000","last_peered":"2024-03-14T12:14:50.976344+0000","last_clean":"2024-03-14T12:14:50.976344+0000","last_became_active":"2024-03-14T12:14:49.799496+0000","last_became_peered":"2024-03-14T12:14:49.799496+0000","last_unstale":"2024-03-14T12:14:50.976344+0000","last_undegraded":"2024-03-14T12:14:50.976344+0000","last_fullsized":"2024-03-14T12:14:50.976344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:16.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889759+0000","last_change":"2024-03-14T12:14:48.798832+0000","last_active":"2024-03-14T12:14:50.889759+0000","last_peered":"2024-03-14T12:14:50.889759+0000","last_clean":"2024-03-14T12:14:50.889759+0000","last_became_active":"2024-03-14T12:14:48.798547+0000","last_became_peered":"2024-03-14T12:14:48.798547+0000","last_unstale":"2024-03-14T12:14:50.889759+0000","last_undegraded":"2024-03-14T12:14:50.889759+0000","last_fullsized":"2024-03-14T12:14:50.889759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:21.371480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976171+0000","last_change":"2024-03-14T12:14:49.799751+0000","last_active":"2024-03-14T12:14:50.976171+0000","last_peered":"2024-03-14T12:14:50.976171+0000","last_clean":"2024-03-14T12:14:50.976171+0000","last_became_active":"2024-03-14T12:14:49.799484+0000","last_became_peered":"2024-03-14T12:14:49.799484+0000","last_unstale":"2024-03-14T12:14:50.976171+0000","last_undegraded":"2024-03-14T12:14:50.976171+0000","last_fullsized":"2024-03-14T12:14:50.976171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:10:24.769615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795753+0000","last_change":"2024-03-14T12:14:48.797335+0000","last_active":"2024-03-14T12:14:50.795753+0000","last_peered":"2024-03-14T12:14:50.795753+0000","last_clean":"2024-03-14T12:14:50.795753+0000","last_became_active":"2024-03-14T12:14:48.797021+0000","last_became_peered":"2024-03-14T12:14:48.797021+0000","last_unstale":"2024-03-14T12:14:50.795753+0000","last_undegraded":"2024-03-14T12:14:50.795753+0000","last_fullsized":"2024-03-14T12:14:50.795753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:02.162022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795348+0000","last_change":"2024-03-14T12:14:49.802709+0000","last_active":"2024-03-14T12:14:50.795348+0000","last_peered":"2024-03-14T12:14:50.795348+0000","last_clean":"2024-03-14T12:14:50.795348+0000","last_became_active":"2024-03-14T12:14:49.800983+0000","last_became_peered":"2024-03-14T12:14:49.800983+0000","last_unstale":"2024-03-14T12:14:50.795348+0000","last_undegraded":"2024-03-14T12:14:50.795348+0000","last_fullsized":"2024-03-14T12:14:50.795348+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:08:34.373406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976246+0000","last_change":"2024-03-14T12:14:48.802456+0000","last_active":"2024-03-14T12:14:50.976246+0000","last_peered":"2024-03-14T12:14:50.976246+0000","last_clean":"2024-03-14T12:14:50.976246+0000","last_became_active":"2024-03-14T12:14:48.802122+0000","last_became_peered":"2024-03-14T12:14:48.802122+0000","last_unstale":"2024-03-14T12:14:50.976246+0000","last_undegraded":"2024-03-14T12:14:50.976246+0000","last_fullsized":"2024-03-14T12:14:50.976246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:36:20.050890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976225+0000","last_change":"2024-03-14T12:14:49.802244+0000","last_active":"2024-03-14T12:14:50.976225+0000","last_peered":"2024-03-14T12:14:50.976225+0000","last_clean":"2024-03-14T12:14:50.976225+0000","last_became_active":"2024-03-14T12:14:49.801982+0000","last_became_peered":"2024-03-14T12:14:49.801982+0000","last_unstale":"2024-03-14T12:14:50.976225+0000","last_undegraded":"2024-03-14T12:14:50.976225+0000","last_fullsized":"2024-03-14T12:14:50.976225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:18:41.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.836091+0000","last_change":"2024-03-14T12:14:48.801773+0000","last_active":"2024-03-14T12:14:50.836091+0000","last_peered":"2024-03-14T12:14:50.836091+0000","last_clean":"2024-03-14T12:14:50.836091+0000","last_became_active":"2024-03-14T12:14:48.801489+0000","last_became_peered":"2024-03-14T12:14:48.801489+0000","last_unstale":"2024-03-14T12:14:50.836091+0000","last_undegraded":"2024-03-14T12:14:50.836091+0000","last_fullsized":"2024-03-14T12:14:50.836091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:08:32.656175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.976852+0000","last_change":"2024-03-14T12:14:49.794589+0000","last_active":"2024-03-14T12:14:50.976852+0000","last_peered":"2024-03-14T12:14:50.976852+0000","last_clean":"2024-03-14T12:14:50.976852+0000","last_became_active":"2024-03-14T12:14:49.794319+0000","last_became_peered":"2024-03-14T12:14:49.794319+0000","last_unstale":"2024-03-14T12:14:50.976852+0000","last_undegraded":"2024-03-14T12:14:50.976852+0000","last_fullsized":"2024-03-14T12:14:50.976852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:10:48.711082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.884104+0000","last_change":"2024-03-14T12:14:48.795688+0000","last_active":"2024-03-14T12:14:50.884104+0000","last_peered":"2024-03-14T12:14:50.884104+0000","last_clean":"2024-03-14T12:14:50.884104+0000","last_became_active":"2024-03-14T12:14:48.794004+0000","last_became_peered":"2024-03-14T12:14:48.794004+0000","last_unstale":"2024-03-14T12:14:50.884104+0000","last_undegraded":"2024-03-14T12:14:50.884104+0000","last_fullsized":"2024-03-14T12:14:50.884104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:15.910454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.883826+0000","last_change":"2024-03-14T12:14:49.804754+0000","last_active":"2024-03-14T12:14:50.883826+0000","last_peered":"2024-03-14T12:14:50.883826+0000","last_clean":"2024-03-14T12:14:50.883826+0000","last_became_active":"2024-03-14T12:14:49.804479+0000","last_became_peered":"2024-03-14T12:14:49.804479+0000","last_unstale":"2024-03-14T12:14:50.883826+0000","last_undegraded":"2024-03-14T12:14:50.883826+0000","last_fullsized":"2024-03-14T12:14:50.883826+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:15:59.525379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.836010+0000","last_change":"2024-03-14T12:14:48.794824+0000","last_active":"2024-03-14T12:14:50.836010+0000","last_peered":"2024-03-14T12:14:50.836010+0000","last_clean":"2024-03-14T12:14:50.836010+0000","last_became_active":"2024-03-14T12:14:48.794550+0000","last_became_peered":"2024-03-14T12:14:48.794550+0000","last_unstale":"2024-03-14T12:14:50.836010+0000","last_undegraded":"2024-03-14T12:14:50.836010+0000","last_fullsized":"2024-03-14T12:14:50.836010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:21.371480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889710+0000","last_change":"2024-03-14T12:14:49.797416+0000","last_active":"2024-03-14T12:14:50.889710+0000","last_peered":"2024-03-14T12:14:50.889710+0000","last_clean":"2024-03-14T12:14:50.889710+0000","last_became_active":"2024-03-14T12:14:49.797149+0000","last_became_peered":"2024-03-14T12:14:49.797149+0000","last_unstale":"2024-03-14T12:14:50.889710+0000","last_undegraded":"2024-03-14T12:14:50.889710+0000","last_fullsized":"2024-03-14T12:14:50.889710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:36:21.055267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.883935+0000","last_change":"2024-03-14T12:14:48.802067+0000","last_active":"2024-03-14T12:14:50.883935+0000","last_peered":"2024-03-14T12:14:50.883935+0000","last_clean":"2024-03-14T12:14:50.883935+0000","last_became_active":"2024-03-14T12:14:48.801710+0000","last_became_peered":"2024-03-14T12:14:48.801710+0000","last_unstale":"2024-03-14T12:14:50.883935+0000","last_undegraded":"2024-03-14T12:14:50.883935+0000","last_fullsized":"2024-03-14T12:14:50.883935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:56:05.116234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.883869+0000","last_change":"2024-03-14T12:14:49.794099+0000","last_active":"2024-03-14T12:14:50.883869+0000","last_peered":"2024-03-14T12:14:50.883869+0000","last_clean":"2024-03-14T12:14:50.883869+0000","last_became_active":"2024-03-14T12:14:49.793806+0000","last_became_peered":"2024-03-14T12:14:49.793806+0000","last_unstale":"2024-03-14T12:14:50.883869+0000","last_undegraded":"2024-03-14T12:14:50.883869+0000","last_fullsized":"2024-03-14T12:14:50.883869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:51:10.076907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.835729+0000","last_change":"2024-03-14T12:14:48.797228+0000","last_active":"2024-03-14T12:14:50.835729+0000","last_peered":"2024-03-14T12:14:50.835729+0000","last_clean":"2024-03-14T12:14:50.835729+0000","last_became_active":"2024-03-14T12:14:48.796957+0000","last_became_peered":"2024-03-14T12:14:48.796957+0000","last_unstale":"2024-03-14T12:14:50.835729+0000","last_undegraded":"2024-03-14T12:14:50.835729+0000","last_fullsized":"2024-03-14T12:14:50.835729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:56:05.116234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794645+0000","last_change":"2024-03-14T12:14:49.806788+0000","last_active":"2024-03-14T12:14:50.794645+0000","last_peered":"2024-03-14T12:14:50.794645+0000","last_clean":"2024-03-14T12:14:50.794645+0000","last_became_active":"2024-03-14T12:14:49.806531+0000","last_became_peered":"2024-03-14T12:14:49.806531+0000","last_unstale":"2024-03-14T12:14:50.794645+0000","last_undegraded":"2024-03-14T12:14:50.794645+0000","last_fullsized":"2024-03-14T12:14:50.794645+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:33:04.523501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795859+0000","last_change":"2024-03-14T12:14:48.801872+0000","last_active":"2024-03-14T12:14:50.795859+0000","last_peered":"2024-03-14T12:14:50.795859+0000","last_clean":"2024-03-14T12:14:50.795859+0000","last_became_active":"2024-03-14T12:14:48.801607+0000","last_became_peered":"2024-03-14T12:14:48.801607+0000","last_unstale":"2024-03-14T12:14:50.795859+0000","last_undegraded":"2024-03-14T12:14:50.795859+0000","last_fullsized":"2024-03-14T12:14:50.795859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.882504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795556+0000","last_change":"2024-03-14T12:14:49.806533+0000","last_active":"2024-03-14T12:14:50.795556+0000","last_peered":"2024-03-14T12:14:50.795556+0000","last_clean":"2024-03-14T12:14:50.795556+0000","last_became_active":"2024-03-14T12:14:49.806219+0000","last_became_peered":"2024-03-14T12:14:49.806219+0000","last_unstale":"2024-03-14T12:14:50.795556+0000","last_undegraded":"2024-03-14T12:14:50.795556+0000","last_fullsized":"2024-03-14T12:14:50.795556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:54:17.524668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.794260+0000","last_change":"2024-03-14T12:14:48.796589+0000","last_active":"2024-03-14T12:14:50.794260+0000","last_peered":"2024-03-14T12:14:50.794260+0000","last_clean":"2024-03-14T12:14:50.794260+0000","last_became_active":"2024-03-14T12:14:48.796328+0000","last_became_peered":"2024-03-14T12:14:48.796328+0000","last_unstale":"2024-03-14T12:14:50.794260+0000","last_undegraded":"2024-03-14T12:14:50.794260+0000","last_fullsized":"2024-03-14T12:14:50.794260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:58:33.939734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795514+0000","last_change":"2024-03-14T12:14:49.802104+0000","last_active":"2024-03-14T12:14:50.795514+0000","last_peered":"2024-03-14T12:14:50.795514+0000","last_clean":"2024-03-14T12:14:50.795514+0000","last_became_active":"2024-03-14T12:14:49.801676+0000","last_became_peered":"2024-03-14T12:14:49.801676+0000","last_unstale":"2024-03-14T12:14:50.795514+0000","last_undegraded":"2024-03-14T12:14:50.795514+0000","last_fullsized":"2024-03-14T12:14:50.795514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:48.770586+0000","last_clean_scrub_stamp":"2024-03-14T12:14:48.770586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:58:12.693037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.889649+0000","last_change":"2024-03-14T12:14:48.800788+0000","last_active":"2024-03-14T12:14:50.889649+0000","last_peered":"2024-03-14T12:14:50.889649+0000","last_clean":"2024-03-14T12:14:50.889649+0000","last_became_active":"2024-03-14T12:14:48.800081+0000","last_became_peered":"2024-03-14T12:14:48.800081+0000","last_unstale":"2024-03-14T12:14:50.889649+0000","last_undegraded":"2024-03-14T12:14:50.889649+0000","last_fullsized":"2024-03-14T12:14:50.889649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:14:47.766209+0000","last_clean_scrub_stamp":"2024-03-14T12:14:47.766209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:10:23.765238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-14T12:14:50.795710+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:14:50.795710+0000","last_peered":"2024-03-14T12:14:50.795710+0000","last_clean":"2024-03-14T12:14:50.795710+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:14:50.795710+0000","last_undegraded":"2024-03-14T12:14:50.795710+0000","last_fullsized":"2024-03-14T12:14:50.795710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":741168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":152344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":150780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":152344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":741078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:14:57.423 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T12:14:57.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-14T12:14:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.567+0000 7ff5f922b640 1 Processor -- start 2024-03-14T12:14:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.567+0000 7ff5f922b640 1 -- start start 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f922b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff5f40cf180 0x7ff5f415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f922b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5f415e3a0 0x7ff5f40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f922b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f40b2f70 con 0x7ff5f41549d0 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f922b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f4154db0 con 0x7ff5f415e3a0 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f922b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f406d610 con 0x7ff5f40cf180 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f359e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5f415e3a0 0x7ff5f40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f2d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff5f41549d0 0x7ff5f406daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52644/0 (socket says 172.21.15.82:52644) 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f2d9d640 1 -- 172.21.15.82:0/3551730065 learned_addr learned my addr 172.21.15.82:0/3551730065 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5ebfff640 1 --2- 172.21.15.82:0/3551730065 >> v2:172.21.15.164:3300/0 conn(0x7ff5f40cf180 0x7ff5f415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f259c640 1 -- 172.21.15.82:0/3551730065 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88223408 0 0) 0x7ff5f40b2f70 con 0x7ff5f41549d0 2024-03-14T12:14:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f359e640 1 -- 172.21.15.82:0/3551730065 >> v2:172.21.15.164:3300/0 conn(0x7ff5f40cf180 msgr2=0x7ff5f415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f359e640 1 --2- 172.21.15.82:0/3551730065 >> v2:172.21.15.164:3300/0 conn(0x7ff5f40cf180 0x7ff5f415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f359e640 1 -- 172.21.15.82:0/3551730065 >> v1:172.21.15.82:6789/0 conn(0x7ff5f41549d0 legacy=0x7ff5f406daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.568+0000 7ff5f359e640 1 -- 172.21.15.82:0/3551730065 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40b29b0 con 0x7ff5f415e3a0 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.569+0000 7ff5f359e640 1 --2- 172.21.15.82:0/3551730065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f40707a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40d0470 tx=0x7ff5ec02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec41904457ca85c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.569+0000 7ff5f259c640 1 -- 172.21.15.82:0/3551730065 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec039070 con 0x7ff5f415e3a0 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.569+0000 7ff5f259c640 1 -- 172.21.15.82:0/3551730065 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec02cd00 con 0x7ff5f415e3a0 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.569+0000 7ff5f259c640 1 -- 172.21.15.82:0/3551730065 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec034660 con 0x7ff5f415e3a0 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 -- 172.21.15.82:0/3551730065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 msgr2=0x7ff5f40707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 --2- 172.21.15.82:0/3551730065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f40707a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40d0470 tx=0x7ff5ec02bf50 comp rx=0 tx=0).stop 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 -- 172.21.15.82:0/3551730065 shutdown_connections 2024-03-14T12:14:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 --2- 172.21.15.82:0/3551730065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f40707a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 --2- 172.21.15.82:0/3551730065 >> v2:172.21.15.164:3300/0 conn(0x7ff5f40cf180 0x7ff5f415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 -- 172.21.15.82:0/3551730065 >> 172.21.15.82:0/3551730065 conn(0x7ff5f406aef0 msgr2=0x7ff5f406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 -- 172.21.15.82:0/3551730065 shutdown_connections 2024-03-14T12:14:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.570+0000 7ff5f922b640 1 -- 172.21.15.82:0/3551730065 wait complete. 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.571+0000 7ff5f922b640 1 Processor -- start 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.571+0000 7ff5f922b640 1 -- start start 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5f40cf180 0x7ff5f41f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5f41549d0 0x7ff5f41568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f4070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f406d2a0 con 0x7ff5f415e3a0 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f406cfc0 con 0x7ff5f41549d0 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f415ec30 con 0x7ff5f40cf180 2024-03-14T12:14:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f4070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f4070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58848/0 (socket says 172.21.15.82:58848) 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 -- 172.21.15.82:0/2462631674 learned_addr learned my addr 172.21.15.82:0/2462631674 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f2d9d640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5f40cf180 0x7ff5f41f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5ebfff640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5f41549d0 0x7ff5f41568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 -- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5f40cf180 msgr2=0x7ff5f41f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5f40cf180 0x7ff5f41f1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 -- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5f41549d0 msgr2=0x7ff5f41568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5f41549d0 0x7ff5f41568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 -- 172.21.15.82:0/2462631674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f41f1b50 con 0x7ff5f415e3a0 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f359e640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f4070fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e80 tx=0x7ff5ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5eb7fe640 1 -- 172.21.15.82:0/2462631674 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec039050 con 0x7ff5f415e3a0 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5eb7fe640 1 -- 172.21.15.82:0/2462631674 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec045070 con 0x7ff5f415e3a0 2024-03-14T12:14:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f41f1e30 con 0x7ff5f415e3a0 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5eb7fe640 1 -- 172.21.15.82:0/2462631674 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec009450 con 0x7ff5f415e3a0 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.572+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f41f23f0 con 0x7ff5f415e3a0 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.573+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f404f050 con 0x7ff5f415e3a0 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.574+0000 7ff5eb7fe640 1 -- 172.21.15.82:0/2462631674 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff5ec03e850 con 0x7ff5f415e3a0 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.574+0000 7ff5eb7fe640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5d406bbd0 0x7ff5d406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.574+0000 7ff5f2d9d640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5d406bbd0 0x7ff5d406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.574+0000 7ff5eb7fe640 1 -- 172.21.15.82:0/2462631674 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5ec0ad840 con 0x7ff5f415e3a0 2024-03-14T12:14:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.575+0000 7ff5f2d9d640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5d406bbd0 0x7ff5d406e090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000f60 tx=0x7ff5e4007b50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.579+0000 7ff5eb7fe640 1 -- 172.21.15.82:0/2462631674 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff5ec032e20 con 0x7ff5f415e3a0 2024-03-14T12:14:57.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.822+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff5f406cad0 con 0x7ff5f415e3a0 2024-03-14T12:14:57.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.824+0000 7ff5eb7fe640 1 -- 172.21.15.82:0/2462631674 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff5ec07c4e0 con 0x7ff5f415e3a0 2024-03-14T12:14:57.823 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-03-14T12:14:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg== 2024-03-14T12:14:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-03-14T12:14:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-03-14T12:14:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-14T12:14:57.823 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-03-14T12:14:57.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5d406bbd0 msgr2=0x7ff5d406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5d406bbd0 0x7ff5d406e090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000f60 tx=0x7ff5e4007b50 comp rx=0 tx=0).stop 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 msgr2=0x7ff5f4070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f4070fa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e80 tx=0x7ff5ec004a40 comp rx=0 tx=0).stop 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 shutdown_connections 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f415e3a0 0x7ff5f4070fa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5f41549d0 0x7ff5f41568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5d406bbd0 0x7ff5d406e090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 --2- 172.21.15.82:0/2462631674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5f40cf180 0x7ff5f41f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 >> 172.21.15.82:0/2462631674 conn(0x7ff5f406aef0 msgr2=0x7ff5f41536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 shutdown_connections 2024-03-14T12:14:57.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.826+0000 7ff5f922b640 1 -- 172.21.15.82:0/2462631674 wait complete. 2024-03-14T12:14:57.836 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:14:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.937+0000 7f9c230c6640 1 Processor -- start 2024-03-14T12:14:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.937+0000 7f9c230c6640 1 -- start start 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c230c6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9c1c15df30 0x7f9c1c06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c230c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c06f7f0 0x7f9c1c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c230c6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c1c0b39c0 0x7f9c1c071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c230c6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c0d1590 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c230c6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c161b70 con 0x7f9c1c15df30 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c230c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c161e50 con 0x7f9c1c06f7f0 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c1c0b39c0 0x7f9c1c071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c1c0b39c0 0x7f9c1c071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58854/0 (socket says 172.21.15.82:58854) 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 -- 172.21.15.82:0/3168993399 learned_addr learned my addr 172.21.15.82:0/3168993399 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 -- 172.21.15.82:0/3168993399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c06f7f0 msgr2=0x7f9c1c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 --2- 172.21.15.82:0/3168993399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c06f7f0 0x7f9c1c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 -- 172.21.15.82:0/3168993399 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c15df30 msgr2=0x7f9c1c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c20e62640 1 --2- 172.21.15.82:0/3168993399 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c15df30 0x7f9c1c06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 --2- 172.21.15.82:0/3168993399 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c15df30 0x7f9c1c06ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c21663640 1 -- 172.21.15.82:0/3168993399 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c162130 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.938+0000 7f9c20e62640 1 --2- 172.21.15.82:0/3168993399 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c15df30 0x7f9c1c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.939+0000 7f9c21663640 1 --2- 172.21.15.82:0/3168993399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c071740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c04f910 tx=0x7f9c1402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2383ee60c71ef11f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.939+0000 7f9c1b7fe640 1 -- 172.21.15.82:0/3168993399 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1402ca70 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.939+0000 7f9c1b7fe640 1 -- 172.21.15.82:0/3168993399 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1402cc10 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.939+0000 7f9c1b7fe640 1 -- 172.21.15.82:0/3168993399 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c140343f0 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.939+0000 7f9c230c6640 1 -- 172.21.15.82:0/3168993399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 msgr2=0x7f9c1c071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.939+0000 7f9c230c6640 1 --2- 172.21.15.82:0/3168993399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c071740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c04f910 tx=0x7f9c1402bed0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.940+0000 7f9c230c6640 1 -- 172.21.15.82:0/3168993399 shutdown_connections 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.940+0000 7f9c230c6640 1 --2- 172.21.15.82:0/3168993399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c071740 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.940+0000 7f9c230c6640 1 --2- 172.21.15.82:0/3168993399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c06f7f0 0x7f9c1c06b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.940+0000 7f9c230c6640 1 --2- 172.21.15.82:0/3168993399 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c15df30 0x7f9c1c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.940+0000 7f9c230c6640 1 -- 172.21.15.82:0/3168993399 >> 172.21.15.82:0/3168993399 conn(0x7f9c1c0d09d0 msgr2=0x7f9c1c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:57.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.940+0000 7f9c230c6640 1 -- 172.21.15.82:0/3168993399 shutdown_connections 2024-03-14T12:14:57.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.940+0000 7f9c230c6640 1 -- 172.21.15.82:0/3168993399 wait complete. 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.941+0000 7f9c230c6640 1 Processor -- start 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.941+0000 7f9c230c6640 1 -- start start 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.941+0000 7f9c230c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c06f7f0 0x7f9c1c147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.941+0000 7f9c230c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c230c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15df30 0x7f9c1c159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c230c6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c06fe40 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c230c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c162860 con 0x7f9c1c06f7f0 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c230c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c06ffc0 con 0x7f9c1c15df30 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c0aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58860/0 (socket says 172.21.15.82:58860) 2024-03-14T12:14:57.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 -- 172.21.15.82:0/645302103 learned_addr learned my addr 172.21.15.82:0/645302103 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c21663640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15df30 0x7f9c1c159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c20e62640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c06f7f0 0x7f9c1c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 -- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15df30 msgr2=0x7f9c1c159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15df30 0x7f9c1c159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 -- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c06f7f0 msgr2=0x7f9c1c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c06f7f0 0x7f9c1c147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 -- 172.21.15.82:0/645302103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c15b6c0 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c21663640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15df30 0x7f9c1c159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c20e62640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c06f7f0 0x7f9c1c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c0aeb60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800b440 tx=0x7f9c0800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c197fa640 1 -- 172.21.15.82:0/645302103 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08015020 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c197fa640 1 -- 172.21.15.82:0/645302103 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08011890 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c15b960 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c197fa640 1 -- 172.21.15.82:0/645302103 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08011c80 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.942+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c15bf20 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.943+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1c04f050 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.944+0000 7f9c197fa640 1 -- 172.21.15.82:0/645302103 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9c08018020 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.944+0000 7f9c197fa640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0006bbd0 0x7f9c0006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.944+0000 7f9c197fa640 1 -- 172.21.15.82:0/645302103 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c0808ab50 con 0x7f9c1c0b39c0 2024-03-14T12:14:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.944+0000 7f9c20e62640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0006bbd0 0x7f9c0006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.945+0000 7f9c20e62640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0006bbd0 0x7f9c0006e090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9c10006fd0 tx=0x7f9c10008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:57.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:57.948+0000 7f9c197fa640 1 -- 172.21.15.82:0/645302103 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9c0801f020 con 0x7f9c1c0b39c0 2024-03-14T12:14:58.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.155+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c1c15a0c0 con 0x7f9c1c0b39c0 2024-03-14T12:14:58.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.155+0000 7f9c197fa640 1 -- 172.21.15.82:0/645302103 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f9c0800d040 con 0x7f9c1c0b39c0 2024-03-14T12:14:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:14:58.155 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2478084830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2478084830},{"type":"v1","addr":"172.21.15.82:6835","nonce":2478084830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4834,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1811632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1811632800},{"type":"v1","addr":"172.21.15.164:6834","nonce":1811632800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4176962755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4176962755},{"type":"v1","addr":"172.21.15.164:6835","nonce":4176962755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:14:49.135548+0000","modified":"2024-03-14T12:14:51.133632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/682556190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":682556190},{"type":"v1","addr":"172.21.15.82:6837","nonce":682556190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T12:14:58.155 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 15 2024-03-14T12:14:58.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0006bbd0 msgr2=0x7f9c0006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0006bbd0 0x7f9c0006e090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9c10006fd0 tx=0x7f9c10008040 comp rx=0 tx=0).stop 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 msgr2=0x7f9c1c0aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c0aeb60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800b440 tx=0x7f9c0800b900 comp rx=0 tx=0).stop 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 shutdown_connections 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0006bbd0 0x7f9c0006e090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15df30 0x7f9c1c159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c0b39c0 0x7f9c1c0aeb60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 --2- 172.21.15.82:0/645302103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c06f7f0 0x7f9c1c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 >> 172.21.15.82:0/645302103 conn(0x7f9c1c0d09d0 msgr2=0x7f9c1c0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 shutdown_connections 2024-03-14T12:14:58.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.157+0000 7f9c230c6640 1 -- 172.21.15.82:0/645302103 wait complete. 2024-03-14T12:14:58.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T12:14:49.135548+0000', 'modified': '2024-03-14T12:14:51.133632+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5061}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5061': {'gid': 5061, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/682556190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 682556190}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 682556190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T12:14:58.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T12:14:58.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-14T12:14:58.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.674+0000 7f1f29529640 1 Processor -- start 2024-03-14T12:14:58.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.674+0000 7f1f29529640 1 -- start start 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f29529640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f2415f1f0 0x7f1f2406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f29529640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1f240b58f0 0x7f1f2406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f29529640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f240b01d0 con 0x7f1f240af150 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f29529640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f240aff90 con 0x7f1f2415f1f0 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f29529640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2415e710 con 0x7f1f240b58f0 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f227fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f2415f1f0 0x7f1f2406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f227fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f2415f1f0 0x7f1f2406ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58862/0 (socket says 172.21.15.82:58862) 2024-03-14T12:14:58.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f227fc640 1 -- 172.21.15.82:0/532591340 learned_addr learned my addr 172.21.15.82:0/532591340 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f237fe640 1 --2- 172.21.15.82:0/532591340 >> v2:172.21.15.164:3300/0 conn(0x7f1f240b58f0 0x7f1f2406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.675+0000 7f1f227fc640 1 -- 172.21.15.82:0/532591340 >> v2:172.21.15.164:3300/0 conn(0x7f1f240b58f0 msgr2=0x7f1f2406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.676+0000 7f1f227fc640 1 --2- 172.21.15.82:0/532591340 >> v2:172.21.15.164:3300/0 conn(0x7f1f240b58f0 0x7f1f2406b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.676+0000 7f1f227fc640 1 -- 172.21.15.82:0/532591340 >> v1:172.21.15.164:6789/0 conn(0x7f1f240af150 legacy=0x7f1f240af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.676+0000 7f1f227fc640 1 -- 172.21.15.82:0/532591340 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2415e9f0 con 0x7f1f2415f1f0 2024-03-14T12:14:58.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.676+0000 7f1f237fe640 1 --2- 172.21.15.82:0/532591340 >> v2:172.21.15.164:3300/0 conn(0x7f1f240b58f0 0x7f1f2406b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:58.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.676+0000 7f1f227fc640 1 --2- 172.21.15.82:0/532591340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f2406ad10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1f240d1b00 tx=0x7f1f1802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7ac0d7582b57ca server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.676+0000 7f1f21ffb640 1 -- 172.21.15.82:0/532591340 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1802e070 con 0x7f1f2415f1f0 2024-03-14T12:14:58.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f21ffb640 1 -- 172.21.15.82:0/532591340 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f18037070 con 0x7f1f2415f1f0 2024-03-14T12:14:58.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f21ffb640 1 -- 172.21.15.82:0/532591340 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18031a00 con 0x7f1f2415f1f0 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 -- 172.21.15.82:0/532591340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 msgr2=0x7f1f2406ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 --2- 172.21.15.82:0/532591340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f2406ad10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1f240d1b00 tx=0x7f1f1802b880 comp rx=0 tx=0).stop 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 -- 172.21.15.82:0/532591340 shutdown_connections 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 --2- 172.21.15.82:0/532591340 >> v2:172.21.15.164:3300/0 conn(0x7f1f240b58f0 0x7f1f2406b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 --2- 172.21.15.82:0/532591340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f2406ad10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 -- 172.21.15.82:0/532591340 >> 172.21.15.82:0/532591340 conn(0x7f1f240d0a20 msgr2=0x7f1f240d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 -- 172.21.15.82:0/532591340 shutdown_connections 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.677+0000 7f1f29529640 1 -- 172.21.15.82:0/532591340 wait complete. 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.678+0000 7f1f29529640 1 Processor -- start 2024-03-14T12:14:58.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.678+0000 7f1f29529640 1 -- start start 2024-03-14T12:14:58.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f29529640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f240af150 0x7f1f24147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f29529640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f240b58f0 0x7f1f24155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f29529640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f24155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f29529640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f24157cd0 con 0x7f1f2415f1f0 2024-03-14T12:14:58.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f29529640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2415e400 con 0x7f1f240af150 2024-03-14T12:14:58.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f29529640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f24157e50 con 0x7f1f240b58f0 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f24155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f24155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58874/0 (socket says 172.21.15.82:58874) 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f237fe640 1 -- 172.21.15.82:0/3533089641 learned_addr learned my addr 172.21.15.82:0/3533089641 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f240af150 0x7f1f24147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f227fc640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f240b58f0 0x7f1f24155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f237fe640 1 -- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f240b58f0 msgr2=0x7f1f24155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.679+0000 7f1f237fe640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f240b58f0 0x7f1f24155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f237fe640 1 -- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f240af150 msgr2=0x7f1f24147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f237fe640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f240af150 0x7f1f24147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f237fe640 1 -- 172.21.15.82:0/3533089641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f24153270 con 0x7f1f2415f1f0 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f240af150 0x7f1f24147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f227fc640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f240b58f0 0x7f1f24155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f237fe640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f24155890 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c820 tx=0x7f1f1c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f0b7fe640 1 -- 172.21.15.82:0/3533089641 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c015070 con 0x7f1f2415f1f0 2024-03-14T12:14:58.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f0b7fe640 1 -- 172.21.15.82:0/3533089641 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c007a30 con 0x7f1f2415f1f0 2024-03-14T12:14:58.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f24153550 con 0x7f1f2415f1f0 2024-03-14T12:14:58.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f0b7fe640 1 -- 172.21.15.82:0/3533089641 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c007e20 con 0x7f1f2415f1f0 2024-03-14T12:14:58.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.680+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f24147f50 con 0x7f1f2415f1f0 2024-03-14T12:14:58.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.681+0000 7f1f0b7fe640 1 -- 172.21.15.82:0/3533089641 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1f1c018070 con 0x7f1f2415f1f0 2024-03-14T12:14:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.682+0000 7f1f0b7fe640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1f1006bbd0 0x7f1f1006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.682+0000 7f1f0b7fe640 1 -- 172.21.15.82:0/3533089641 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f1c08a980 con 0x7f1f2415f1f0 2024-03-14T12:14:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.682+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1f1006bbd0 0x7f1f1006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.682+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1f1006bbd0 0x7f1f1006e090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f14004520 tx=0x7f1f14009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.683+0000 7f1f2c1e8740 1 Processor -- start 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.683+0000 7f1f2c1e8740 1 -- start start 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560baba46110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba46650 0x560baba1e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f9c0 0x560baba1eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bab9837c0 con 0x560baba1f9c0 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bab92cb50 con 0x560baba46650 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bab97f860 con 0x560baba1f5e0 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f9c0 0x560baba1eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f9c0 0x560baba1eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58876/0 (socket says 172.21.15.82:58876) 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 -- 172.21.15.82:0/2318249525 learned_addr learned my addr 172.21.15.82:0/2318249525 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560baba46110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f227fc640 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba46650 0x560baba1e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 -- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 msgr2=0x560baba46110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560baba46110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 -- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba46650 msgr2=0x560baba1e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba46650 0x560baba1e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 -- 172.21.15.82:0/2318249525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bab9409a0 con 0x560baba1f9c0 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f227fc640 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba46650 0x560baba1e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f237fe640 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f9c0 0x560baba1eb60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c060fe0 tx=0x7f1f1c058f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d962b8b1db0fe8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f08ff9640 1 -- 172.21.15.82:0/2318249525 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c01f030 con 0x560baba1f9c0 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f08ff9640 1 -- 172.21.15.82:0/2318249525 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c00ce10 con 0x560baba1f9c0 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.684+0000 7f1f08ff9640 1 -- 172.21.15.82:0/2318249525 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c057100 con 0x560baba1f9c0 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2318249525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f9c0 msgr2=0x560baba1eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f9c0 0x560baba1eb60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c060fe0 tx=0x7f1f1c058f40 comp rx=0 tx=0).stop 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2318249525 shutdown_connections 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f9c0 0x560baba1eb60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba46650 0x560baba1e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2318249525 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560baba46110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2318249525 >> 172.21.15.82:0/2318249525 conn(0x560bab9e6400 msgr2=0x560baba64300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2318249525 shutdown_connections 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2318249525 wait complete. 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.685+0000 7f1f2c1e8740 1 -- start start 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 1 Processor -- start 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560baba74370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba63940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba4ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bab9837c0 con 0x560baba63e80 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bab93ff90 con 0x560baba1f9c0 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x560baba45c80 con 0x560baba1f5e0 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f2c1e8740 10 client.0 _ll_register_callbacks cb 0x560bab97b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:14:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba4ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba4ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58892/0 (socket says 172.21.15.82:58892) 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.686+0000 7f1f237fe640 1 -- 172.21.15.82:0/3672314112 learned_addr learned my addr 172.21.15.82:0/3672314112 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f237fe640 1 -- 172.21.15.82:0/3672314112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 msgr2=0x560baba74370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f237fe640 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560baba74370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f237fe640 1 -- 172.21.15.82:0/3672314112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 msgr2=0x560baba63940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f237fe640 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba63940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f237fe640 1 -- 172.21.15.82:0/3672314112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bab6982d0 con 0x560baba63e80 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f237fe640 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba4ce30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c088ee0 tx=0x7f1f1c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df00c08a58c1fcc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f01ffb640 1 -- 172.21.15.82:0/3672314112 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c074020 con 0x560baba63e80 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f01ffb640 1 -- 172.21.15.82:0/3672314112 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f1c058450 con 0x560baba63e80 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f1c00ce10 con 0x560baba63e80 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f01ffb640 1 -- 172.21.15.82:0/3672314112 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1c07f050 con 0x560baba63e80 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 10 client.5262 fetch_fsmap learned FSMap version 15 2024-03-14T12:14:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x560bab7626b0 con 0x560baba63e80 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f01ffb640 1 -- 172.21.15.82:0/3672314112 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c00ce10 con 0x560baba63e80 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f01ffb640 1 -- 172.21.15.82:0/3672314112 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f1c082020 con 0x560baba63e80 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 10 client.5262 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 10 client.5262 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 20 client.5262 populate_metadata read hostname 'smithi082' 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba61ce0 0x560baba620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 4 client.5262 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bab990d00 con 0x560baba61ce0 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.687+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba61ce0 0x560baba620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f01ffb640 10 client.5262 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba61ce0 0x560baba620c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b55b27141170d98 server_cookie=289baabb655fc54 in_seq=0 out_seq=0 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f01ffb640 1 -- 172.21.15.82:0/3672314112 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560bab990d00 con 0x560baba61ce0 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f01ffb640 10 client.5262 handle_command_reply: tid=0 2024-03-14T12:14:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 client.5262 shutdown 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 msgr2=0x560baba4ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba4ce30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c088ee0 tx=0x7f1f1c012000 comp rx=0 tx=0).stop 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 shutdown_connections 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba61ce0 0x560baba620c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba4ce30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba63940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3672314112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560baba74370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f01ffb640 0 client.5262 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.688+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 >> 172.21.15.82:0/3672314112 conn(0x560bab9e6400 msgr2=0x560baba472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.689+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 shutdown_connections 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.689+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3672314112 wait complete. 2024-03-14T12:14:58.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.689+0000 7f1f2c1e8740 20 client.5262 trim_cache size 0 max 16384 2024-03-14T12:14:58.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.704+0000 7f1f2c1e8740 1 Processor -- start 2024-03-14T12:14:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f2c1e8740 1 -- start start 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f5e0 0x560babac8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f9c0 0x560baba7d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba63e80 0x560baba7de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f24154c10 con 0x560baba1f5e0 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f24154010 con 0x560baba63e80 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f24154610 con 0x560baba1f9c0 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f5e0 0x560babac8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f5e0 0x560babac8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58902/0 (socket says 172.21.15.82:58902) 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 -- 172.21.15.82:0/3242464264 learned_addr learned my addr 172.21.15.82:0/3242464264 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 -- 172.21.15.82:0/3242464264 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f9c0 msgr2=0x560baba7d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3242464264 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f9c0 0x560baba7d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 -- 172.21.15.82:0/3242464264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba63e80 msgr2=0x560baba7de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3242464264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba63e80 0x560baba7de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.705+0000 7f1f22ffd640 1 -- 172.21.15.82:0/3242464264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560baba3d0d0 con 0x560baba1f5e0 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/3242464264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f5e0 0x560babac8eb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x560baba67b00 tx=0x7f1f14010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99e685eb7369aa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f03fff640 1 -- 172.21.15.82:0/3242464264 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14006d50 con 0x560baba1f5e0 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f03fff640 1 -- 172.21.15.82:0/3242464264 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14014870 con 0x560baba1f5e0 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f03fff640 1 -- 172.21.15.82:0/3242464264 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14014c20 con 0x560baba1f5e0 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3242464264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f5e0 msgr2=0x560babac8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3242464264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f5e0 0x560babac8eb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x560baba67b00 tx=0x7f1f14010000 comp rx=0 tx=0).stop 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3242464264 shutdown_connections 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3242464264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba63e80 0x560baba7de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3242464264 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f9c0 0x560baba7d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/3242464264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba1f5e0 0x560babac8eb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3242464264 >> 172.21.15.82:0/3242464264 conn(0x560bab9e6400 msgr2=0x560baba44f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3242464264 shutdown_connections 2024-03-14T12:14:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.706+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/3242464264 wait complete. 2024-03-14T12:14:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.707+0000 7f1f2c1e8740 1 -- start start 2024-03-14T12:14:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.707+0000 7f1f2c1e8740 1 Processor -- start 2024-03-14T12:14:58.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560babac4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba3f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba3f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bab9837c0 con 0x560baba63e80 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bab93ff90 con 0x560baba1f9c0 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x560baba77620 con 0x560baba1f5e0 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 10 client.0 _ll_register_callbacks cb 0x7f1f1c058bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba3f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba3f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58914/0 (socket says 172.21.15.82:58914) 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 -- 172.21.15.82:0/2946281742 learned_addr learned my addr 172.21.15.82:0/2946281742 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560babac4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f227fc640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba3f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 -- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 msgr2=0x560babac4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560babac4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 -- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 msgr2=0x560baba3f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba3f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 -- 172.21.15.82:0/2946281742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560babac4b50 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f227fc640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba3f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f237fe640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba3f740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c060fe0 tx=0x7f1f1c088e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771a9ca20bd9ef3e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f01ffb640 1 -- 172.21.15.82:0/2946281742 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c074070 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f01ffb640 1 -- 172.21.15.82:0/2946281742 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f1c054b50 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f14014870 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f01ffb640 1 -- 172.21.15.82:0/2946281742 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1c01f040 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 10 client.5268 fetch_fsmap learned FSMap version 15 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.708+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x560baba14e90 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f01ffb640 1 -- 172.21.15.82:0/2946281742 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14014870 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f01ffb640 1 -- 172.21.15.82:0/2946281742 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f1c082040 con 0x560baba63e80 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 10 client.5268 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:14:58.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 10 client.5268 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 20 client.5268 populate_metadata read hostname 'smithi082' 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba5a570 0x560baba18400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 4 client.5268 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560bab98b050 con 0x560baba5a570 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba5a570 0x560baba18400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f22ffd640 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba5a570 0x560baba18400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfaa286899e08d35 server_cookie=a309bc1e1c166f19 in_seq=0 out_seq=0 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f01ffb640 10 client.5268 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f01ffb640 1 -- 172.21.15.82:0/2946281742 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560bab98b050 con 0x560baba5a570 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f01ffb640 10 client.5268 handle_command_reply: tid=0 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 client.5268 shutdown 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 msgr2=0x560baba3f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba3f740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c060fe0 tx=0x7f1f1c088e10 comp rx=0 tx=0).stop 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 shutdown_connections 2024-03-14T12:14:58.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x560baba5a570 0x560baba18400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560baba63e80 0x560baba3f740 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f01ffb640 0 client.5268 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560baba1f9c0 0x560baba3f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 --2- 172.21.15.82:0/2946281742 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x560baba1f5e0 0x560babac4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 >> 172.21.15.82:0/2946281742 conn(0x560bab9e6400 msgr2=0x560baba44e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 shutdown_connections 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.709+0000 7f1f2c1e8740 1 -- 172.21.15.82:0/2946281742 wait complete. 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f2c1e8740 20 client.5268 trim_cache size 0 max 16384 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2", 2024-03-14T12:14:58.715 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 5061, 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 8.9179673580000003, 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 15, 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 34, 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 34, 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 16.858832653 2024-03-14T12:14:58.716 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:14:58.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1f1006bbd0 msgr2=0x7f1f1006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1f1006bbd0 0x7f1f1006e090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f14004520 tx=0x7f1f14009040 comp rx=0 tx=0).stop 2024-03-14T12:14:58.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 msgr2=0x7f1f24155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f24155890 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c820 tx=0x7f1f1c00cce0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 shutdown_connections 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f2415f1f0 0x7f1f24155890 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1f1006bbd0 0x7f1f1006e090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f240b58f0 0x7f1f24155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 --2- 172.21.15.82:0/3533089641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f240af150 0x7f1f24147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 >> 172.21.15.82:0/3533089641 conn(0x7f1f240d0a20 msgr2=0x7f1f24073e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 shutdown_connections 2024-03-14T12:14:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.710+0000 7f1f29529640 1 -- 172.21.15.82:0/3533089641 wait complete. 2024-03-14T12:14:58.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:14:58.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.822+0000 7f4a22b05640 1 Processor -- start 2024-03-14T12:14:58.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.822+0000 7f4a22b05640 1 -- start start 2024-03-14T12:14:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a22b05640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4a1c06e060 0x7f4a1c06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a22b05640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4a1c06e4a0 0x7f4a1c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a22b05640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0cf230 con 0x7f4a1c145720 2024-03-14T12:14:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a22b05640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cfe40 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a22b05640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cf510 con 0x7f4a1c06e060 2024-03-14T12:14:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a210a2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4a1c06e4a0 0x7f4a1c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a210a2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4a1c06e4a0 0x7f4a1c06b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58930/0 (socket says 172.21.15.82:58930) 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a210a2640 1 -- 172.21.15.82:0/2078516475 learned_addr learned my addr 172.21.15.82:0/2078516475 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a13fff640 1 --2- 172.21.15.82:0/2078516475 >> v2:172.21.15.164:3300/0 conn(0x7f4a1c06e060 0x7f4a1c06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a210a2640 1 -- 172.21.15.82:0/2078516475 >> v2:172.21.15.164:3300/0 conn(0x7f4a1c06e060 msgr2=0x7f4a1c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a210a2640 1 --2- 172.21.15.82:0/2078516475 >> v2:172.21.15.164:3300/0 conn(0x7f4a1c06e060 0x7f4a1c06ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a210a2640 1 -- 172.21.15.82:0/2078516475 >> v1:172.21.15.164:6789/0 conn(0x7f4a1c145720 legacy=0x7f4a1c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a210a2640 1 -- 172.21.15.82:0/2078516475 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0cf7f0 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.823+0000 7f4a13fff640 1 --2- 172.21.15.82:0/2078516475 >> v2:172.21.15.164:3300/0 conn(0x7f4a1c06e060 0x7f4a1c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.824+0000 7f4a210a2640 1 --2- 172.21.15.82:0/2078516475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c06b250 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c073520 tx=0x7f4a1402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b60575ab68e35a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.824+0000 7f4a137fe640 1 -- 172.21.15.82:0/2078516475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1402ca80 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.824+0000 7f4a137fe640 1 -- 172.21.15.82:0/2078516475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1402cc20 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.824+0000 7f4a137fe640 1 -- 172.21.15.82:0/2078516475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140343f0 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 -- 172.21.15.82:0/2078516475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 msgr2=0x7f4a1c06b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 --2- 172.21.15.82:0/2078516475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c06b250 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c073520 tx=0x7f4a1402bee0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 -- 172.21.15.82:0/2078516475 shutdown_connections 2024-03-14T12:14:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 --2- 172.21.15.82:0/2078516475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c06b250 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 --2- 172.21.15.82:0/2078516475 >> v2:172.21.15.164:3300/0 conn(0x7f4a1c06e060 0x7f4a1c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 -- 172.21.15.82:0/2078516475 >> 172.21.15.82:0/2078516475 conn(0x7f4a1c0d0a20 msgr2=0x7f4a1c0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 -- 172.21.15.82:0/2078516475 shutdown_connections 2024-03-14T12:14:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.825+0000 7f4a22b05640 1 -- 172.21.15.82:0/2078516475 wait complete. 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.826+0000 7f4a22b05640 1 Processor -- start 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.826+0000 7f4a22b05640 1 -- start start 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a1c06e060 0x7f4a1c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c1f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a1c145720 0x7f4a1c0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cef00 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c06f3b0 con 0x7f4a1c145720 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15c2a0 con 0x7f4a1c06e060 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c1f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c1f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58932/0 (socket says 172.21.15.82:58932) 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 -- 172.21.15.82:0/3249666230 learned_addr learned my addr 172.21.15.82:0/3249666230 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 -- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a1c06e060 msgr2=0x7f4a1c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a208a1640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a1c06e060 0x7f4a1c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a210a2640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a1c145720 0x7f4a1c0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a1c06e060 0x7f4a1c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 -- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a1c145720 msgr2=0x7f4a1c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a1c145720 0x7f4a1c0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 -- 172.21.15.82:0/3249666230 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0cb2f0 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a210a2640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a1c145720 0x7f4a1c0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a208a1640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a1c06e060 0x7f4a1c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a13fff640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c1f4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e7e0 tx=0x7f4a0c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a117fa640 1 -- 172.21.15.82:0/3249666230 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00f040 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a117fa640 1 -- 172.21.15.82:0/3249666230 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c004030 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0cb5d0 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a117fa640 1 -- 172.21.15.82:0/3249666230 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004400 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.827+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0cbb90 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.828+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c04f050 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.829+0000 7f4a117fa640 1 -- 172.21.15.82:0/3249666230 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4a0c00c6d0 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.829+0000 7f4a117fa640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a0806bbd0 0x7f4a0806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.829+0000 7f4a208a1640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a0806bbd0 0x7f4a0806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.829+0000 7f4a117fa640 1 -- 172.21.15.82:0/3249666230 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a0c08a7f0 con 0x7f4a1c06e4a0 2024-03-14T12:14:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.830+0000 7f4a208a1640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a0806bbd0 0x7f4a0806e090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a180015e0 tx=0x7f4a18009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:58.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:58.834+0000 7f4a117fa640 1 -- 172.21.15.82:0/3249666230 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4a0c017020 con 0x7f4a1c06e4a0 2024-03-14T12:14:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.044+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a1c06bc00 con 0x7f4a1c06e4a0 2024-03-14T12:14:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.044+0000 7f4a117fa640 1 -- 172.21.15.82:0/3249666230 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4a0c059370 con 0x7f4a1c06e4a0 2024-03-14T12:14:59.044 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a0806bbd0 msgr2=0x7f4a0806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a0806bbd0 0x7f4a0806e090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a180015e0 tx=0x7f4a18009040 comp rx=0 tx=0).stop 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 msgr2=0x7f4a1c1f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c1f4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e7e0 tx=0x7f4a0c00eca0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 shutdown_connections 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a1c145720 0x7f4a1c0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c06e4a0 0x7f4a1c1f4d90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4a0806bbd0 0x7f4a0806e090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 --2- 172.21.15.82:0/3249666230 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a1c06e060 0x7f4a1c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 >> 172.21.15.82:0/3249666230 conn(0x7f4a1c0d0a20 msgr2=0x7f4a1c0c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 shutdown_connections 2024-03-14T12:14:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.046+0000 7f4a22b05640 1 -- 172.21.15.82:0/3249666230 wait complete. 2024-03-14T12:14:59.056 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:14:59.057 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:14:59.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.057 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:14:59.072 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:14:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:14:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:14:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42621sec preferred_lft 42621sec 2024-03-14T12:14:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c443:69ff:fec0:3488/64 scope link 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::887e:60ff:fe47:4333/64 scope link 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:14:59.074 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a6:e3:19:81:dc:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:14:59.075 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a4e3:19ff:fe81:dc41/64 scope link 2024-03-14T12:14:59.075 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:14:59.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.075 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:14:59.138 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:14:59.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.140 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:14:59.196 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:14:59.196 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:14:59.197 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:14:59.197 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:14:59.197 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:14:59.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.198 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:14:59.198 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:14:59.198 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:14:59.198 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:14:59.199 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:14:59.199 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:14:59.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.293 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:14:59.307 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-14T12:14:59.307 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:14:59.308 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:14:59.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.308 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-14T12:14:59.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:59.375 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:59.430 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:14:59.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:14:59.430 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:59.484 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T12:14:59.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:59.568 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:59.568 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:14:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:14:59.576 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:14:59.576 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:14:59.576 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:14:59.576 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T12:14:59.576 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:14:59.577 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:14:59.577 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:14:59.577 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:14:59.578 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:14:59.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:14:59.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:14:59.605 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:14:59.664 INFO:teuthology.orchestra.run.smithi082.stdout:client4733 2024-03-14T12:14:59.665 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:14:59.665 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:14:59.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:14:59.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.862+0000 7ffaa5044640 1 Processor -- start 2024-03-14T12:14:59.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.862+0000 7ffaa5044640 1 -- start start 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffaa5044640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ffaa0151230 0x7ffaa00cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffaa5044640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00b3da0 con 0x7ffaa015dde0 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffaa5044640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa00b4080 con 0x7ffaa00b4b70 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffaa5044640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa0070d40 con 0x7ffaa0151230 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9f59e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffaa015dde0 0x7ffaa006cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52646/0 (socket says 172.21.15.82:52646) 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9f59e640 1 -- 172.21.15.82:0/3778171241 learned_addr learned my addr 172.21.15.82:0/3778171241 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9ed9d640 1 --2- 172.21.15.82:0/3778171241 >> v2:172.21.15.164:3300/0 conn(0x7ffaa0151230 0x7ffaa00cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989030847 0 0) 0x7ffaa00b3da0 con 0x7ffaa015dde0 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa8c004630 con 0x7ffaa015dde0 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440132832 0 0) 0x7ffa8c004630 con 0x7ffaa015dde0 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 >> v2:172.21.15.164:3300/0 conn(0x7ffaa0151230 msgr2=0x7ffaa00cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:59.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 --2- 172.21.15.82:0/3778171241 >> v2:172.21.15.164:3300/0 conn(0x7ffaa0151230 0x7ffaa00cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 >> v1:172.21.15.164:6789/0 conn(0x7ffaa00b4b70 legacy=0x7ffaa0070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00b4360 con 0x7ffaa015dde0 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7ffa980019f0 con 0x7ffaa015dde0 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 >> v1:172.21.15.82:6789/0 conn(0x7ffaa015dde0 legacy=0x7ffaa006cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa8c004ac0 0x7ffa8c004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffa8c005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa8c0062d0 0x7ffa8c00cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0070d40 con 0x7ffa8c005910 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c00d170 con 0x7ffa8c004ac0 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.863+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c00d370 con 0x7ffa8c0062d0 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa97fff640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffa8c005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa9ed9d640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa8c004ac0 0x7ffa8c004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa9f59e640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa8c0062d0 0x7ffa8c00cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa97fff640 1 -- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa8c0062d0 msgr2=0x7ffa8c00cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa97fff640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa8c0062d0 0x7ffa8c00cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa97fff640 1 -- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa8c004ac0 msgr2=0x7ffa8c004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa97fff640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa8c004ac0 0x7ffa8c004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa97fff640 1 -- 172.21.15.82:0/3778171241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c004630 con 0x7ffa8c005910 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.864+0000 7ffa9f59e640 1 -- 172.21.15.82:0/3778171241 reap_dead start 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffa97fff640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffa8c005d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffa88001750 tx=0x7ffa8c006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71d0ca1cbdf55d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88001370 con 0x7ffa8c005910 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa880052d0 con 0x7ffa8c005910 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffa9e59c640 1 -- 172.21.15.82:0/3778171241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88003830 con 0x7ffa8c005910 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffaa5044640 1 -- 172.21.15.82:0/3778171241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 msgr2=0x7ffa8c005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffaa5044640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffa8c005d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffa88001750 tx=0x7ffa8c006940 comp rx=0 tx=0).stop 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffaa5044640 1 -- 172.21.15.82:0/3778171241 shutdown_connections 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffaa5044640 1 --2- 172.21.15.82:0/3778171241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffa8c005d90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffaa5044640 1 -- 172.21.15.82:0/3778171241 >> 172.21.15.82:0/3778171241 conn(0x7ffaa00b1c70 msgr2=0x7ffaa00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:14:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffaa5044640 1 -- 172.21.15.82:0/3778171241 shutdown_connections 2024-03-14T12:14:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.865+0000 7ffaa5044640 1 -- 172.21.15.82:0/3778171241 wait complete. 2024-03-14T12:14:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 Processor -- start 2024-03-14T12:14:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 -- start start 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffaa006cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffaa0154260 0x7ffaa01546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaa01f3d70 0x7ffaa01f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa006c690 con 0x7ffa8c005910 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa006c3b0 con 0x7ffaa01f3d70 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffaa5044640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa015e210 con 0x7ffaa0154260 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffa9ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffaa006cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffa9ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffaa006cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58948/0 (socket says 172.21.15.82:58948) 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.867+0000 7ffa9ed9d640 1 -- 172.21.15.82:0/1710577384 learned_addr learned my addr 172.21.15.82:0/1710577384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa97fff640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffaa0154260 0x7ffaa01546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9ed9d640 1 -- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffaa0154260 msgr2=0x7ffaa01546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9f59e640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaa01f3d70 0x7ffaa01f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9ed9d640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffaa0154260 0x7ffaa01546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9ed9d640 1 -- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaa01f3d70 msgr2=0x7ffaa01f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9ed9d640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaa01f3d70 0x7ffaa01f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9ed9d640 1 -- 172.21.15.82:0/1710577384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa01f4ac0 con 0x7ffa8c005910 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa97fff640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffaa0154260 0x7ffaa01546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9f59e640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaa01f3d70 0x7ffaa01f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa9ed9d640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffaa006cc50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffa90011aa0 tx=0x7ffa90011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa977fe640 1 -- 172.21.15.82:0/1710577384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000fb80 con 0x7ffa8c005910 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa977fe640 1 -- 172.21.15.82:0/1710577384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9000fd20 con 0x7ffa8c005910 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa0151920 con 0x7ffa8c005910 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffa977fe640 1 -- 172.21.15.82:0/1710577384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90014660 con 0x7ffa8c005910 2024-03-14T12:14:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.868+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa0151ee0 con 0x7ffa8c005910 2024-03-14T12:14:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.869+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa004f050 con 0x7ffa8c005910 2024-03-14T12:14:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.869+0000 7ffa977fe640 1 -- 172.21.15.82:0/1710577384 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ffa90012040 con 0x7ffa8c005910 2024-03-14T12:14:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.870+0000 7ffa977fe640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa8006bc00 0x7ffa8006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:14:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.870+0000 7ffa977fe640 1 -- 172.21.15.82:0/1710577384 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa9008b140 con 0x7ffa8c005910 2024-03-14T12:14:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.870+0000 7ffa97fff640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa8006bc00 0x7ffa8006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:14:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.870+0000 7ffa97fff640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa8006bc00 0x7ffa8006e0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa88033000 tx=0x7ffa88009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:14:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:14:59.874+0000 7ffa977fe640 1 -- 172.21.15.82:0/1710577384 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ffa90019030 con 0x7ffa8c005910 2024-03-14T12:15:00.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.083+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffaa00b2f40 con 0x7ffa8c005910 2024-03-14T12:15:00.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.083+0000 7ffa977fe640 1 -- 172.21.15.82:0/1710577384 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffa900595e0 con 0x7ffa8c005910 2024-03-14T12:15:00.083 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa8006bc00 msgr2=0x7ffa8006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa8006bc00 0x7ffa8006e0c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa88033000 tx=0x7ffa88009030 comp rx=0 tx=0).stop 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 msgr2=0x7ffaa006cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffaa006cc50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffa90011aa0 tx=0x7ffa90011f60 comp rx=0 tx=0).stop 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 shutdown_connections 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffaa01f3d70 0x7ffaa01f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa8006bc00 0x7ffa8006e0c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffaa0154260 0x7ffaa01546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:00.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 --2- 172.21.15.82:0/1710577384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa8c005910 0x7ffaa006cc50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:00.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 >> 172.21.15.82:0/1710577384 conn(0x7ffaa00b1c70 msgr2=0x7ffaa015ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:00.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 shutdown_connections 2024-03-14T12:15:00.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:00.085+0000 7ffaa5044640 1 -- 172.21.15.82:0/1710577384 wait complete. 2024-03-14T12:15:00.095 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:15:00.095 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:15:00.096 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-14T12:15:00.096 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T12:15:00.096 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T12:15:00.096 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:15:00.096 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:15:00.096 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:15:00.096 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:15:00.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.096 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:15:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42656sec preferred_lft 42656sec 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff 2024-03-14T12:15:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::882:aaff:fed8:c2be/64 scope link 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::dc5a:2ff:fe9f:88ac/64 scope link 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether f6:de:82:8b:5d:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::f4de:82ff:fe8b:5dfb/64 scope link 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:15:00.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.118 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:15:00.185 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-14T12:15:00.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.187 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-14T12:15:00.245 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-14T12:15:00.245 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:15:00.245 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:15:00.245 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:15:00.245 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:15:00.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.245 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:15:00.246 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:15:00.246 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:15:00.246 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:15:00.246 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:15:00.246 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:15:00.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.345 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:15:00.361 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-14T12:15:00.361 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T12:15:00.361 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:15:00.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-03-14T12:15:00.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:15:00.431 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T12:15:00.485 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:15:00.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:15:00.486 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:15:00.541 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T12:15:00.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:15:00.625 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:15:00.626 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:15:00.632 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:15:00.632 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with new device syntax: 1@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:15:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:15:00.635 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:15:00.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:15:00.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:00.662 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:15:00.718 INFO:teuthology.orchestra.run.smithi164.stdout:client4978 2024-03-14T12:15:00.718 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:00.718 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:10.774 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:15:10.775 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-14T12:15:10.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.889+0000 7fd3eb69a640 1 Processor -- start 2024-03-14T12:15:10.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3eb69a640 1 -- start start 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3eb69a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e40d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3eb69a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3e415e200 0x7fd3e40d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3eb69a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3e40d28e0 0x7fd3e40b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3eb69a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e40d2d50 con 0x7fd3e415e200 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3eb69a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e40cfd80 con 0x7fd3e40d28e0 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3eb69a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40cece0 con 0x7fd3e415de20 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3e8c35640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3e415e200 0x7fd3e40d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3e8c35640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3e415e200 0x7fd3e40d23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46634/0 (socket says 172.21.15.82:46634) 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.890+0000 7fd3e8c35640 1 -- 172.21.15.82:0/3463428894 learned_addr learned my addr 172.21.15.82:0/3463428894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/3463428894 >> v2:172.21.15.164:3300/0 conn(0x7fd3e40d28e0 0x7fd3e40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e9436640 1 --2- 172.21.15.82:0/3463428894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e40d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e8c35640 1 -- 172.21.15.82:0/3463428894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 msgr2=0x7fd3e40d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/3463428894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e40d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e8c35640 1 -- 172.21.15.82:0/3463428894 >> v2:172.21.15.164:3300/0 conn(0x7fd3e40d28e0 msgr2=0x7fd3e40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/3463428894 >> v2:172.21.15.164:3300/0 conn(0x7fd3e40d28e0 0x7fd3e40b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e8c35640 1 -- 172.21.15.82:0/3463428894 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e40cefc0 con 0x7fd3e415e200 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/3463428894 >> v2:172.21.15.164:3300/0 conn(0x7fd3e40d28e0 0x7fd3e40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.891+0000 7fd3e9436640 1 --2- 172.21.15.82:0/3463428894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e40d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/3463428894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e40d23a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd3e408d160 tx=0x7fd3d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe865c58541cd8f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3dbfff640 1 -- 172.21.15.82:0/3463428894 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d402e070 con 0x7fd3e415e200 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3dbfff640 1 -- 172.21.15.82:0/3463428894 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d4037070 con 0x7fd3e415e200 2024-03-14T12:15:10.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3dbfff640 1 -- 172.21.15.82:0/3463428894 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4031a00 con 0x7fd3e415e200 2024-03-14T12:15:10.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 -- 172.21.15.82:0/3463428894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 msgr2=0x7fd3e40d23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/3463428894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e40d23a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd3e408d160 tx=0x7fd3d402b880 comp rx=0 tx=0).stop 2024-03-14T12:15:10.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 -- 172.21.15.82:0/3463428894 shutdown_connections 2024-03-14T12:15:10.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/3463428894 >> v2:172.21.15.164:3300/0 conn(0x7fd3e40d28e0 0x7fd3e40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/3463428894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e40d23a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/3463428894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e40d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 -- 172.21.15.82:0/3463428894 >> 172.21.15.82:0/3463428894 conn(0x7fd3e409d030 msgr2=0x7fd3e40d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 -- 172.21.15.82:0/3463428894 shutdown_connections 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.892+0000 7fd3eb69a640 1 -- 172.21.15.82:0/3463428894 wait complete. 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.893+0000 7fd3eb69a640 1 Processor -- start 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3eb69a640 1 -- start start 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3eb69a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e40d28e0 0x7fd3e414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3eb69a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3eb69a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3eb69a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40d12f0 con 0x7fd3e415e200 2024-03-14T12:15:10.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3eb69a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40cf580 con 0x7fd3e40d28e0 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3eb69a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40ce940 con 0x7fd3e415de20 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3e9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3e9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e409daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46646/0 (socket says 172.21.15.82:46646) 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.894+0000 7fd3e9c37640 1 -- 172.21.15.82:0/2182654606 learned_addr learned my addr 172.21.15.82:0/2182654606 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9436640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e40d28e0 0x7fd3e414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9c37640 1 -- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 msgr2=0x7fd3e415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9c37640 1 -- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e40d28e0 msgr2=0x7fd3e414f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e40d28e0 0x7fd3e414f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9c37640 1 -- 172.21.15.82:0/2182654606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e40cd850 con 0x7fd3e415e200 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9436640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e40d28e0 0x7fd3e414f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e409daa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c820 tx=0x7fd3dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:10.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3d9ffb640 1 -- 172.21.15.82:0/2182654606 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc015070 con 0x7fd3e415e200 2024-03-14T12:15:10.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3d9ffb640 1 -- 172.21.15.82:0/2182654606 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc007a30 con 0x7fd3e415e200 2024-03-14T12:15:10.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e409fbc0 con 0x7fd3e415e200 2024-03-14T12:15:10.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3d9ffb640 1 -- 172.21.15.82:0/2182654606 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc007e20 con 0x7fd3e415e200 2024-03-14T12:15:10.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.895+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e40a0180 con 0x7fd3e415e200 2024-03-14T12:15:10.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.897+0000 7fd3d9ffb640 1 -- 172.21.15.82:0/2182654606 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd3dc018070 con 0x7fd3e415e200 2024-03-14T12:15:10.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.897+0000 7fd3d9ffb640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3c806bbd0 0x7fd3c806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.897+0000 7fd3d9ffb640 1 -- 172.21.15.82:0/2182654606 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3dc08a970 con 0x7fd3e415e200 2024-03-14T12:15:10.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.897+0000 7fd3e9436640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3c806bbd0 0x7fd3c806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.898+0000 7fd3e9436640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3c806bbd0 0x7fd3c806e090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000d430 tx=0x7fd3e0007530 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:10.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.898+0000 7fd3ee27e740 1 Processor -- start 2024-03-14T12:15:10.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.898+0000 7fd3ee27e740 1 -- start start 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292f21a60 0x562292ed9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292ed95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed4210 0x562292f03d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562292ea1360 con 0x562292ed7ae0 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562292e3dec0 con 0x562292ed4210 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x562292de6d30 con 0x562292f21a60 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292ed95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292ed95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46656/0 (socket says 172.21.15.82:46656) 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 -- 172.21.15.82:0/1110870279 learned_addr learned my addr 172.21.15.82:0/1110870279 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e9436640 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292f21a60 0x562292ed9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed4210 0x562292f03d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 -- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292f21a60 msgr2=0x562292ed9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292f21a60 0x562292ed9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 -- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed4210 msgr2=0x562292f03d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed4210 0x562292f03d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 -- 172.21.15.82:0/1110870279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562292de5bc0 con 0x562292ed7ae0 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.899+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292ed95d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40090b0 tx=0x7fd3d4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6259cf05e2e75537 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ceffd640 1 -- 172.21.15.82:0/1110870279 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4031a90 con 0x562292ed7ae0 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ceffd640 1 -- 172.21.15.82:0/1110870279 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d4037040 con 0x562292ed7ae0 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ceffd640 1 -- 172.21.15.82:0/1110870279 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4031e50 con 0x562292ed7ae0 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 -- 172.21.15.82:0/1110870279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 msgr2=0x562292ed95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292ed95d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40090b0 tx=0x7fd3d4030000 comp rx=0 tx=0).stop 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 -- 172.21.15.82:0/1110870279 shutdown_connections 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed4210 0x562292f03d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292ed95d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/1110870279 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292f21a60 0x562292ed9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 -- 172.21.15.82:0/1110870279 >> 172.21.15.82:0/1110870279 conn(0x562292ed6480 msgr2=0x562292f1ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 -- 172.21.15.82:0/1110870279 shutdown_connections 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.900+0000 7fd3ee27e740 1 -- 172.21.15.82:0/1110870279 wait complete. 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.901+0000 7fd3ee27e740 1 -- start start 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.901+0000 7fd3ee27e740 1 Processor -- start 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292ed7ae0 0x562292f26e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 0x562292f02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292efc190 0x562292f02f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562292ddc4a0 con 0x562292efc190 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562292e39f60 con 0x562292f21a60 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x562292ed7ff0 con 0x562292ed7ae0 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3ee27e740 10 client.0 _ll_register_callbacks cb 0x562292e37d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292efc190 0x562292f02f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292efc190 0x562292f02f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46664/0 (socket says 172.21.15.82:46664) 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 -- 172.21.15.82:0/4159128219 learned_addr learned my addr 172.21.15.82:0/4159128219 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9436640 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292ed7ae0 0x562292f26e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 0x562292f02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 -- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292ed7ae0 msgr2=0x562292f26e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292ed7ae0 0x562292f26e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 -- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 msgr2=0x562292f02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 0x562292f02a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 -- 172.21.15.82:0/4159128219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562292b523f0 con 0x562292efc190 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292efc190 0x562292f02f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc058b20 tx=0x7fd3dc058f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20675fb143d9e0b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.902+0000 7fd3c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3c3fff640 1 -- 172.21.15.82:0/4159128219 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc087060 con 0x562292efc190 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3c3fff640 1 -- 172.21.15.82:0/4159128219 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3dc0888b0 con 0x562292efc190 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3d4037040 con 0x562292efc190 2024-03-14T12:15:10.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3c3fff640 1 -- 172.21.15.82:0/4159128219 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3dc01f040 con 0x562292efc190 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 10 client.5292 fetch_fsmap learned FSMap version 15 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x562292c1c7d0 con 0x562292efc190 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3c3fff640 1 -- 172.21.15.82:0/4159128219 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc088ab0 con 0x562292efc190 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3c3fff640 1 -- 172.21.15.82:0/4159128219 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3dc058270 con 0x562292efc190 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 10 client.5292 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 10 client.5292 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 20 client.5292 populate_metadata read hostname 'smithi082' 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f2f830 0x562292f2fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 4 client.5292 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.903+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3d4037040 con 0x562292f2f830 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.904+0000 7fd3e9436640 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f2f830 0x562292f2fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.904+0000 7fd3c3fff640 10 client.5292 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.904+0000 7fd3e9436640 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f2f830 0x562292f2fc40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad75e592f891549c server_cookie=ed770d33102eaf75 in_seq=0 out_seq=0 2024-03-14T12:15:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.904+0000 7fd3c3fff640 1 -- 172.21.15.82:0/4159128219 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fd3d4037040 con 0x562292f2f830 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.904+0000 7fd3c3fff640 10 client.5292 handle_command_reply: tid=0 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.904+0000 7fd3ee27e740 1 client.5292 shutdown 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292efc190 msgr2=0x562292f02f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292efc190 0x562292f02f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc058b20 tx=0x7fd3dc058f60 comp rx=0 tx=0).stop 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 shutdown_connections 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f2f830 0x562292f2fc40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292efc190 0x562292f02f60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 0x562292f02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3c3fff640 0 client.5292 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/4159128219 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292ed7ae0 0x562292f26e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 >> 172.21.15.82:0/4159128219 conn(0x562292ed4210 msgr2=0x562292ed50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 shutdown_connections 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 1 -- 172.21.15.82:0/4159128219 wait complete. 2024-03-14T12:15:10.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.905+0000 7fd3ee27e740 20 client.5292 trim_cache size 0 max 16384 2024-03-14T12:15:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.916+0000 7fd3ee27e740 1 Processor -- start 2024-03-14T12:15:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.916+0000 7fd3ee27e740 1 -- start start 2024-03-14T12:15:10.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292efa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 0x562292ed9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 0x562292f1ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e41ffde0 con 0x562292ed7ae0 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e41f7c70 con 0x562292f21a60 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40cd570 con 0x562292efc190 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292efa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292efa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46672/0 (socket says 172.21.15.82:46672) 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 -- 172.21.15.82:0/3605007565 learned_addr learned my addr 172.21.15.82:0/3605007565 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 -- 172.21.15.82:0/3605007565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 msgr2=0x562292ed9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 --2- 172.21.15.82:0/3605007565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 0x562292ed9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 -- 172.21.15.82:0/3605007565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 msgr2=0x562292f1ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 --2- 172.21.15.82:0/3605007565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 0x562292f1ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.917+0000 7fd3e9436640 1 -- 172.21.15.82:0/3605007565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562292efcdf0 con 0x562292ed7ae0 2024-03-14T12:15:10.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3e9436640 1 --2- 172.21.15.82:0/3605007565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292efa920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000dcc0 tx=0x7fd3e0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe0b38912d45922 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ce7fc640 1 -- 172.21.15.82:0/3605007565 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0017b00 con 0x562292ed7ae0 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ce7fc640 1 -- 172.21.15.82:0/3605007565 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e0017ca0 con 0x562292ed7ae0 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ce7fc640 1 -- 172.21.15.82:0/3605007565 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0025250 con 0x562292ed7ae0 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 -- 172.21.15.82:0/3605007565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 msgr2=0x562292efa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/3605007565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292efa920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000dcc0 tx=0x7fd3e0010000 comp rx=0 tx=0).stop 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 -- 172.21.15.82:0/3605007565 shutdown_connections 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/3605007565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292f21a60 0x562292f1ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/3605007565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 0x562292ed9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/3605007565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292ed7ae0 0x562292efa920 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 -- 172.21.15.82:0/3605007565 >> 172.21.15.82:0/3605007565 conn(0x562292ed4210 msgr2=0x562292ed3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 -- 172.21.15.82:0/3605007565 shutdown_connections 2024-03-14T12:15:10.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.918+0000 7fd3ee27e740 1 -- 172.21.15.82:0/3605007565 wait complete. 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.919+0000 7fd3ee27e740 1 -- start start 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.919+0000 7fd3ee27e740 1 Processor -- start 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed7ae0 0x562292f70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 0x562292efa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3ee27e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292f21a60 0x562292f38bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e41f7670 con 0x562292f21a60 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562292efa8a0 con 0x562292ed7ae0 2024-03-14T12:15:10.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3ee27e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x562292f8a620 con 0x562292efc190 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3ee27e740 10 client.0 _ll_register_callbacks cb 0x7fd3d40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292f21a60 0x562292f38bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292f21a60 0x562292f38bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46676/0 (socket says 172.21.15.82:46676) 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 -- 172.21.15.82:0/287172815 learned_addr learned my addr 172.21.15.82:0/287172815 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e8c35640 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 0x562292efa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9436640 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed7ae0 0x562292f70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 -- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 msgr2=0x562292efa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 0x562292efa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 -- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed7ae0 msgr2=0x562292f70350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed7ae0 0x562292f70350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 -- 172.21.15.82:0/287172815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562292f8a870 con 0x562292f21a60 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3e9c37640 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292f21a60 0x562292f38bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc007410 tx=0x7fd3dc058d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1584b0399e88b4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.920+0000 7fd3c3fff640 1 -- 172.21.15.82:0/287172815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc087040 con 0x562292f21a60 2024-03-14T12:15:10.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3c3fff640 1 -- 172.21.15.82:0/287172815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3dc054d80 con 0x562292f21a60 2024-03-14T12:15:10.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3e0017ca0 con 0x562292f21a60 2024-03-14T12:15:10.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3c3fff640 1 -- 172.21.15.82:0/287172815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3dc082050 con 0x562292f21a60 2024-03-14T12:15:10.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 10 client.5298 fetch_fsmap learned FSMap version 15 2024-03-14T12:15:10.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x562292f39130 con 0x562292f21a60 2024-03-14T12:15:10.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3c3fff640 1 -- 172.21.15.82:0/287172815 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e0017ca0 con 0x562292f21a60 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3c3fff640 1 -- 172.21.15.82:0/287172815 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3dc077040 con 0x562292f21a60 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 10 client.5298 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 10 client.5298 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 20 client.5298 populate_metadata read hostname 'smithi082' 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f1ffe0 0x562292f203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 4 client.5298 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x562292e4b400 con 0x562292f1ffe0 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3e9436640 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f1ffe0 0x562292f203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3c3fff640 10 client.5298 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.921+0000 7fd3e9436640 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f1ffe0 0x562292f203f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb3010e8b48eeb8d server_cookie=b93c1536ca3c1218 in_seq=0 out_seq=0 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3c3fff640 1 -- 172.21.15.82:0/287172815 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26750 (crc 0 0 0) 0x562292e4b400 con 0x562292f1ffe0 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3c3fff640 10 client.5298 handle_command_reply: tid=0 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 client.5298 shutdown 2024-03-14T12:15:10.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292f21a60 msgr2=0x562292f38bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292f21a60 0x562292f38bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc007410 tx=0x7fd3dc058d20 comp rx=0 tx=0).stop 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 shutdown_connections 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x562292f1ffe0 0x562292f203f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562292f21a60 0x562292f38bf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x562292efc190 0x562292efa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3c3fff640 0 client.5298 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 --2- 172.21.15.82:0/287172815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562292ed7ae0 0x562292f70350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 >> 172.21.15.82:0/287172815 conn(0x562292ed4210 msgr2=0x562292ed3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 shutdown_connections 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 1 -- 172.21.15.82:0/287172815 wait complete. 2024-03-14T12:15:10.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.922+0000 7fd3ee27e740 20 client.5298 trim_cache size 0 max 16384 2024-03-14T12:15:10.929 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:15:10.929 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:15:10.929 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.929 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:10.929 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:15:10.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.929 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.930 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.931 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 30, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 29, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 7965, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 33877, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 9, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.005170909, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000452120, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002731586, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000554574, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 28, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.107559169, 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003841398 2024-03-14T12:15:10.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000221, 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000110 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.933 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 43, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 40, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 12472, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 127827, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.006727635, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001960234, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.004143868, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000690503, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 39, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.014505610, 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000371938 2024-03-14T12:15:10.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 59, 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000008395, 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000142 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 94336 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.935 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 86, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 71, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 121429, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 25558, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.009478607, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001711067, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.009542360, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000946430, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 71, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.059848517, 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000842936 2024-03-14T12:15:10.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17, 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000003091, 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000181 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 113328, 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 13296 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.937 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.938 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 44, 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002825580, 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000064217 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:15:10.939 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000016539, 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000008269 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.940 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 25, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 25, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 25, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002965121, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000118604 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:15:10.941 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 10, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:15:10.942 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 49, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 49, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:15:10.943 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:15:10.944 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 1 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.945 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:15:10.946 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "5061" 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 2 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:15:10.948 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4733", 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 5, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 18, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:15:10.949 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4978", 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 23, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:15:10.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:15:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 6, 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:15:10.952 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 6, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 2, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4196743, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:15:10.953 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005494824, 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001831608 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:15:10.954 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 2, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 2, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 43368, 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.955 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 25, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 25, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 2, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:15:10.956 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 24, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002709692, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000112903 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.958 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:15:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.961 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.962 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:15:10.963 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.964 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000255429, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000255429 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:15:10.965 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:15:10.966 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.967 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.967 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.967 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.967 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:15:10.967 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.967 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.968 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.968 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.968 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:15:10.968 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.968 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.968 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.969 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.969 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:15:10.969 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:15:10.969 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:15:10.969 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.970 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.970 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.970 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.970 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.970 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.971 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.971 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:15:10.971 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.971 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.971 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.971 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-03-14T12:15:10.972 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-03-14T12:15:10.972 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2024-03-14T12:15:10.972 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-03-14T12:15:10.972 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:15:10.972 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 24, 2024-03-14T12:15:10.972 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 12, 2024-03-14T12:15:10.973 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 6, 2024-03-14T12:15:10.973 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:15:10.973 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.973 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.973 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.973 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:15:10.974 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.974 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.974 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.974 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:15:10.974 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:15:10.975 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:15:10.975 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:15:10.975 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:15:10.975 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:15:10.975 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:15:10.976 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:15:10.976 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:15:10.976 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:15:10.976 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:15:10.976 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:15:10.976 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:15:10.977 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:15:10.977 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:15:10.977 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:15:10.977 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:15:10.977 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:15:10.977 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:15:10.978 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:15:10.978 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:15:10.978 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:15:10.978 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:15:10.978 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:15:10.979 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:15:10.979 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:15:10.979 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:15:10.979 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:15:10.979 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:15:10.979 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:15:10.980 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:15:10.980 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:15:10.980 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:15:10.980 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:15:10.980 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:15:10.980 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 224571, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 87, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:15:10.981 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 52240, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 213, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:15:10.982 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 41, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 13559, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 41, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 41, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.396992598, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.009682746 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 41, 2024-03-14T12:15:10.983 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 65 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 41, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 41, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 3, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 7, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:15:10.984 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:15:10.985 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:15:10.986 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 34, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 15, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.987 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.988 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.989 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 151, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 123007, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 151, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 151, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 123007, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.990 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 41, 2024-03-14T12:15:10.991 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 13559, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 41, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 13559, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:15:10.992 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 41, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 41, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 41, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 41, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.993 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:10.994 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x56140c7886e0": [ 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:10.995 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 6, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2439, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 6, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2439, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:10.996 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.997 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.997 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:10.997 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:15:10.997 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:15:10.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3c806bbd0 msgr2=0x7fd3c806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3c806bbd0 0x7fd3c806e090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000d430 tx=0x7fd3e0007530 comp rx=0 tx=0).stop 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 msgr2=0x7fd3e409daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e409daa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c820 tx=0x7fd3dc00cce0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 shutdown_connections 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e415e200 0x7fd3e409daa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3c806bbd0 0x7fd3c806e090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3e415de20 0x7fd3e415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 --2- 172.21.15.82:0/2182654606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3e40d28e0 0x7fd3e414f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 >> 172.21.15.82:0/2182654606 conn(0x7fd3e409d030 msgr2=0x7fd3e40b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 shutdown_connections 2024-03-14T12:15:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:10.923+0000 7fd3eb69a640 1 -- 172.21.15.82:0/2182654606 wait complete. 2024-03-14T12:15:11.000 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:11.000 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:11.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:11.016 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:15:11.072 INFO:teuthology.orchestra.run.smithi082.stdout:client4733 2024-03-14T12:15:11.072 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:11.072 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:11.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:11.092 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:15:11.151 INFO:teuthology.orchestra.run.smithi164.stdout:client4978 2024-03-14T12:15:11.151 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:11.151 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:11.167 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-03-14T12:15:11.167 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> import uuid 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> n = 1000 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> create_hard_links = False 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> if 0 > 0: 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> create_hard_links = True 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> path = "/home/ubuntu/cephtest/mnt.0/test_dir/test_file" 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> 2024-03-14T12:15:11.168 DEBUG:teuthology.orchestra.run.smithi082:> dpath = os.path.dirname(path) 2024-03-14T12:15:11.169 DEBUG:teuthology.orchestra.run.smithi082:> fnameprefix = os.path.basename(path) 2024-03-14T12:15:11.169 DEBUG:teuthology.orchestra.run.smithi082:> os.makedirs(dpath, exist_ok=True) 2024-03-14T12:15:11.169 DEBUG:teuthology.orchestra.run.smithi082:> 2024-03-14T12:15:11.169 DEBUG:teuthology.orchestra.run.smithi082:> try: 2024-03-14T12:15:11.169 DEBUG:teuthology.orchestra.run.smithi082:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-03-14T12:15:11.169 DEBUG:teuthology.orchestra.run.smithi082:> 2024-03-14T12:15:11.170 DEBUG:teuthology.orchestra.run.smithi082:> for i in range(n): 2024-03-14T12:15:11.170 DEBUG:teuthology.orchestra.run.smithi082:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-03-14T12:15:11.170 DEBUG:teuthology.orchestra.run.smithi082:> with open(fpath, '"'"'w'"'"') as f: 2024-03-14T12:15:11.170 DEBUG:teuthology.orchestra.run.smithi082:> f.write(f"{i}") 2024-03-14T12:15:11.170 DEBUG:teuthology.orchestra.run.smithi082:> if True: 2024-03-14T12:15:11.170 DEBUG:teuthology.orchestra.run.smithi082:> f.flush() 2024-03-14T12:15:11.170 DEBUG:teuthology.orchestra.run.smithi082:> os.fsync(f.fileno()) 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> if False: 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> os.unlink(fpath) 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> if False: 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> os.fsync(dirfd) 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> if create_hard_links: 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> for j in range(0): 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> if False: 2024-03-14T12:15:11.171 DEBUG:teuthology.orchestra.run.smithi082:> os.fsync(dirfd) 2024-03-14T12:15:11.172 DEBUG:teuthology.orchestra.run.smithi082:> finally: 2024-03-14T12:15:11.172 DEBUG:teuthology.orchestra.run.smithi082:> os.close(dirfd) 2024-03-14T12:15:11.172 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-03-14T12:15:14.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:14.860 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:15:14.860 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-14T12:15:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.003+0000 7f2a37012640 1 Processor -- start 2024-03-14T12:15:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.003+0000 7f2a37012640 1 -- start start 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a37012640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a300706c0 0x7f2a3006dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a37012640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a3006c440 0x7f2a30070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a37012640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2a3015ace0 0x7f2a3015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a37012640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3006e320 con 0x7f2a3006c440 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a37012640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a30070fe0 con 0x7f2a3015ace0 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a37012640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015b2b0 con 0x7f2a300706c0 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a3006c440 0x7f2a30070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a3006c440 0x7f2a30070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56824/0 (socket says 172.21.15.82:56824) 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 -- 172.21.15.82:0/1497508459 learned_addr learned my addr 172.21.15.82:0/1497508459 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a34dae640 1 --2- 172.21.15.82:0/1497508459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a300706c0 0x7f2a3006dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a355af640 1 --2- 172.21.15.82:0/1497508459 >> v2:172.21.15.164:3300/0 conn(0x7f2a3015ace0 0x7f2a3015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 -- 172.21.15.82:0/1497508459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a300706c0 msgr2=0x7f2a3006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1497508459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a300706c0 0x7f2a3006dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 -- 172.21.15.82:0/1497508459 >> v2:172.21.15.164:3300/0 conn(0x7f2a3015ace0 msgr2=0x7f2a3015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1497508459 >> v2:172.21.15.164:3300/0 conn(0x7f2a3015ace0 0x7f2a3015e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a2ffff640 1 -- 172.21.15.82:0/1497508459 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3015b590 con 0x7f2a3006c440 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a34dae640 1 --2- 172.21.15.82:0/1497508459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a300706c0 0x7f2a3006dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.004+0000 7f2a355af640 1 --2- 172.21.15.82:0/1497508459 >> v2:172.21.15.164:3300/0 conn(0x7f2a3015ace0 0x7f2a3015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1497508459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3006c440 0x7f2a30070aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2a30070650 tx=0x7f2a1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52b9b24af018eb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a2f7fe640 1 -- 172.21.15.82:0/1497508459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c035020 con 0x7f2a3006c440 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a2f7fe640 1 -- 172.21.15.82:0/1497508459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c031870 con 0x7f2a3006c440 2024-03-14T12:15:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a2f7fe640 1 -- 172.21.15.82:0/1497508459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c031c40 con 0x7f2a3006c440 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 -- 172.21.15.82:0/1497508459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3006c440 msgr2=0x7f2a30070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 --2- 172.21.15.82:0/1497508459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3006c440 0x7f2a30070aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2a30070650 tx=0x7f2a1c02b910 comp rx=0 tx=0).stop 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 -- 172.21.15.82:0/1497508459 shutdown_connections 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 --2- 172.21.15.82:0/1497508459 >> v2:172.21.15.164:3300/0 conn(0x7f2a3015ace0 0x7f2a3015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 --2- 172.21.15.82:0/1497508459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3006c440 0x7f2a30070aa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 --2- 172.21.15.82:0/1497508459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a300706c0 0x7f2a3006dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 -- 172.21.15.82:0/1497508459 >> 172.21.15.82:0/1497508459 conn(0x7f2a300cf910 msgr2=0x7f2a300cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 -- 172.21.15.82:0/1497508459 shutdown_connections 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.005+0000 7f2a37012640 1 -- 172.21.15.82:0/1497508459 wait complete. 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 Processor -- start 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 -- start start 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a3006c440 0x7f2a300b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a300706c0 0x7f2a300b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3015ace0 0x7f2a300b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015be30 con 0x7f2a3015ace0 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015bb50 con 0x7f2a300706c0 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.007+0000 7f2a37012640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015ebd0 con 0x7f2a3006c440 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3015ace0 0x7f2a300b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3015ace0 0x7f2a300b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56840/0 (socket says 172.21.15.82:56840) 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 -- 172.21.15.82:0/828213021 learned_addr learned my addr 172.21.15.82:0/828213021 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a34dae640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a3006c440 0x7f2a300b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a300706c0 0x7f2a300b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 -- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a3006c440 msgr2=0x7f2a300b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a3006c440 0x7f2a300b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 -- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a300706c0 msgr2=0x7f2a300b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a300706c0 0x7f2a300b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 -- 172.21.15.82:0/828213021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30154ef0 con 0x7f2a3015ace0 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a34dae640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a3006c440 0x7f2a300b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a355af640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3015ace0 0x7f2a300b2c70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2a28007e90 tx=0x7f2a2800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a2d7fa640 1 -- 172.21.15.82:0/828213021 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2801a070 con 0x7f2a3015ace0 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a2d7fa640 1 -- 172.21.15.82:0/828213021 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a28003990 con 0x7f2a3015ace0 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a301551d0 con 0x7f2a3015ace0 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a2d7fa640 1 -- 172.21.15.82:0/828213021 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28003d60 con 0x7f2a3015ace0 2024-03-14T12:15:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.008+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a30155790 con 0x7f2a3015ace0 2024-03-14T12:15:15.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.010+0000 7f2a2d7fa640 1 -- 172.21.15.82:0/828213021 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2a2801d070 con 0x7f2a3015ace0 2024-03-14T12:15:15.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.010+0000 7f2a2d7fa640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a1406bbd0 0x7f2a1406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.010+0000 7f2a2d7fa640 1 -- 172.21.15.82:0/828213021 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a2808ab40 con 0x7f2a3015ace0 2024-03-14T12:15:15.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.010+0000 7f2a34dae640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a1406bbd0 0x7f2a1406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.011+0000 7f2a34dae640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a1406bbd0 0x7f2a1406e090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006fd0 tx=0x7f2a24008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.011+0000 7f2a39bf9740 1 Processor -- start 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.011+0000 7f2a39bf9740 1 -- start start 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586495525c0 0x558649571050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586495707b0 0x55864952aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558649554d70 0x5586495551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586494f2360 con 0x5586495525c0 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864948eec0 con 0x558649554d70 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x558649437d30 con 0x5586495707b0 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586495525c0 0x558649571050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586495525c0 0x558649571050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56850/0 (socket says 172.21.15.82:56850) 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 -- 172.21.15.82:0/3958831034 learned_addr learned my addr 172.21.15.82:0/3958831034 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586495707b0 0x55864952aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a355af640 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558649554d70 0x5586495551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 -- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586495707b0 msgr2=0x55864952aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586495707b0 0x55864952aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 -- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558649554d70 msgr2=0x5586495551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558649554d70 0x5586495551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 -- 172.21.15.82:0/3958831034 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558649436bc0 con 0x5586495525c0 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a34dae640 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586495525c0 0x558649571050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400fa60 tx=0x7f2a24012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d639bdf3f8e830c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a1a7fc640 1 -- 172.21.15.82:0/3958831034 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2401b050 con 0x5586495525c0 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a1a7fc640 1 -- 172.21.15.82:0/3958831034 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2401c040 con 0x5586495525c0 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.012+0000 7f2a1a7fc640 1 -- 172.21.15.82:0/3958831034 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24014c00 con 0x5586495525c0 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 -- 172.21.15.82:0/3958831034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586495525c0 msgr2=0x558649571050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586495525c0 0x558649571050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400fa60 tx=0x7f2a24012000 comp rx=0 tx=0).stop 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 -- 172.21.15.82:0/3958831034 shutdown_connections 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558649554d70 0x5586495551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586495707b0 0x55864952aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/3958831034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586495525c0 0x558649571050 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 -- 172.21.15.82:0/3958831034 >> 172.21.15.82:0/3958831034 conn(0x558649527480 msgr2=0x55864956fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 -- 172.21.15.82:0/3958831034 shutdown_connections 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.013+0000 7f2a39bf9740 1 -- 172.21.15.82:0/3958831034 wait complete. 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.014+0000 7f2a39bf9740 1 -- start start 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.014+0000 7f2a39bf9740 1 Processor -- start 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x55864954f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649567220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864942d4a0 con 0x558649554d70 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864948af60 con 0x5586495707b0 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586495529a0 con 0x558649571a70 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x55864954f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 10 client.0 _ll_register_callbacks cb 0x558649488d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x55864954f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56852/0 (socket says 172.21.15.82:56852) 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 -- 172.21.15.82:0/1443171903 learned_addr learned my addr 172.21.15.82:0/1443171903 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649567220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a355af640 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 -- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 msgr2=0x5586495402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 -- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 msgr2=0x558649567220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649567220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 -- 172.21.15.82:0/1443171903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586491a33f0 con 0x558649554d70 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a34dae640 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x55864954f050 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55864952c5e0 tx=0x7f2a24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94228ace7f6ed0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/1443171903 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2402e030 con 0x558649554d70 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/1443171903 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a24025070 con 0x558649554d70 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a2401c040 con 0x558649554d70 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/1443171903 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a2401b020 con 0x558649554d70 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.015+0000 7f2a39bf9740 10 client.5310 fetch_fsmap learned FSMap version 15 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55864926d7d0 con 0x558649554d70 2024-03-14T12:15:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/1443171903 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2401c040 con 0x558649554d70 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/1443171903 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2a2402a660 con 0x558649554d70 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a39bf9740 10 client.5310 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a39bf9740 10 client.5310 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a39bf9740 20 client.5310 populate_metadata read hostname 'smithi082' 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495299c0 0x558649529dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a39bf9740 4 client.5310 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55864949c400 con 0x5586495299c0 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495299c0 0x558649529dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495299c0 0x558649529dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d35fa85b0fa4d4 server_cookie=3b5580f7e7b6efd9 in_seq=0 out_seq=0 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.016+0000 7f2a0f7fe640 10 client.5310 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/1443171903 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55864949c400 con 0x5586495299c0 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a0f7fe640 10 client.5310 handle_command_reply: tid=0 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 client.5310 shutdown 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 msgr2=0x55864954f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x55864954f050 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55864952c5e0 tx=0x7f2a24004730 comp rx=0 tx=0).stop 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 shutdown_connections 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495299c0 0x558649529dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649567220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a0f7fe640 0 client.5310 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1443171903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x55864954f050 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 >> 172.21.15.82:0/1443171903 conn(0x5586495525c0 msgr2=0x5586495256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 shutdown_connections 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1443171903 wait complete. 2024-03-14T12:15:15.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.017+0000 7f2a39bf9740 20 client.5310 trim_cache size 0 max 16384 2024-03-14T12:15:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 Processor -- start 2024-03-14T12:15:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 -- start start 2024-03-14T12:15:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649554d70 0x558649529da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649555340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649571a70 0x55864954ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a301679f0 con 0x558649571a70 2024-03-14T12:15:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a30166c30 con 0x5586495707b0 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a30156c00 con 0x558649554d70 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a355af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649571a70 0x55864954ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.029+0000 7f2a355af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649571a70 0x55864954ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56866/0 (socket says 172.21.15.82:56866) 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a355af640 1 -- 172.21.15.82:0/1035770607 learned_addr learned my addr 172.21.15.82:0/1035770607 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a34dae640 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649554d70 0x558649529da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649555340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a355af640 1 -- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649554d70 msgr2=0x558649529da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a355af640 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649554d70 0x558649529da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a355af640 1 -- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 msgr2=0x558649555340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a355af640 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649555340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a355af640 1 -- 172.21.15.82:0/1035770607 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55864956eaf0 con 0x558649571a70 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649555340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a355af640 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649571a70 0x55864954ff40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2a28058cc0 tx=0x7f2a28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84efa2790da4658 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a19ffb640 1 -- 172.21.15.82:0/1035770607 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28055630 con 0x558649571a70 2024-03-14T12:15:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a19ffb640 1 -- 172.21.15.82:0/1035770607 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a28087070 con 0x558649571a70 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.030+0000 7f2a19ffb640 1 -- 172.21.15.82:0/1035770607 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280557d0 con 0x558649571a70 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1035770607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649571a70 msgr2=0x55864954ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649571a70 0x55864954ff40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2a28058cc0 tx=0x7f2a28004a40 comp rx=0 tx=0).stop 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1035770607 shutdown_connections 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649571a70 0x55864954ff40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x558649555340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/1035770607 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649554d70 0x558649529da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1035770607 >> 172.21.15.82:0/1035770607 conn(0x5586495525c0 msgr2=0x558649567030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1035770607 shutdown_connections 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 -- 172.21.15.82:0/1035770607 wait complete. 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 -- start start 2024-03-14T12:15:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.031+0000 7f2a39bf9740 1 Processor -- start 2024-03-14T12:15:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x558649558780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x55864954d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a39bf9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3014d8c0 con 0x558649554d70 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586495df9c0 con 0x5586495707b0 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a39bf9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x558649524190 con 0x558649571a70 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a39bf9740 10 client.0 _ll_register_callbacks cb 0x7f2a24009140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x558649558780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x558649558780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56878/0 (socket says 172.21.15.82:56878) 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 -- 172.21.15.82:0/2052058447 learned_addr learned my addr 172.21.15.82:0/2052058447 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x55864954d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a355af640 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 -- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 msgr2=0x5586495dedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495dedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 -- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 msgr2=0x55864954d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x55864954d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.032+0000 7f2a34dae640 1 -- 172.21.15.82:0/2052058447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586495243e0 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a34dae640 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x558649558780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400afd0 tx=0x7f2a24015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d4051b6de7368d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/2052058447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2402e040 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/2052058447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a24020440 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a28087070 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/2052058447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a24048050 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 10 client.5316 fetch_fsmap learned FSMap version 15 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5586495df760 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/2052058447 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a28087070 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/2052058447 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2a2404b020 con 0x558649554d70 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 10 client.5316 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 10 client.5316 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 20 client.5316 populate_metadata read hostname 'smithi082' 2024-03-14T12:15:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495df2f0 0x55864952a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 4 client.5316 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x558649496750 con 0x5586495df2f0 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.033+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495df2f0 0x55864952a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a0f7fe640 10 client.5316 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a2ffff640 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495df2f0 0x55864952a4b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ce743a472b58bc server_cookie=bcf5e42472d56f11 in_seq=0 out_seq=0 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a0f7fe640 1 -- 172.21.15.82:0/2052058447 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26922 (crc 0 0 0) 0x558649496750 con 0x5586495df2f0 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a0f7fe640 10 client.5316 handle_command_reply: tid=0 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 client.5316 shutdown 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 msgr2=0x558649558780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x558649558780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400afd0 tx=0x7f2a24015000 comp rx=0 tx=0).stop 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 shutdown_connections 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586495df2f0 0x55864952a4b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558649571a70 0x5586495dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586495707b0 0x55864954d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a0f7fe640 0 client.5316 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 --2- 172.21.15.82:0/2052058447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558649554d70 0x558649558780 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 >> 172.21.15.82:0/2052058447 conn(0x5586495525c0 msgr2=0x55864956e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 shutdown_connections 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.034+0000 7f2a39bf9740 1 -- 172.21.15.82:0/2052058447 wait complete. 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a39bf9740 20 client.5316 trim_cache size 0 max 16384 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.040 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:15.041 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.042 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1038, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1037, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 255917, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 4657032, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 11, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.068261045, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.028150497, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.015592103, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.018924402, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1036, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.123014713, 2024-03-14T12:15:15.043 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000118740 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000417, 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000104 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.044 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 3051, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 4045, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 795227, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 1912002, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.234539249, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.157574210, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.110045292, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000793576, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4044, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.055539848, 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000013733 2024-03-14T12:15:15.045 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6402, 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000764358, 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000119 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 96128 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.046 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 94, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 75, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 124215, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 27522, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.009984188, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001898682, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.009722768, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.001070592, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 75, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.059911152, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000798815 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.047 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 26, 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000004178, 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000160 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 113600, 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 13920 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:15:15.048 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2051, 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.402402752, 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000196198 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.049 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000016539, 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000008269 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.050 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 1031, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 1031, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1031, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.551109444, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000534538 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:15:15.051 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 1011, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 1011, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 1011, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1002, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1003, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 1061, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 1060, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:15:15.052 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 836, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 2395, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:15:15.053 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 1, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 1000, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 1001, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 1000, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 3 2024-03-14T12:15:15.054 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:15:15.055 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:15:15.056 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "5061" 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 2 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:15:15.057 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4733", 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 8912, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 19, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 889, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 2, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 890, 2024-03-14T12:15:15.058 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 890, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 890, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 888, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 2554 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4978", 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 41, 2024-03-14T12:15:15.059 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.060 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 2008, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 2008, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 3, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:15:15.061 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 3, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 8387435, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1004, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.450794506, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001445014 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.062 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1014, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 1014, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 13, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 13, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 1011, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 1011, 2024-03-14T12:15:15.063 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1003, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1003, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 55400, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 1032, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:15:15.064 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 1031, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 1002, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1000, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.544801480, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000544801 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 29, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003286110, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000113314 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.065 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.066 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002766425, 2024-03-14T12:15:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002766425 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.071 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000255429, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000255429 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.072 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.073 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.074 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 1007, 2024-03-14T12:15:15.075 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 503, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 11, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:15:15.076 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:15:15.077 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:15:15.078 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 256162, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 257, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 2024, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 23, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:15:15.079 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 539005, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 5858, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:15:15.080 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 1047, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 4270619, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 1047, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1047, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.859965340, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001776471 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1047, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1075 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:15.081 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 1047, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 1047, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 1004, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 8, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-03-14T12:15:15.082 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:15:15.083 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:15:15.084 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 34, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 23, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.086 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:15:15.087 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 4175, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 794288, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 4175, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 4175, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 794288, 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:15.088 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:15.089 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 1047, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 4270619, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1047, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 4270619, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.090 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 1047, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 1047, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1047, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1047, 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:15.091 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:15.092 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:15.093 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x56140c7886e0": [ 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2008, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 4193131, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2008, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1004, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 4193131, 2024-03-14T12:15:15.094 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:15:15.095 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a1406bbd0 msgr2=0x7f2a1406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a1406bbd0 0x7f2a1406e090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006fd0 tx=0x7f2a24008040 comp rx=0 tx=0).stop 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3015ace0 msgr2=0x7f2a300b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3015ace0 0x7f2a300b2c70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2a28007e90 tx=0x7f2a2800d4a0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 shutdown_connections 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3015ace0 0x7f2a300b2c70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a300706c0 0x7f2a300b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a1406bbd0 0x7f2a1406e090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 --2- 172.21.15.82:0/828213021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a3006c440 0x7f2a300b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 >> 172.21.15.82:0/828213021 conn(0x7f2a300cf910 msgr2=0x7f2a3006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 shutdown_connections 2024-03-14T12:15:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:15.035+0000 7f2a37012640 1 -- 172.21.15.82:0/828213021 wait complete. 2024-03-14T12:15:15.098 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:15.098 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:15.114 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:15.114 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:15.114 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:15.114 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:15.114 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:15.114 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.114 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_0", '"'"'r'"'"') as f: 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:15.115 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:15.156 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_0 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_0 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627777 Links: 1 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.281295081 +0000 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.284295026 +0000 2024-03-14T12:15:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.284295026 +0000 2024-03-14T12:15:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.281295081 +0000 2024-03-14T12:15:15.175 INFO:tasks.cephfs.mount:File test_dir/test_file_0 became visible from 1 after 0s 2024-03-14T12:15:15.175 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:15.175 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:15.222 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:15.229 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:15.229 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:15.229 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:15.229 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_1", '"'"'r'"'"') as f: 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:15.230 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:15.271 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_1 2024-03-14T12:15:15.287 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_1 2024-03-14T12:15:15.287 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:15.288 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627778 Links: 1 2024-03-14T12:15:15.288 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:15.288 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:15.288 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.288294951 +0000 2024-03-14T12:15:15.288 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.288294951 +0000 2024-03-14T12:15:15.288 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.288294951 +0000 2024-03-14T12:15:15.288 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.288294951 +0000 2024-03-14T12:15:15.288 INFO:tasks.cephfs.mount:File test_dir/test_file_1 became visible from 1 after 0s 2024-03-14T12:15:15.288 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:15.288 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:15.336 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:15.343 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:15.343 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_2", '"'"'r'"'"') as f: 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:15.344 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:15.385 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_2 2024-03-14T12:15:15.402 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_2 2024-03-14T12:15:15.402 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:15.402 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627779 Links: 1 2024-03-14T12:15:15.403 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:15.403 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:15.403 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.296294802 +0000 2024-03-14T12:15:15.403 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.297294784 +0000 2024-03-14T12:15:15.403 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.297294784 +0000 2024-03-14T12:15:15.403 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.296294802 +0000 2024-03-14T12:15:15.403 INFO:tasks.cephfs.mount:File test_dir/test_file_2 became visible from 1 after 0s 2024-03-14T12:15:15.403 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:15.403 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:15.450 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_3", '"'"'r'"'"') as f: 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:15.458 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:15.459 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:15.459 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:15.459 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:15.459 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:15.459 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:15.500 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_3 2024-03-14T12:15:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_3 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627780 Links: 1 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.300294728 +0000 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.300294728 +0000 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.300294728 +0000 2024-03-14T12:15:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.300294728 +0000 2024-03-14T12:15:15.517 INFO:tasks.cephfs.mount:File test_dir/test_file_3 became visible from 1 after 0s 2024-03-14T12:15:15.517 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:15.517 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:15.565 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:15.572 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:15.572 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_4", '"'"'r'"'"') as f: 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:15.573 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:15.614 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_4 2024-03-14T12:15:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_4 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627781 Links: 1 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.306294616 +0000 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.306294616 +0000 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.306294616 +0000 2024-03-14T12:15:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.306294616 +0000 2024-03-14T12:15:15.632 INFO:tasks.cephfs.mount:File test_dir/test_file_4 became visible from 1 after 0s 2024-03-14T12:15:15.633 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:15.633 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:15.687 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_5", '"'"'r'"'"') as f: 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:15.688 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:15.689 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:15.689 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:15.729 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_5 2024-03-14T12:15:16.061 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.097 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.097 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_5 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627782 Links: 1 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.313294486 +0000 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.313294486 +0000 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.313294486 +0000 2024-03-14T12:15:16.098 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.313294486 +0000 2024-03-14T12:15:16.098 INFO:tasks.cephfs.mount:File test_dir/test_file_5 became visible from 1 after 0s 2024-03-14T12:15:16.098 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.099 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.114 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.114 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_6", '"'"'r'"'"') as f: 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.115 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.156 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_6 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_6 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627783 Links: 1 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.320294356 +0000 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.320294356 +0000 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.320294356 +0000 2024-03-14T12:15:16.172 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.320294356 +0000 2024-03-14T12:15:16.172 INFO:tasks.cephfs.mount:File test_dir/test_file_6 became visible from 1 after 0s 2024-03-14T12:15:16.172 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.173 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.220 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.227 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.227 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.227 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_7", '"'"'r'"'"') as f: 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.228 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.269 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_7 2024-03-14T12:15:16.284 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_7 2024-03-14T12:15:16.284 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.284 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627784 Links: 1 2024-03-14T12:15:16.284 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.284 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.284 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.324294282 +0000 2024-03-14T12:15:16.284 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.324294282 +0000 2024-03-14T12:15:16.285 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.324294282 +0000 2024-03-14T12:15:16.285 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.324294282 +0000 2024-03-14T12:15:16.285 INFO:tasks.cephfs.mount:File test_dir/test_file_7 became visible from 1 after 0s 2024-03-14T12:15:16.285 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.285 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.332 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.339 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.339 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.339 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.339 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.339 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_8", '"'"'r'"'"') as f: 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.340 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.381 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_8 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_8 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627785 Links: 1 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.330294170 +0000 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.330294170 +0000 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.330294170 +0000 2024-03-14T12:15:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.330294170 +0000 2024-03-14T12:15:16.397 INFO:tasks.cephfs.mount:File test_dir/test_file_8 became visible from 1 after 0s 2024-03-14T12:15:16.397 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.398 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.444 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.452 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.452 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.452 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_9", '"'"'r'"'"') as f: 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.453 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.494 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_9 2024-03-14T12:15:16.509 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_9 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627786 Links: 1 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.333294114 +0000 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.333294114 +0000 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.333294114 +0000 2024-03-14T12:15:16.510 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.333294114 +0000 2024-03-14T12:15:16.510 INFO:tasks.cephfs.mount:File test_dir/test_file_9 became visible from 1 after 0s 2024-03-14T12:15:16.510 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.511 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.558 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.565 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_10", '"'"'r'"'"') as f: 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.566 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.607 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_10 2024-03-14T12:15:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_10 2024-03-14T12:15:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.622 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627787 Links: 1 2024-03-14T12:15:16.623 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.623 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.623 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.337294040 +0000 2024-03-14T12:15:16.623 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.337294040 +0000 2024-03-14T12:15:16.623 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.337294040 +0000 2024-03-14T12:15:16.623 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.337294040 +0000 2024-03-14T12:15:16.623 INFO:tasks.cephfs.mount:File test_dir/test_file_10 became visible from 1 after 0s 2024-03-14T12:15:16.623 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.623 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.670 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.677 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.677 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.677 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.677 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.677 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_11", '"'"'r'"'"') as f: 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.678 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.719 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_11 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_11 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627788 Links: 1 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.340293984 +0000 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.340293984 +0000 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.340293984 +0000 2024-03-14T12:15:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.340293984 +0000 2024-03-14T12:15:16.736 INFO:tasks.cephfs.mount:File test_dir/test_file_11 became visible from 1 after 0s 2024-03-14T12:15:16.736 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.736 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.783 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.795 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_12", '"'"'r'"'"') as f: 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.796 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.838 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_12 2024-03-14T12:15:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_12 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627789 Links: 1 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.346293872 +0000 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.346293872 +0000 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.346293872 +0000 2024-03-14T12:15:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.346293872 +0000 2024-03-14T12:15:16.854 INFO:tasks.cephfs.mount:File test_dir/test_file_12 became visible from 1 after 0s 2024-03-14T12:15:16.854 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.854 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:16.902 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:16.909 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_13", '"'"'r'"'"') as f: 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:16.910 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:16.951 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_13 2024-03-14T12:15:16.966 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_13 2024-03-14T12:15:16.966 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:16.966 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627790 Links: 1 2024-03-14T12:15:16.966 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:16.966 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:16.967 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.353293742 +0000 2024-03-14T12:15:16.967 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.353293742 +0000 2024-03-14T12:15:16.967 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.353293742 +0000 2024-03-14T12:15:16.967 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.353293742 +0000 2024-03-14T12:15:16.967 INFO:tasks.cephfs.mount:File test_dir/test_file_13 became visible from 1 after 0s 2024-03-14T12:15:16.967 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:16.967 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.014 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.021 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.021 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.021 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.021 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_14", '"'"'r'"'"') as f: 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.022 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.023 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.023 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.064 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_14 2024-03-14T12:15:17.079 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_14 2024-03-14T12:15:17.079 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.079 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627791 Links: 1 2024-03-14T12:15:17.079 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.079 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.080 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.358293649 +0000 2024-03-14T12:15:17.080 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.358293649 +0000 2024-03-14T12:15:17.080 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.358293649 +0000 2024-03-14T12:15:17.080 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.358293649 +0000 2024-03-14T12:15:17.080 INFO:tasks.cephfs.mount:File test_dir/test_file_14 became visible from 1 after 0s 2024-03-14T12:15:17.080 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.080 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.126 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.134 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_15", '"'"'r'"'"') as f: 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.135 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.176 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_15 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_15 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627792 Links: 1 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.361293593 +0000 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.361293593 +0000 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.361293593 +0000 2024-03-14T12:15:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.361293593 +0000 2024-03-14T12:15:17.192 INFO:tasks.cephfs.mount:File test_dir/test_file_15 became visible from 1 after 0s 2024-03-14T12:15:17.192 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.192 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.224 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.238 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_16", '"'"'r'"'"') as f: 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.239 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.240 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.240 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.242 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_16 2024-03-14T12:15:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_16 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627793 Links: 1 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.365293519 +0000 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.365293519 +0000 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.365293519 +0000 2024-03-14T12:15:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.365293519 +0000 2024-03-14T12:15:17.257 INFO:tasks.cephfs.mount:File test_dir/test_file_16 became visible from 1 after 0s 2024-03-14T12:15:17.257 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.257 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.305 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.312 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.312 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.312 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.312 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_17", '"'"'r'"'"') as f: 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.313 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.338 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.340 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_17 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_17 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627794 Links: 1 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.370293426 +0000 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.371293407 +0000 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.371293407 +0000 2024-03-14T12:15:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.370293426 +0000 2024-03-14T12:15:17.355 INFO:tasks.cephfs.mount:File test_dir/test_file_17 became visible from 1 after 0s 2024-03-14T12:15:17.356 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.356 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.403 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.410 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_18", '"'"'r'"'"') as f: 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.411 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.453 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_18 2024-03-14T12:15:17.453 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_18 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627795 Links: 1 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.374293351 +0000 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.374293351 +0000 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.374293351 +0000 2024-03-14T12:15:17.468 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.374293351 +0000 2024-03-14T12:15:17.468 INFO:tasks.cephfs.mount:File test_dir/test_file_18 became visible from 1 after 0s 2024-03-14T12:15:17.468 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.468 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.515 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.524 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.524 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.524 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_19", '"'"'r'"'"') as f: 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.525 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.566 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_19 2024-03-14T12:15:17.567 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_19 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627796 Links: 1 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.380293239 +0000 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.380293239 +0000 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.380293239 +0000 2024-03-14T12:15:17.582 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.380293239 +0000 2024-03-14T12:15:17.583 INFO:tasks.cephfs.mount:File test_dir/test_file_19 became visible from 1 after 0s 2024-03-14T12:15:17.583 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.583 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.630 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_20", '"'"'r'"'"') as f: 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.638 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.639 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.639 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.639 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.639 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.639 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.639 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.639 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.680 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_20 2024-03-14T12:15:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_20 2024-03-14T12:15:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.696 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627797 Links: 1 2024-03-14T12:15:17.696 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.696 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.696 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.383293184 +0000 2024-03-14T12:15:17.696 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.383293184 +0000 2024-03-14T12:15:17.696 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.383293184 +0000 2024-03-14T12:15:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.383293184 +0000 2024-03-14T12:15:17.696 INFO:tasks.cephfs.mount:File test_dir/test_file_20 became visible from 1 after 0s 2024-03-14T12:15:17.696 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.696 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.743 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.751 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.751 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.751 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.751 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.751 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_21", '"'"'r'"'"') as f: 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.752 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.793 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_21 2024-03-14T12:15:17.808 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_21 2024-03-14T12:15:17.808 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.808 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627798 Links: 1 2024-03-14T12:15:17.808 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.809 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.809 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.393292997 +0000 2024-03-14T12:15:17.809 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.393292997 +0000 2024-03-14T12:15:17.809 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.393292997 +0000 2024-03-14T12:15:17.809 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.393292997 +0000 2024-03-14T12:15:17.809 INFO:tasks.cephfs.mount:File test_dir/test_file_21 became visible from 1 after 0s 2024-03-14T12:15:17.809 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.809 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.857 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.863 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.863 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.863 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.863 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.863 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.863 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_22", '"'"'r'"'"') as f: 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.864 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:17.905 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_22 2024-03-14T12:15:17.920 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_22 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627799 Links: 1 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.396292942 +0000 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.396292942 +0000 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.396292942 +0000 2024-03-14T12:15:17.921 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.396292942 +0000 2024-03-14T12:15:17.921 INFO:tasks.cephfs.mount:File test_dir/test_file_22 became visible from 1 after 0s 2024-03-14T12:15:17.921 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:17.921 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:17.968 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:17.976 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:17.976 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:17.976 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:17.976 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:17.976 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:17.976 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_23", '"'"'r'"'"') as f: 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:17.977 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.018 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_23 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_23 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627800 Links: 1 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.400292867 +0000 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.400292867 +0000 2024-03-14T12:15:18.034 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.400292867 +0000 2024-03-14T12:15:18.035 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.400292867 +0000 2024-03-14T12:15:18.035 INFO:tasks.cephfs.mount:File test_dir/test_file_23 became visible from 1 after 0s 2024-03-14T12:15:18.035 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.035 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.063 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.077 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.077 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.077 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_24", '"'"'r'"'"') as f: 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.078 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.079 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.079 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.082 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.083 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_24 2024-03-14T12:15:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_24 2024-03-14T12:15:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.098 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627801 Links: 1 2024-03-14T12:15:18.098 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.098 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.099 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.405292774 +0000 2024-03-14T12:15:18.099 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.405292774 +0000 2024-03-14T12:15:18.099 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.405292774 +0000 2024-03-14T12:15:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.405292774 +0000 2024-03-14T12:15:18.099 INFO:tasks.cephfs.mount:File test_dir/test_file_24 became visible from 1 after 0s 2024-03-14T12:15:18.099 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.099 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.099 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.146 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.153 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_25", '"'"'r'"'"') as f: 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.154 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.155 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.155 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.155 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.196 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_25 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_25 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627802 Links: 1 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.412292644 +0000 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.412292644 +0000 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.412292644 +0000 2024-03-14T12:15:18.212 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.412292644 +0000 2024-03-14T12:15:18.212 INFO:tasks.cephfs.mount:File test_dir/test_file_25 became visible from 1 after 0s 2024-03-14T12:15:18.213 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.213 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.222 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_26", '"'"'r'"'"') as f: 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.237 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.238 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.238 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.238 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.260 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.261 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_26 2024-03-14T12:15:18.276 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_26 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627803 Links: 1 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.415292588 +0000 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.415292588 +0000 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.415292588 +0000 2024-03-14T12:15:18.277 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.415292588 +0000 2024-03-14T12:15:18.277 INFO:tasks.cephfs.mount:File test_dir/test_file_26 became visible from 1 after 0s 2024-03-14T12:15:18.277 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.277 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.325 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.333 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.333 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.333 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_27", '"'"'r'"'"') as f: 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.334 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.335 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.337 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_27 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_27 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627804 Links: 1 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.418292532 +0000 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.419292514 +0000 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.419292514 +0000 2024-03-14T12:15:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.418292532 +0000 2024-03-14T12:15:18.352 INFO:tasks.cephfs.mount:File test_dir/test_file_27 became visible from 1 after 0s 2024-03-14T12:15:18.353 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.353 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.400 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.407 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.407 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_28", '"'"'r'"'"') as f: 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.408 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.409 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.446 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.448 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_28 2024-03-14T12:15:18.463 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_28 2024-03-14T12:15:18.463 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.463 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627805 Links: 1 2024-03-14T12:15:18.464 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.464 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.464 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.422292458 +0000 2024-03-14T12:15:18.464 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.422292458 +0000 2024-03-14T12:15:18.464 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.422292458 +0000 2024-03-14T12:15:18.464 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.422292458 +0000 2024-03-14T12:15:18.464 INFO:tasks.cephfs.mount:File test_dir/test_file_28 became visible from 1 after 0s 2024-03-14T12:15:18.464 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.464 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.511 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.519 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.519 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.519 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.519 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_29", '"'"'r'"'"') as f: 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.520 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.560 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.561 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_29 2024-03-14T12:15:18.576 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_29 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627806 Links: 1 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.428292346 +0000 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.428292346 +0000 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.428292346 +0000 2024-03-14T12:15:18.577 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.428292346 +0000 2024-03-14T12:15:18.577 INFO:tasks.cephfs.mount:File test_dir/test_file_29 became visible from 1 after 0s 2024-03-14T12:15:18.577 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.577 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.625 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.633 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_30", '"'"'r'"'"') as f: 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.634 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.672 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.674 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_30 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_30 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627807 Links: 1 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.432292272 +0000 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.432292272 +0000 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.432292272 +0000 2024-03-14T12:15:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.432292272 +0000 2024-03-14T12:15:18.690 INFO:tasks.cephfs.mount:File test_dir/test_file_30 became visible from 1 after 0s 2024-03-14T12:15:18.690 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.690 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.737 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.744 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.744 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.744 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.744 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.744 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_31", '"'"'r'"'"') as f: 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.745 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.746 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.746 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.746 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.746 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.746 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.785 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.787 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_31 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_31 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627808 Links: 1 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.435292216 +0000 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.435292216 +0000 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.435292216 +0000 2024-03-14T12:15:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.435292216 +0000 2024-03-14T12:15:18.802 INFO:tasks.cephfs.mount:File test_dir/test_file_31 became visible from 1 after 0s 2024-03-14T12:15:18.803 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.803 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.860 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_32", '"'"'r'"'"') as f: 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.861 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.866 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.866 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.866 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.866 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.867 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.867 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.867 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:18.905 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_32 2024-03-14T12:15:18.906 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_32 2024-03-14T12:15:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:18.921 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627809 Links: 1 2024-03-14T12:15:18.921 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:18.921 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:18.922 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.440292123 +0000 2024-03-14T12:15:18.922 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.441292105 +0000 2024-03-14T12:15:18.922 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.441292105 +0000 2024-03-14T12:15:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.440292123 +0000 2024-03-14T12:15:18.922 INFO:tasks.cephfs.mount:File test_dir/test_file_32 became visible from 1 after 0s 2024-03-14T12:15:18.922 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:18.922 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:18.968 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_33", '"'"'r'"'"') as f: 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:18.977 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:18.978 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:18.978 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.017 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.019 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_33 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_33 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627810 Links: 1 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.443292067 +0000 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.443292067 +0000 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.443292067 +0000 2024-03-14T12:15:19.034 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.443292067 +0000 2024-03-14T12:15:19.034 INFO:tasks.cephfs.mount:File test_dir/test_file_33 became visible from 1 after 0s 2024-03-14T12:15:19.035 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.035 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.082 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.091 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.091 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.091 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.091 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.091 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_34", '"'"'r'"'"') as f: 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.092 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.129 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.130 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_34 2024-03-14T12:15:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_34 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627811 Links: 1 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.446292012 +0000 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.446292012 +0000 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.446292012 +0000 2024-03-14T12:15:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.446292012 +0000 2024-03-14T12:15:19.146 INFO:tasks.cephfs.mount:File test_dir/test_file_34 became visible from 1 after 0s 2024-03-14T12:15:19.147 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.147 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.193 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.201 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.201 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.201 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.201 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_35", '"'"'r'"'"') as f: 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.202 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.228 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_35 2024-03-14T12:15:19.242 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.243 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_35 2024-03-14T12:15:19.243 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.243 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627812 Links: 1 2024-03-14T12:15:19.243 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.244 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.244 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.451291918 +0000 2024-03-14T12:15:19.244 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.451291918 +0000 2024-03-14T12:15:19.244 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.451291918 +0000 2024-03-14T12:15:19.244 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.451291918 +0000 2024-03-14T12:15:19.244 INFO:tasks.cephfs.mount:File test_dir/test_file_35 became visible from 1 after 0s 2024-03-14T12:15:19.244 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.244 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.291 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.298 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_36", '"'"'r'"'"') as f: 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.299 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.300 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.300 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.300 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.300 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.306 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.308 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_36 2024-03-14T12:15:19.323 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_36 2024-03-14T12:15:19.323 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.323 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627813 Links: 1 2024-03-14T12:15:19.323 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.324 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.324 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.454291863 +0000 2024-03-14T12:15:19.324 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.454291863 +0000 2024-03-14T12:15:19.324 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.454291863 +0000 2024-03-14T12:15:19.324 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.454291863 +0000 2024-03-14T12:15:19.324 INFO:tasks.cephfs.mount:File test_dir/test_file_36 became visible from 1 after 0s 2024-03-14T12:15:19.324 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.324 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.355 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_37", '"'"'r'"'"') as f: 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.356 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.371 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.373 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_37 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_37 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627814 Links: 1 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.458291788 +0000 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.458291788 +0000 2024-03-14T12:15:19.389 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.458291788 +0000 2024-03-14T12:15:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.458291788 +0000 2024-03-14T12:15:19.390 INFO:tasks.cephfs.mount:File test_dir/test_file_37 became visible from 1 after 0s 2024-03-14T12:15:19.390 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.390 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.405 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_38", '"'"'r'"'"') as f: 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.420 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.421 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.421 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.421 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.421 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.421 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.421 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.421 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.437 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.439 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_38 2024-03-14T12:15:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_38 2024-03-14T12:15:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.455 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627815 Links: 1 2024-03-14T12:15:19.455 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.455 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.455 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.461291732 +0000 2024-03-14T12:15:19.456 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.461291732 +0000 2024-03-14T12:15:19.456 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.461291732 +0000 2024-03-14T12:15:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.461291732 +0000 2024-03-14T12:15:19.456 INFO:tasks.cephfs.mount:File test_dir/test_file_38 became visible from 1 after 0s 2024-03-14T12:15:19.456 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.456 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.457 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.502 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.510 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_39", '"'"'r'"'"') as f: 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.511 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.512 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.517 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.519 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_39 2024-03-14T12:15:19.534 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_39 2024-03-14T12:15:19.534 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.534 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627816 Links: 1 2024-03-14T12:15:19.534 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.534 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.534 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.464291677 +0000 2024-03-14T12:15:19.535 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.464291677 +0000 2024-03-14T12:15:19.535 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.464291677 +0000 2024-03-14T12:15:19.535 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.464291677 +0000 2024-03-14T12:15:19.535 INFO:tasks.cephfs.mount:File test_dir/test_file_39 became visible from 1 after 0s 2024-03-14T12:15:19.535 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.535 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.582 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.584 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_40", '"'"'r'"'"') as f: 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.585 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.627 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_40 2024-03-14T12:15:19.632 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_40 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627817 Links: 1 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.468291602 +0000 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.468291602 +0000 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.468291602 +0000 2024-03-14T12:15:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.468291602 +0000 2024-03-14T12:15:19.643 INFO:tasks.cephfs.mount:File test_dir/test_file_40 became visible from 1 after 0s 2024-03-14T12:15:19.644 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.644 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.700 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.700 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.700 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.700 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.700 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.700 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_41", '"'"'r'"'"') as f: 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.701 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.702 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.702 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.702 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.702 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.743 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_41 2024-03-14T12:15:19.745 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.759 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_41 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627818 Links: 1 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.471291546 +0000 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.472291528 +0000 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.472291528 +0000 2024-03-14T12:15:19.760 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.471291546 +0000 2024-03-14T12:15:19.760 INFO:tasks.cephfs.mount:File test_dir/test_file_41 became visible from 1 after 0s 2024-03-14T12:15:19.761 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.761 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.806 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.815 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.815 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_42", '"'"'r'"'"') as f: 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.816 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.817 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.817 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.817 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.858 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_42 2024-03-14T12:15:19.860 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_42 2024-03-14T12:15:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.873 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627819 Links: 1 2024-03-14T12:15:19.873 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.873 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.874 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.476291453 +0000 2024-03-14T12:15:19.874 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.476291453 +0000 2024-03-14T12:15:19.874 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.476291453 +0000 2024-03-14T12:15:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.476291453 +0000 2024-03-14T12:15:19.874 INFO:tasks.cephfs.mount:File test_dir/test_file_42 became visible from 1 after 0s 2024-03-14T12:15:19.874 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.874 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:19.922 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:19.928 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_43", '"'"'r'"'"') as f: 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:19.929 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:19.970 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_43 2024-03-14T12:15:19.971 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_43 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627820 Links: 1 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.479291398 +0000 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.479291398 +0000 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.479291398 +0000 2024-03-14T12:15:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.479291398 +0000 2024-03-14T12:15:19.986 INFO:tasks.cephfs.mount:File test_dir/test_file_43 became visible from 1 after 0s 2024-03-14T12:15:19.987 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:19.987 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.041 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_44", '"'"'r'"'"') as f: 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.042 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.065 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.067 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_44 2024-03-14T12:15:20.082 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_44 2024-03-14T12:15:20.082 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.082 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627821 Links: 1 2024-03-14T12:15:20.082 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.082 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.082 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.482291342 +0000 2024-03-14T12:15:20.082 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.482291342 +0000 2024-03-14T12:15:20.083 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.482291342 +0000 2024-03-14T12:15:20.083 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.482291342 +0000 2024-03-14T12:15:20.083 INFO:tasks.cephfs.mount:File test_dir/test_file_44 became visible from 1 after 0s 2024-03-14T12:15:20.083 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.083 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.084 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_45", '"'"'r'"'"') as f: 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.099 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.100 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.100 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.100 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.100 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.100 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.100 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.102 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_45 2024-03-14T12:15:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_45 2024-03-14T12:15:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.118 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627822 Links: 1 2024-03-14T12:15:20.118 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.118 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.118 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.485291286 +0000 2024-03-14T12:15:20.118 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.485291286 +0000 2024-03-14T12:15:20.118 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.485291286 +0000 2024-03-14T12:15:20.118 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.485291286 +0000 2024-03-14T12:15:20.118 INFO:tasks.cephfs.mount:File test_dir/test_file_45 became visible from 1 after 0s 2024-03-14T12:15:20.118 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.118 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.133 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.147 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_46", '"'"'r'"'"') as f: 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.148 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.149 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.150 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_46 2024-03-14T12:15:20.165 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_46 2024-03-14T12:15:20.165 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.165 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627823 Links: 1 2024-03-14T12:15:20.165 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.165 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.165 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.488291230 +0000 2024-03-14T12:15:20.166 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.488291230 +0000 2024-03-14T12:15:20.166 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.488291230 +0000 2024-03-14T12:15:20.166 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.488291230 +0000 2024-03-14T12:15:20.166 INFO:tasks.cephfs.mount:File test_dir/test_file_46 became visible from 1 after 0s 2024-03-14T12:15:20.166 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.166 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.167 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_47", '"'"'r'"'"') as f: 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.181 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.182 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.182 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.182 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.182 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.182 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.182 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.182 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.214 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.216 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_47 2024-03-14T12:15:20.224 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_47 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627824 Links: 1 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.492291156 +0000 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.492291156 +0000 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.492291156 +0000 2024-03-14T12:15:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.492291156 +0000 2024-03-14T12:15:20.231 INFO:tasks.cephfs.mount:File test_dir/test_file_47 became visible from 1 after 0s 2024-03-14T12:15:20.232 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.232 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.261 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.275 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.275 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_48", '"'"'r'"'"') as f: 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.276 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.277 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.279 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.281 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_48 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_48 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627825 Links: 1 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.495291100 +0000 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.495291100 +0000 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.495291100 +0000 2024-03-14T12:15:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.495291100 +0000 2024-03-14T12:15:20.296 INFO:tasks.cephfs.mount:File test_dir/test_file_48 became visible from 1 after 0s 2024-03-14T12:15:20.296 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.296 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.327 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.336 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.341 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.341 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_49", '"'"'r'"'"') as f: 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.342 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.343 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.345 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.346 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_49 2024-03-14T12:15:20.361 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_49 2024-03-14T12:15:20.361 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.361 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627826 Links: 1 2024-03-14T12:15:20.361 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.361 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.361 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.499291025 +0000 2024-03-14T12:15:20.362 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.499291025 +0000 2024-03-14T12:15:20.362 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.499291025 +0000 2024-03-14T12:15:20.362 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.499291025 +0000 2024-03-14T12:15:20.362 INFO:tasks.cephfs.mount:File test_dir/test_file_49 became visible from 1 after 0s 2024-03-14T12:15:20.362 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.362 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.402 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.410 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.417 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.418 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_50", '"'"'r'"'"') as f: 2024-03-14T12:15:20.418 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.418 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.418 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.418 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.418 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.418 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.419 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.419 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.419 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.448 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.450 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_50 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_50 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627827 Links: 1 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.503290951 +0000 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.503290951 +0000 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.503290951 +0000 2024-03-14T12:15:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.503290951 +0000 2024-03-14T12:15:20.466 INFO:tasks.cephfs.mount:File test_dir/test_file_50 became visible from 1 after 0s 2024-03-14T12:15:20.467 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.467 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.513 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.513 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.521 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_51", '"'"'r'"'"') as f: 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.522 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.562 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.564 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_51 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_51 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627828 Links: 1 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.506290895 +0000 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.506290895 +0000 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.506290895 +0000 2024-03-14T12:15:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.506290895 +0000 2024-03-14T12:15:20.579 INFO:tasks.cephfs.mount:File test_dir/test_file_51 became visible from 1 after 0s 2024-03-14T12:15:20.579 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.580 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.627 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.627 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.634 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.634 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.634 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.634 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.634 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_52", '"'"'r'"'"') as f: 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.635 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.676 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_52 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_52 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627829 Links: 1 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.510290821 +0000 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.510290821 +0000 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.510290821 +0000 2024-03-14T12:15:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.510290821 +0000 2024-03-14T12:15:20.692 INFO:tasks.cephfs.mount:File test_dir/test_file_52 became visible from 1 after 0s 2024-03-14T12:15:20.692 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.692 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.739 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.739 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.747 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.747 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.747 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.747 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.747 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.747 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_53", '"'"'r'"'"') as f: 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.748 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.786 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.788 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_53 2024-03-14T12:15:20.803 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_53 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627830 Links: 1 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.516290709 +0000 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.516290709 +0000 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.516290709 +0000 2024-03-14T12:15:20.804 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.516290709 +0000 2024-03-14T12:15:20.804 INFO:tasks.cephfs.mount:File test_dir/test_file_53 became visible from 1 after 0s 2024-03-14T12:15:20.804 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.804 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.852 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.853 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_54", '"'"'r'"'"') as f: 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.860 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.861 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.861 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.861 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.861 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.861 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:20.903 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_54 2024-03-14T12:15:20.906 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_54 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627831 Links: 1 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.520290635 +0000 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.520290635 +0000 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.520290635 +0000 2024-03-14T12:15:20.918 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.520290635 +0000 2024-03-14T12:15:20.918 INFO:tasks.cephfs.mount:File test_dir/test_file_54 became visible from 1 after 0s 2024-03-14T12:15:20.919 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:20.919 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:20.965 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.970 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:20.973 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:20.973 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:20.973 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_55", '"'"'r'"'"') as f: 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:20.974 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.016 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_55 2024-03-14T12:15:21.019 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_55 2024-03-14T12:15:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.032 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627832 Links: 1 2024-03-14T12:15:21.032 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.032 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.032 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.523290579 +0000 2024-03-14T12:15:21.032 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.523290579 +0000 2024-03-14T12:15:21.032 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.523290579 +0000 2024-03-14T12:15:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.523290579 +0000 2024-03-14T12:15:21.032 INFO:tasks.cephfs.mount:File test_dir/test_file_55 became visible from 1 after 0s 2024-03-14T12:15:21.032 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.032 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.079 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.084 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.088 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_56", '"'"'r'"'"') as f: 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.089 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.090 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.090 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.090 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.090 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.090 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.090 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.090 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.131 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_56 2024-03-14T12:15:21.132 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_56 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627833 Links: 1 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.526290523 +0000 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.526290523 +0000 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.526290523 +0000 2024-03-14T12:15:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.526290523 +0000 2024-03-14T12:15:21.147 INFO:tasks.cephfs.mount:File test_dir/test_file_56 became visible from 1 after 0s 2024-03-14T12:15:21.147 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.147 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.194 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.195 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_57", '"'"'r'"'"') as f: 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.203 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.204 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.204 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.204 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.204 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.228 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.230 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_57 2024-03-14T12:15:21.244 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_57 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627834 Links: 1 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.532290411 +0000 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.532290411 +0000 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.532290411 +0000 2024-03-14T12:15:21.246 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.532290411 +0000 2024-03-14T12:15:21.246 INFO:tasks.cephfs.mount:File test_dir/test_file_57 became visible from 1 after 0s 2024-03-14T12:15:21.247 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.247 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.293 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.293 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.301 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.301 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.301 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.301 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.301 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.301 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.301 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_58", '"'"'r'"'"') as f: 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.302 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.308 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.310 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_58 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_58 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627835 Links: 1 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.535290356 +0000 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.535290356 +0000 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.535290356 +0000 2024-03-14T12:15:21.325 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.535290356 +0000 2024-03-14T12:15:21.325 INFO:tasks.cephfs.mount:File test_dir/test_file_58 became visible from 1 after 0s 2024-03-14T12:15:21.325 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.325 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.342 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.356 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.356 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.356 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_59", '"'"'r'"'"') as f: 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.357 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.375 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_59 2024-03-14T12:15:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_59 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627836 Links: 1 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.539290281 +0000 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.539290281 +0000 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.539290281 +0000 2024-03-14T12:15:21.390 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.539290281 +0000 2024-03-14T12:15:21.391 INFO:tasks.cephfs.mount:File test_dir/test_file_59 became visible from 1 after 0s 2024-03-14T12:15:21.391 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.391 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.407 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.421 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.421 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.421 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.421 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_60", '"'"'r'"'"') as f: 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.422 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.438 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.439 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_60 2024-03-14T12:15:21.440 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_60 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627837 Links: 1 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.542290225 +0000 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.542290225 +0000 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.542290225 +0000 2024-03-14T12:15:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.542290225 +0000 2024-03-14T12:15:21.455 INFO:tasks.cephfs.mount:File test_dir/test_file_60 became visible from 1 after 0s 2024-03-14T12:15:21.456 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.456 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.456 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.470 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_61", '"'"'r'"'"') as f: 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.471 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.503 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.504 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.505 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_61 2024-03-14T12:15:21.519 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_61 2024-03-14T12:15:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.520 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627838 Links: 1 2024-03-14T12:15:21.521 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.521 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.521 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.545290170 +0000 2024-03-14T12:15:21.521 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.545290170 +0000 2024-03-14T12:15:21.521 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.545290170 +0000 2024-03-14T12:15:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.545290170 +0000 2024-03-14T12:15:21.521 INFO:tasks.cephfs.mount:File test_dir/test_file_61 became visible from 1 after 0s 2024-03-14T12:15:21.521 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.521 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.568 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.571 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_62", '"'"'r'"'"') as f: 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.577 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.578 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.578 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.578 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.584 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.586 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_62 2024-03-14T12:15:21.601 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_62 2024-03-14T12:15:21.601 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.602 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627839 Links: 1 2024-03-14T12:15:21.602 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.602 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.602 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.551290058 +0000 2024-03-14T12:15:21.602 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.551290058 +0000 2024-03-14T12:15:21.602 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.551290058 +0000 2024-03-14T12:15:21.602 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.551290058 +0000 2024-03-14T12:15:21.602 INFO:tasks.cephfs.mount:File test_dir/test_file_62 became visible from 1 after 0s 2024-03-14T12:15:21.602 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.602 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.634 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_63", '"'"'r'"'"') as f: 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.649 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.650 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.650 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.650 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.650 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.650 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.652 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_63 2024-03-14T12:15:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_63 2024-03-14T12:15:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.668 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627840 Links: 1 2024-03-14T12:15:21.668 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.668 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.668 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.554290002 +0000 2024-03-14T12:15:21.668 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.554290002 +0000 2024-03-14T12:15:21.668 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.554290002 +0000 2024-03-14T12:15:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.554290002 +0000 2024-03-14T12:15:21.668 INFO:tasks.cephfs.mount:File test_dir/test_file_63 became visible from 1 after 0s 2024-03-14T12:15:21.668 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.668 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.692 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.706 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.706 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.706 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.706 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.706 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_64", '"'"'r'"'"') as f: 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.707 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.715 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.717 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_64 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_64 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627841 Links: 1 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.559289909 +0000 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.559289909 +0000 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.559289909 +0000 2024-03-14T12:15:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.559289909 +0000 2024-03-14T12:15:21.733 INFO:tasks.cephfs.mount:File test_dir/test_file_64 became visible from 1 after 0s 2024-03-14T12:15:21.733 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.733 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.747 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.762 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.762 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.762 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.762 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.762 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.762 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_65", '"'"'r'"'"') as f: 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.763 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.781 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.782 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_65 2024-03-14T12:15:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_65 2024-03-14T12:15:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.798 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627842 Links: 1 2024-03-14T12:15:21.798 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.798 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.799 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.563289835 +0000 2024-03-14T12:15:21.799 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.563289835 +0000 2024-03-14T12:15:21.799 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.563289835 +0000 2024-03-14T12:15:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.563289835 +0000 2024-03-14T12:15:21.799 INFO:tasks.cephfs.mount:File test_dir/test_file_65 became visible from 1 after 0s 2024-03-14T12:15:21.799 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.799 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.808 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.822 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.822 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.822 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.822 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_66", '"'"'r'"'"') as f: 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.823 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.848 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.850 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_66 2024-03-14T12:15:21.854 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:15:21.857 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:15:21.861 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_66 2024-03-14T12:15:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.866 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627843 Links: 1 2024-03-14T12:15:21.866 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.866 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.867 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.569289723 +0000 2024-03-14T12:15:21.867 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.569289723 +0000 2024-03-14T12:15:21.867 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.569289723 +0000 2024-03-14T12:15:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.569289723 +0000 2024-03-14T12:15:21.867 INFO:tasks.cephfs.mount:File test_dir/test_file_66 became visible from 1 after 0s 2024-03-14T12:15:21.867 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.867 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.914 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.922 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_67", '"'"'r'"'"') as f: 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.923 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.924 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.925 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_67 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_67 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627844 Links: 1 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.572289667 +0000 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.572289667 +0000 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.572289667 +0000 2024-03-14T12:15:21.941 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.572289667 +0000 2024-03-14T12:15:21.941 INFO:tasks.cephfs.mount:File test_dir/test_file_67 became visible from 1 after 0s 2024-03-14T12:15:21.942 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:21.942 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:21.972 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_68", '"'"'r'"'"') as f: 2024-03-14T12:15:21.987 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:21.988 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:21.991 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_68 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_68 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627845 Links: 1 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.575289611 +0000 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.576289593 +0000 2024-03-14T12:15:22.006 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.576289593 +0000 2024-03-14T12:15:22.007 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.575289611 +0000 2024-03-14T12:15:22.007 INFO:tasks.cephfs.mount:File test_dir/test_file_68 became visible from 1 after 0s 2024-03-14T12:15:22.007 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.007 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.036 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.050 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_69", '"'"'r'"'"') as f: 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.051 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.052 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.052 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.054 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.056 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_69 2024-03-14T12:15:22.067 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_69 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627846 Links: 1 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.584289444 +0000 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.584289444 +0000 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.584289444 +0000 2024-03-14T12:15:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.584289444 +0000 2024-03-14T12:15:22.071 INFO:tasks.cephfs.mount:File test_dir/test_file_69 became visible from 1 after 0s 2024-03-14T12:15:22.071 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.072 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.085 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.099 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.099 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.099 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.099 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.099 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.099 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_70", '"'"'r'"'"') as f: 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.100 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.103 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_70 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_70 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627847 Links: 1 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.587289388 +0000 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.587289388 +0000 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.587289388 +0000 2024-03-14T12:15:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.587289388 +0000 2024-03-14T12:15:22.119 INFO:tasks.cephfs.mount:File test_dir/test_file_70 became visible from 1 after 0s 2024-03-14T12:15:22.120 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.120 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.120 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.134 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.135 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.135 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.135 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_71", '"'"'r'"'"') as f: 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.136 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.150 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.153 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_71 2024-03-14T12:15:22.168 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.168 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_71 2024-03-14T12:15:22.168 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.168 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627848 Links: 1 2024-03-14T12:15:22.169 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.169 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.169 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.590289332 +0000 2024-03-14T12:15:22.169 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.590289332 +0000 2024-03-14T12:15:22.169 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.590289332 +0000 2024-03-14T12:15:22.169 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.590289332 +0000 2024-03-14T12:15:22.169 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.170 INFO:tasks.cephfs.mount:File test_dir/test_file_71 became visible from 1 after 0s 2024-03-14T12:15:22.170 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.170 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.215 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.216 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.226 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_72", '"'"'r'"'"') as f: 2024-03-14T12:15:22.227 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.228 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.265 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_72 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_72 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627849 Links: 1 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.594289258 +0000 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.594289258 +0000 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.594289258 +0000 2024-03-14T12:15:22.280 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.594289258 +0000 2024-03-14T12:15:22.280 INFO:tasks.cephfs.mount:File test_dir/test_file_72 became visible from 1 after 0s 2024-03-14T12:15:22.281 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.281 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_73", '"'"'r'"'"') as f: 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.296 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.297 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.297 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.297 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.297 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.297 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.297 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.297 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.329 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.329 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.331 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_73 2024-03-14T12:15:22.338 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.346 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_73 2024-03-14T12:15:22.346 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.346 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627850 Links: 1 2024-03-14T12:15:22.346 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.347 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.347 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.599289165 +0000 2024-03-14T12:15:22.347 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.599289165 +0000 2024-03-14T12:15:22.347 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.599289165 +0000 2024-03-14T12:15:22.347 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.599289165 +0000 2024-03-14T12:15:22.347 INFO:tasks.cephfs.mount:File test_dir/test_file_73 became visible from 1 after 0s 2024-03-14T12:15:22.347 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.347 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.348 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.394 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.401 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_74", '"'"'r'"'"') as f: 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.402 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.403 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.403 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.403 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.405 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_74 2024-03-14T12:15:22.411 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.420 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_74 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627851 Links: 1 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.603289091 +0000 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.603289091 +0000 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.603289091 +0000 2024-03-14T12:15:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.603289091 +0000 2024-03-14T12:15:22.421 INFO:tasks.cephfs.mount:File test_dir/test_file_74 became visible from 1 after 0s 2024-03-14T12:15:22.421 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.422 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.450 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.464 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.464 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.464 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_75", '"'"'r'"'"') as f: 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.465 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.469 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.470 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_75 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_75 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627852 Links: 1 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.609288979 +0000 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.609288979 +0000 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.609288979 +0000 2024-03-14T12:15:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.609288979 +0000 2024-03-14T12:15:22.487 INFO:tasks.cephfs.mount:File test_dir/test_file_75 became visible from 1 after 0s 2024-03-14T12:15:22.487 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.487 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.515 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.515 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.530 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_76", '"'"'r'"'"') as f: 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.531 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.534 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.535 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_76 2024-03-14T12:15:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_76 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627853 Links: 1 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.612288923 +0000 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.612288923 +0000 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.612288923 +0000 2024-03-14T12:15:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.612288923 +0000 2024-03-14T12:15:22.551 INFO:tasks.cephfs.mount:File test_dir/test_file_76 became visible from 1 after 0s 2024-03-14T12:15:22.551 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.552 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.564 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.578 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.578 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_77", '"'"'r'"'"') as f: 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.579 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.599 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.601 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_77 2024-03-14T12:15:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_77 2024-03-14T12:15:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.616 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627854 Links: 1 2024-03-14T12:15:22.616 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.617 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.617 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.617288830 +0000 2024-03-14T12:15:22.617 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.617288830 +0000 2024-03-14T12:15:22.617 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.617288830 +0000 2024-03-14T12:15:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.617288830 +0000 2024-03-14T12:15:22.617 INFO:tasks.cephfs.mount:File test_dir/test_file_77 became visible from 1 after 0s 2024-03-14T12:15:22.617 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.617 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.629 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.629 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.644 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_78", '"'"'r'"'"') as f: 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.645 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.664 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.666 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_78 2024-03-14T12:15:22.676 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.682 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_78 2024-03-14T12:15:22.682 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.682 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627855 Links: 1 2024-03-14T12:15:22.682 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.682 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.682 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.621288755 +0000 2024-03-14T12:15:22.682 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.621288755 +0000 2024-03-14T12:15:22.683 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.621288755 +0000 2024-03-14T12:15:22.683 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.621288755 +0000 2024-03-14T12:15:22.683 INFO:tasks.cephfs.mount:File test_dir/test_file_78 became visible from 1 after 0s 2024-03-14T12:15:22.683 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.683 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.730 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.737 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_79", '"'"'r'"'"') as f: 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.738 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.741 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.742 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.743 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_79 2024-03-14T12:15:22.757 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_79 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627856 Links: 1 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.626288663 +0000 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.626288663 +0000 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.626288663 +0000 2024-03-14T12:15:22.758 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.626288663 +0000 2024-03-14T12:15:22.758 INFO:tasks.cephfs.mount:File test_dir/test_file_79 became visible from 1 after 0s 2024-03-14T12:15:22.758 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.758 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.788 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.802 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.802 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.802 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.802 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.802 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_80", '"'"'r'"'"') as f: 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.803 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.806 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.808 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_80 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_80 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627857 Links: 1 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.629288607 +0000 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.629288607 +0000 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.629288607 +0000 2024-03-14T12:15:22.823 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.629288607 +0000 2024-03-14T12:15:22.824 INFO:tasks.cephfs.mount:File test_dir/test_file_80 became visible from 1 after 0s 2024-03-14T12:15:22.824 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.824 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.854 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.856 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.868 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.868 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_81", '"'"'r'"'"') as f: 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.869 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.870 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.871 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.873 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_81 2024-03-14T12:15:22.888 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_81 2024-03-14T12:15:22.888 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.888 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627858 Links: 1 2024-03-14T12:15:22.888 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.889 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.889 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.633288532 +0000 2024-03-14T12:15:22.889 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.633288532 +0000 2024-03-14T12:15:22.889 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.633288532 +0000 2024-03-14T12:15:22.889 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.633288532 +0000 2024-03-14T12:15:22.889 INFO:tasks.cephfs.mount:File test_dir/test_file_81 became visible from 1 after 0s 2024-03-14T12:15:22.889 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.889 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.908 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_82", '"'"'r'"'"') as f: 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.923 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.924 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.924 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.924 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.924 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.924 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.924 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:22.937 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.939 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_82 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_82 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627859 Links: 1 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.636288477 +0000 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.636288477 +0000 2024-03-14T12:15:22.954 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.636288477 +0000 2024-03-14T12:15:22.955 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.636288477 +0000 2024-03-14T12:15:22.955 INFO:tasks.cephfs.mount:File test_dir/test_file_82 became visible from 1 after 0s 2024-03-14T12:15:22.955 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:22.955 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:22.967 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.972 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_83", '"'"'r'"'"') as f: 2024-03-14T12:15:22.982 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:22.983 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.002 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.004 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_83 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_83 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627860 Links: 1 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.641288383 +0000 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.641288383 +0000 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.641288383 +0000 2024-03-14T12:15:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.641288383 +0000 2024-03-14T12:15:23.020 INFO:tasks.cephfs.mount:File test_dir/test_file_83 became visible from 1 after 0s 2024-03-14T12:15:23.020 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.020 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.068 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.075 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_84", '"'"'r'"'"') as f: 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.076 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.077 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.077 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.077 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.081 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.083 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_84 2024-03-14T12:15:23.086 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_84 2024-03-14T12:15:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.099 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627861 Links: 1 2024-03-14T12:15:23.099 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.099 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.099 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.647288272 +0000 2024-03-14T12:15:23.099 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.647288272 +0000 2024-03-14T12:15:23.100 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.647288272 +0000 2024-03-14T12:15:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.647288272 +0000 2024-03-14T12:15:23.100 INFO:tasks.cephfs.mount:File test_dir/test_file_84 became visible from 1 after 0s 2024-03-14T12:15:23.100 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.100 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.133 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.147 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_85", '"'"'r'"'"') as f: 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.148 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.149 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.149 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.149 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.149 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.149 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.190 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_85 2024-03-14T12:15:23.228 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.229 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_85 2024-03-14T12:15:23.229 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.230 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627862 Links: 1 2024-03-14T12:15:23.230 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.230 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.230 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.650288216 +0000 2024-03-14T12:15:23.230 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.650288216 +0000 2024-03-14T12:15:23.230 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.650288216 +0000 2024-03-14T12:15:23.230 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.650288216 +0000 2024-03-14T12:15:23.230 INFO:tasks.cephfs.mount:File test_dir/test_file_85 became visible from 1 after 0s 2024-03-14T12:15:23.230 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.230 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.231 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.245 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_86", '"'"'r'"'"') as f: 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.246 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.247 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.247 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.247 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.247 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.247 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.249 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_86 2024-03-14T12:15:23.254 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_86 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627863 Links: 1 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.653288160 +0000 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.653288160 +0000 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.653288160 +0000 2024-03-14T12:15:23.265 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.653288160 +0000 2024-03-14T12:15:23.265 INFO:tasks.cephfs.mount:File test_dir/test_file_86 became visible from 1 after 0s 2024-03-14T12:15:23.265 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.265 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.295 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.295 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.310 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.310 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_87", '"'"'r'"'"') as f: 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.311 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.312 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.312 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.313 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.314 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_87 2024-03-14T12:15:23.329 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_87 2024-03-14T12:15:23.329 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.330 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627864 Links: 1 2024-03-14T12:15:23.330 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.330 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.330 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.656288104 +0000 2024-03-14T12:15:23.330 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.656288104 +0000 2024-03-14T12:15:23.330 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.656288104 +0000 2024-03-14T12:15:23.330 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.656288104 +0000 2024-03-14T12:15:23.330 INFO:tasks.cephfs.mount:File test_dir/test_file_87 became visible from 1 after 0s 2024-03-14T12:15:23.330 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.331 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.344 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_88", '"'"'r'"'"') as f: 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.359 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.360 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.360 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.360 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.360 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.360 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.360 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.375 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.375 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.377 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_88 2024-03-14T12:15:23.379 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.392 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_88 2024-03-14T12:15:23.392 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.392 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627865 Links: 1 2024-03-14T12:15:23.392 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.392 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.392 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.659288049 +0000 2024-03-14T12:15:23.393 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.659288049 +0000 2024-03-14T12:15:23.393 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.659288049 +0000 2024-03-14T12:15:23.393 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.659288049 +0000 2024-03-14T12:15:23.393 INFO:tasks.cephfs.mount:File test_dir/test_file_88 became visible from 1 after 0s 2024-03-14T12:15:23.393 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.393 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.409 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_89", '"'"'r'"'"') as f: 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.424 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.425 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.425 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.425 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.425 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.425 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.439 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.441 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_89 2024-03-14T12:15:23.441 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.442 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.456 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_89 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627866 Links: 1 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.662287993 +0000 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.662287993 +0000 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.662287993 +0000 2024-03-14T12:15:23.457 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.662287993 +0000 2024-03-14T12:15:23.457 INFO:tasks.cephfs.mount:File test_dir/test_file_89 became visible from 1 after 0s 2024-03-14T12:15:23.458 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.458 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.458 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_90", '"'"'r'"'"') as f: 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.473 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.474 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.474 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.474 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.504 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.505 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.506 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_90 2024-03-14T12:15:23.507 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.521 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_90 2024-03-14T12:15:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.522 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627867 Links: 1 2024-03-14T12:15:23.522 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.522 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.525 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.666287918 +0000 2024-03-14T12:15:23.525 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.666287918 +0000 2024-03-14T12:15:23.525 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.666287918 +0000 2024-03-14T12:15:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.666287918 +0000 2024-03-14T12:15:23.526 INFO:tasks.cephfs.mount:File test_dir/test_file_90 became visible from 1 after 0s 2024-03-14T12:15:23.526 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.526 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.570 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.570 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.573 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.577 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.577 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_91", '"'"'r'"'"') as f: 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.578 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.579 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.586 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.588 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_91 2024-03-14T12:15:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_91 2024-03-14T12:15:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.603 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627868 Links: 1 2024-03-14T12:15:23.603 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.603 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.604 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.669287862 +0000 2024-03-14T12:15:23.604 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.669287862 +0000 2024-03-14T12:15:23.604 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.669287862 +0000 2024-03-14T12:15:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.669287862 +0000 2024-03-14T12:15:23.604 INFO:tasks.cephfs.mount:File test_dir/test_file_91 became visible from 1 after 0s 2024-03-14T12:15:23.604 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.604 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.636 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.650 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_92", '"'"'r'"'"') as f: 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.651 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.652 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.652 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.652 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.652 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.654 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_92 2024-03-14T12:15:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_92 2024-03-14T12:15:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.669 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627869 Links: 1 2024-03-14T12:15:23.669 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.670 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.670 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.674287770 +0000 2024-03-14T12:15:23.670 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.674287770 +0000 2024-03-14T12:15:23.670 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.674287770 +0000 2024-03-14T12:15:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.674287770 +0000 2024-03-14T12:15:23.670 INFO:tasks.cephfs.mount:File test_dir/test_file_92 became visible from 1 after 0s 2024-03-14T12:15:23.670 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.670 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.693 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.707 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_93", '"'"'r'"'"') as f: 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.708 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.709 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.717 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.717 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.719 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_93 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_93 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627870 Links: 1 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.680287658 +0000 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.680287658 +0000 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.680287658 +0000 2024-03-14T12:15:23.734 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.680287658 +0000 2024-03-14T12:15:23.735 INFO:tasks.cephfs.mount:File test_dir/test_file_93 became visible from 1 after 0s 2024-03-14T12:15:23.735 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.735 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.749 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.763 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.763 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_94", '"'"'r'"'"') as f: 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.764 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.782 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.783 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.784 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_94 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_94 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627871 Links: 1 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.684287583 +0000 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.684287583 +0000 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.684287583 +0000 2024-03-14T12:15:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.684287583 +0000 2024-03-14T12:15:23.800 INFO:tasks.cephfs.mount:File test_dir/test_file_94 became visible from 1 after 0s 2024-03-14T12:15:23.800 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.801 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.810 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.824 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.824 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_95", '"'"'r'"'"') as f: 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.825 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.849 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.850 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_95 2024-03-14T12:15:23.851 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.863 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_95 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627872 Links: 1 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.687287528 +0000 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.687287528 +0000 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.687287528 +0000 2024-03-14T12:15:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.687287528 +0000 2024-03-14T12:15:23.866 INFO:tasks.cephfs.mount:File test_dir/test_file_95 became visible from 1 after 0s 2024-03-14T12:15:23.867 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.867 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.915 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.916 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.921 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.921 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.921 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_96", '"'"'r'"'"') as f: 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.922 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.923 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.925 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.926 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_96 2024-03-14T12:15:23.941 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_96 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627873 Links: 1 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.690287472 +0000 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.690287472 +0000 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.690287472 +0000 2024-03-14T12:15:23.942 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.690287472 +0000 2024-03-14T12:15:23.942 INFO:tasks.cephfs.mount:File test_dir/test_file_96 became visible from 1 after 0s 2024-03-14T12:15:23.942 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:23.942 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:23.975 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.989 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:23.989 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:23.989 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:23.989 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_97", '"'"'r'"'"') as f: 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:23.990 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:23.991 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.991 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:23.993 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_97 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_97 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627874 Links: 1 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.695287379 +0000 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.695287379 +0000 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.695287379 +0000 2024-03-14T12:15:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.695287379 +0000 2024-03-14T12:15:24.009 INFO:tasks.cephfs.mount:File test_dir/test_file_97 became visible from 1 after 0s 2024-03-14T12:15:24.009 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:24.010 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:24.038 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_98", '"'"'r'"'"') as f: 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:24.053 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:24.054 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:24.054 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:24.054 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:24.054 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:24.054 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:24.056 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.056 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.058 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_98 2024-03-14T12:15:24.069 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.073 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_98 2024-03-14T12:15:24.073 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:24.073 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627875 Links: 1 2024-03-14T12:15:24.073 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:24.073 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:24.073 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.698287323 +0000 2024-03-14T12:15:24.074 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.698287323 +0000 2024-03-14T12:15:24.074 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.698287323 +0000 2024-03-14T12:15:24.074 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.698287323 +0000 2024-03-14T12:15:24.074 INFO:tasks.cephfs.mount:File test_dir/test_file_98 became visible from 1 after 0s 2024-03-14T12:15:24.074 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:24.074 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:24.087 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.101 DEBUG:teuthology.orchestra.run.smithi164:> stdin-killer -- python3 -c ' 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> import fcntl 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> import sys 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> import time 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_99", '"'"'r'"'"') as f: 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> while True: 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> print("open_background: keeping file open", file=sys.stderr) 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> try: 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> if os.read(0, 4096) == b"": 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> break 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> except BlockingIOError: 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> pass 2024-03-14T12:15:24.102 DEBUG:teuthology.orchestra.run.smithi164:> time.sleep(2) 2024-03-14T12:15:24.103 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-14T12:15:24.104 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.105 DEBUG:teuthology.orchestra.run.smithi164:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_99 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_99 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 2bh/43d Inode: 1099511627876 Links: 1 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-14 12:15:11.701287267 +0000 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-14 12:15:11.701287267 +0000 2024-03-14T12:15:24.121 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-14 12:15:11.701287267 +0000 2024-03-14T12:15:24.122 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-03-14 12:15:11.701287267 +0000 2024-03-14T12:15:24.122 INFO:tasks.cephfs.mount:File test_dir/test_file_99 became visible from 1 after 0s 2024-03-14T12:15:24.122 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:15:24.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-14T12:15:24.122 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.123 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.136 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.152 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.169 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.170 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.170 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.217 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.218 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.228 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.237+0000 7f93caafc640 1 Processor -- start 2024-03-14T12:15:24.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.237+0000 7f93caafc640 1 -- start start 2024-03-14T12:15:24.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.237+0000 7f93caafc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93c415e200 0x7f93c40d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.237+0000 7f93caafc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40d2cf0 con 0x7f93c415de20 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93caafc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40cece0 con 0x7f93c40d28e0 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93caafc640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40cfd80 con 0x7f93c415e200 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c8898640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f93c415de20 0x7f93c40d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37960/0 (socket says 172.21.15.82:37960) 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c8898640 1 -- 172.21.15.82:0/1722078166 learned_addr learned my addr 172.21.15.82:0/1722078166 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c3fff640 1 --2- 172.21.15.82:0/1722078166 >> v2:172.21.15.164:3300/0 conn(0x7f93c415e200 0x7f93c40d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972029397 0 0) 0x7f93b8001110 con 0x7f93c415de20 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b4003520 con 0x7f93c415de20 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1832879943 0 0) 0x7f93b4003520 con 0x7f93c415de20 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 >> v2:172.21.15.164:3300/0 conn(0x7f93c415e200 msgr2=0x7f93c40d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 --2- 172.21.15.82:0/1722078166 >> v2:172.21.15.164:3300/0 conn(0x7f93c415e200 0x7f93c40d23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 >> v1:172.21.15.164:6789/0 conn(0x7f93c40d28e0 legacy=0x7f93c40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40cefc0 con 0x7f93c415de20 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f93b8001840 con 0x7f93c415de20 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 >> v1:172.21.15.82:6789/0 conn(0x7f93c415de20 legacy=0x7f93c40d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93b40039b0 0x7f93b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93b40051c0 0x7f93b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b400c130 con 0x7f93b4004800 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b400c2e0 con 0x7f93b40039b0 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b400c4e0 con 0x7f93b40051c0 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c3fff640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.238+0000 7f93c8898640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93b40039b0 0x7f93b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c9099640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93b40051c0 0x7f93b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c3fff640 1 -- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93b40051c0 msgr2=0x7f93b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c3fff640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93b40051c0 0x7f93b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c3fff640 1 -- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93b40039b0 msgr2=0x7f93b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c3fff640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93b40039b0 0x7f93b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c3fff640 1 -- 172.21.15.82:0/1722078166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4003520 con 0x7f93b4004800 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c8898640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93b40039b0 0x7f93b4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:15:24.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c9099640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93b40051c0 0x7f93b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.239+0000 7f93c9099640 1 -- 172.21.15.82:0/1722078166 reap_dead start 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93c3fff640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93b4004c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009ac0 tx=0x7f93b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546c68b360eb7b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0039070 con 0x7f93b4004800 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b002c6b0 con 0x7f93b4004800 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93c37fe640 1 -- 172.21.15.82:0/1722078166 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0009690 con 0x7f93b4004800 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93caafc640 1 -- 172.21.15.82:0/1722078166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 msgr2=0x7f93b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93caafc640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93b4004c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009ac0 tx=0x7f93b4005830 comp rx=0 tx=0).stop 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93caafc640 1 -- 172.21.15.82:0/1722078166 shutdown_connections 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93caafc640 1 --2- 172.21.15.82:0/1722078166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93b4004c80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93caafc640 1 -- 172.21.15.82:0/1722078166 >> 172.21.15.82:0/1722078166 conn(0x7f93c409d030 msgr2=0x7f93c40d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93caafc640 1 -- 172.21.15.82:0/1722078166 shutdown_connections 2024-03-14T12:15:24.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.240+0000 7f93caafc640 1 -- 172.21.15.82:0/1722078166 wait complete. 2024-03-14T12:15:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.241+0000 7f93caafc640 1 Processor -- start 2024-03-14T12:15:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.241+0000 7f93caafc640 1 -- start start 2024-03-14T12:15:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93caafc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93c4158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93caafc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c409ebe0 0x7f93c4157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93caafc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c4159d30 0x7f93c415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93caafc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40ce940 con 0x7f93b4004800 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93caafc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40cf580 con 0x7f93c409ebe0 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93caafc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40ceac0 con 0x7f93c4159d30 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93c8898640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93c4158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93c8898640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93c4158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41602/0 (socket says 172.21.15.82:41602) 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.242+0000 7f93c8898640 1 -- 172.21.15.82:0/282140884 learned_addr learned my addr 172.21.15.82:0/282140884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c3fff640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c409ebe0 0x7f93c4157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c9099640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c4159d30 0x7f93c415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c8898640 1 -- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c4159d30 msgr2=0x7f93c415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c8898640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c4159d30 0x7f93c415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c8898640 1 -- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c409ebe0 msgr2=0x7f93c4157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c8898640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c409ebe0 0x7f93c4157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c8898640 1 -- 172.21.15.82:0/282140884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c41576a0 con 0x7f93b4004800 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c9099640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c4159d30 0x7f93c415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c8898640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93c4158810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f93b8007110 tx=0x7f93b80075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c17fa640 1 -- 172.21.15.82:0/282140884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b801a070 con 0x7f93b4004800 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c17fa640 1 -- 172.21.15.82:0/282140884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b800ca70 con 0x7f93b4004800 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c4158f60 con 0x7f93b4004800 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93c17fa640 1 -- 172.21.15.82:0/282140884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800ce40 con 0x7f93b4004800 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.243+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c40cb0a0 con 0x7f93b4004800 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.244+0000 7f93c17fa640 1 -- 172.21.15.82:0/282140884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f93b801d070 con 0x7f93b4004800 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.245+0000 7f93c17fa640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93a806bbd0 0x7f93a806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.245+0000 7f93c17fa640 1 -- 172.21.15.82:0/282140884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b808a710 con 0x7f93b4004800 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.245+0000 7f93c3fff640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93a806bbd0 0x7f93a806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.245+0000 7f93c3fff640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93a806bbd0 0x7f93a806e090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f93b0036000 tx=0x7f93b0030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.246+0000 7f93cd7b4740 1 Processor -- start 2024-03-14T12:15:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.246+0000 7f93cd7b4740 1 -- start start 2024-03-14T12:15:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f831a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f831a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8315210 0x5586f8344d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f82e2360 con 0x5586f8318ae0 2024-03-14T12:15:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f827eec0 con 0x5586f8315210 2024-03-14T12:15:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586f8227d30 con 0x5586f8362a60 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f831a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f831a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41618/0 (socket says 172.21.15.82:41618) 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 -- 172.21.15.82:0/423476050 learned_addr learned my addr 172.21.15.82:0/423476050 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c8898640 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f831a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c9099640 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8315210 0x5586f8344d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 -- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 msgr2=0x5586f831a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f831a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 -- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8315210 msgr2=0x5586f8344d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8315210 0x5586f8344d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 -- 172.21.15.82:0/423476050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586f8226bc0 con 0x5586f8318ae0 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.247+0000 7f93c3fff640 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f831a5d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f93b002c320 tx=0x7f93b002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bcaff40e970927 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93ae7fc640 1 -- 172.21.15.82:0/423476050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0043040 con 0x5586f8318ae0 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93ae7fc640 1 -- 172.21.15.82:0/423476050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b003a040 con 0x5586f8318ae0 2024-03-14T12:15:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93ae7fc640 1 -- 172.21.15.82:0/423476050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0039050 con 0x5586f8318ae0 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 -- 172.21.15.82:0/423476050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 msgr2=0x5586f831a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f831a5d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f93b002c320 tx=0x7f93b002e000 comp rx=0 tx=0).stop 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 -- 172.21.15.82:0/423476050 shutdown_connections 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8315210 0x5586f8344d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f831a5d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/423476050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f831a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 -- 172.21.15.82:0/423476050 >> 172.21.15.82:0/423476050 conn(0x5586f8317480 msgr2=0x5586f835fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 -- 172.21.15.82:0/423476050 shutdown_connections 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.248+0000 7f93cd7b4740 1 -- 172.21.15.82:0/423476050 wait complete. 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.249+0000 7f93cd7b4740 1 -- start start 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.249+0000 7f93cd7b4740 1 Processor -- start 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8343a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f8340090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f821d4a0 con 0x5586f8318ae0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f827af60 con 0x5586f8366be0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586f8318ff0 con 0x5586f8362a60 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93cd7b4740 10 client.0 _ll_register_callbacks cb 0x5586f8278d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8343a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8343a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41620/0 (socket says 172.21.15.82:41620) 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 -- 172.21.15.82:0/4011607202 learned_addr learned my addr 172.21.15.82:0/4011607202 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c3fff640 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f8340090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c9099640 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 -- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 msgr2=0x5586f8340090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f8340090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 -- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 msgr2=0x5586f83405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 -- 172.21.15.82:0/4011607202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586f7f933f0 con 0x5586f8318ae0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93c8898640 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8343a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f93b800c650 tx=0x7f93b8058690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c82b1585e89550b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93a37fe640 1 -- 172.21.15.82:0/4011607202 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8070070 con 0x5586f8318ae0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.250+0000 7f93a37fe640 1 -- 172.21.15.82:0/4011607202 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b8088910 con 0x5586f8318ae0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93b003a040 con 0x5586f8318ae0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93a37fe640 1 -- 172.21.15.82:0/4011607202 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b8075050 con 0x5586f8318ae0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 10 client.5328 fetch_fsmap learned FSMap version 15 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5586f805d7d0 con 0x5586f8318ae0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93a37fe640 1 -- 172.21.15.82:0/4011607202 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b003a040 con 0x5586f8318ae0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93a37fe640 1 -- 172.21.15.82:0/4011607202 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f93b8058090 con 0x5586f8318ae0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 10 client.5328 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 10 client.5328 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 20 client.5328 populate_metadata read hostname 'smithi082' 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8370d70 0x5586f8371190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 4 client.5328 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586f828c400 con 0x5586f8370d70 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93c3fff640 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8370d70 0x5586f8371190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93a37fe640 10 client.5328 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.251+0000 7f93c3fff640 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8370d70 0x5586f8371190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80deb6fb3805a549 server_cookie=8d63564bb3e1f0e5 in_seq=0 out_seq=0 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93a37fe640 1 -- 172.21.15.82:0/4011607202 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586f828c400 con 0x5586f8370d70 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93a37fe640 10 client.5328 handle_command_reply: tid=0 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 client.5328 shutdown 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 msgr2=0x5586f8343a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8343a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f93b800c650 tx=0x7f93b8058690 comp rx=0 tx=0).stop 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 shutdown_connections 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8370d70 0x5586f8371190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f8340090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93a37fe640 0 client.5328 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/4011607202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8343a70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 >> 172.21.15.82:0/4011607202 conn(0x5586f8315210 msgr2=0x5586f8319e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 shutdown_connections 2024-03-14T12:15:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 1 -- 172.21.15.82:0/4011607202 wait complete. 2024-03-14T12:15:24.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.252+0000 7f93cd7b4740 20 client.5328 trim_cache size 0 max 16384 2024-03-14T12:15:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 Processor -- start 2024-03-14T12:15:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 -- start start 2024-03-14T12:15:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8366150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f835fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c42008c0 con 0x5586f8318ae0 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c41ffac0 con 0x5586f8366be0 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40cb410 con 0x5586f8362a60 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93c8898640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8366150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93c8898640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8366150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41634/0 (socket says 172.21.15.82:41634) 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.264+0000 7f93c8898640 1 -- 172.21.15.82:0/2261398874 learned_addr learned my addr 172.21.15.82:0/2261398874 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c9099640 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c3fff640 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f835fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c8898640 1 -- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 msgr2=0x5586f835fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c8898640 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f835fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c8898640 1 -- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 msgr2=0x5586f83600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c8898640 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c8898640 1 -- 172.21.15.82:0/2261398874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586f81ff110 con 0x5586f8318ae0 2024-03-14T12:15:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93c8898640 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8366150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f93b8061440 tx=0x7f93b8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8874ed5347412f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93adffb640 1 -- 172.21.15.82:0/2261398874 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8070040 con 0x5586f8318ae0 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93adffb640 1 -- 172.21.15.82:0/2261398874 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b805cce0 con 0x5586f8318ae0 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93adffb640 1 -- 172.21.15.82:0/2261398874 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8058780 con 0x5586f8318ae0 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2261398874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 msgr2=0x5586f8366150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8366150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f93b8061440 tx=0x7f93b8015000 comp rx=0 tx=0).stop 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2261398874 shutdown_connections 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8366be0 0x5586f83600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8362a60 0x5586f835fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.265+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2261398874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8318ae0 0x5586f8366150 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.266+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2261398874 >> 172.21.15.82:0/2261398874 conn(0x5586f8315210 msgr2=0x5586f8318f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.266+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2261398874 shutdown_connections 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.266+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2261398874 wait complete. 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.266+0000 7f93cd7b4740 1 -- start start 2024-03-14T12:15:24.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.266+0000 7f93cd7b4740 1 Processor -- start 2024-03-14T12:15:24.268 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8318ae0 0x5586f8379bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8362a60 0x5586f83612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93cd7b4740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8366be0 0x5586f83b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f8344480 con 0x5586f8362a60 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c41ff4c0 con 0x5586f8318ae0 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93cd7b4740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586f8284140 con 0x5586f8366be0 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93cd7b4740 10 client.0 _ll_register_callbacks cb 0x7f93b40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8362a60 0x5586f83612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8362a60 0x5586f83612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41640/0 (socket says 172.21.15.82:41640) 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 -- 172.21.15.82:0/2554793529 learned_addr learned my addr 172.21.15.82:0/2554793529 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c8898640 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8318ae0 0x5586f8379bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c9099640 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8366be0 0x5586f83b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 -- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8366be0 msgr2=0x5586f83b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8366be0 0x5586f83b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 -- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8318ae0 msgr2=0x5586f8379bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8318ae0 0x5586f8379bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.267+0000 7f93c3fff640 1 -- 172.21.15.82:0/2554793529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586f83cc5b0 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93c3fff640 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8362a60 0x5586f83612f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93b0004650 tx=0x7f93b002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798d3e75e9cc3509 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93a37fe640 1 -- 172.21.15.82:0/2554793529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0039040 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93a37fe640 1 -- 172.21.15.82:0/2554793529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b0058d00 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93b805cce0 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93a37fe640 1 -- 172.21.15.82:0/2554793529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b0043040 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 10 client.5334 fetch_fsmap learned FSMap version 15 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5586f8340070 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93a37fe640 1 -- 172.21.15.82:0/2554793529 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b805cce0 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93a37fe640 1 -- 172.21.15.82:0/2554793529 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f93b0005990 con 0x5586f8362a60 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 10 client.5334 fetch_fsmap finished waiting for FSMap version 15 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 10 client.5334 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-14T12:15:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 20 client.5334 populate_metadata read hostname 'smithi082' 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8340650 0x5586f83474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 4 client.5334 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 --> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x5586f8286750 con 0x5586f8340650 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.268+0000 7f93c8898640 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8340650 0x5586f83474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93a37fe640 10 client.5334 ms_handle_connect on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93c8898640 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8340650 0x5586f83474e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b673f138147f79 server_cookie=147fa0a4bc193560 in_seq=0 out_seq=0 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93a37fe640 1 -- 172.21.15.82:0/2554793529 <== mds.0 v2:172.21.15.82:6836/682556190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26941 (crc 0 0 0) 0x5586f8286750 con 0x5586f8340650 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93a37fe640 10 client.5334 handle_command_reply: tid=0 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 client.5334 shutdown 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8362a60 msgr2=0x5586f83612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8362a60 0x5586f83612f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93b0004650 tx=0x7f93b002e070 comp rx=0 tx=0).stop 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 shutdown_connections 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:6836/682556190,v1:172.21.15.82:6837/682556190] conn(0x5586f8340650 0x5586f83474e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5586f8366be0 0x5586f83b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586f8362a60 0x5586f83612f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 --2- 172.21.15.82:0/2554793529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5586f8318ae0 0x5586f8379bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93a37fe640 0 client.5334 ms_handle_reset on v2:172.21.15.82:6836/682556190 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 >> 172.21.15.82:0/2554793529 conn(0x5586f8315210 msgr2=0x5586f8365590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 shutdown_connections 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.269+0000 7f93cd7b4740 1 -- 172.21.15.82:0/2554793529 wait complete. 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93cd7b4740 20 client.5334 trim_cache size 0 max 16384 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:15:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:15:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1044, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1043, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 257369, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 4722070, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 13, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 7, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.069307845, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.028328836, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.016206329, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.019031303, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1042, 2024-03-14T12:15:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.123156118, 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000118192 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6, 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000591, 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000098 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 3105, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 4110, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 836083, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 1937404, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.239575100, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.160464782, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.111930388, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.001411520, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4109, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.056514337, 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000013753 2024-03-14T12:15:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6481, 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000778646, 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000120 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 99712 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 255, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 233, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 169987, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 1091943, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.030164266, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.012135606, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.021288617, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.002602291, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 233, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.066283089, 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000284476 2024-03-14T12:15:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 304, 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000044738, 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000147 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 114144, 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 15168 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:15:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2063, 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.639869421, 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000310164 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000016539, 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000008269 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:15:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 1149, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 1149, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1149, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.579690599, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001374839 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:15:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 1011, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 1011, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 1011, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 102, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 104, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 1297, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 1295, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:15:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 1000, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 2890, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:15:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 59, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 1036, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 1065, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 1000, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 6, 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 3 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:15:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:15:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "5061" 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 2 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4733", 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 10032, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 19, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 1001, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 2, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 2, 2024-03-14T12:15:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 2, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 1000, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 2890 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4978", 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 704, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 17, 2024-03-14T12:15:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 377, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 75, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 78, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 78, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 78, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:15:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 3014, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 3014, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 4, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 4, 2024-03-14T12:15:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 11018128, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1010, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.658064676, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001641648 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1014, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 1014, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 13, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 13, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 1011, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 1011, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 104, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1104, 2024-03-14T12:15:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1000, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 60400, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 1190, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 1149, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 1004, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:15:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1000, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.544801480, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000544801 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 46, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.893055369, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.019414247 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 101, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.138811896, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001374375 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:15:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:15:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002766425, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002766425 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:15:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:15:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000255429, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000255429 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 1016, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 508, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 21, 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:15:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:15:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:15:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 270888, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 1104, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 1232, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 14, 2024-03-14T12:15:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 3093425, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 38394, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:15:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 1056, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 6917554, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:15:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 1056, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1056, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.887964820, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001787845 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1056, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1088 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 1056, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 1056, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:15:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 1010, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 8, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 7, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:15:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:15:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 34, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:15:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 27, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 3 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:15:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:15:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:15:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 4396, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 862374, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 4396, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 4396, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 862374, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 1056, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 6917554, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1056, 2024-03-14T12:15:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 6917554, 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:15:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 1056, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 1056, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1056, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1056, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:15:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x56140c7886e0": [ 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3014, 2024-03-14T12:15:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 6823824, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3014, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1010, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 6823824, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:15:24.329 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93a806bbd0 msgr2=0x7f93a806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93a806bbd0 0x7f93a806e090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f93b0036000 tx=0x7f93b0030000 comp rx=0 tx=0).stop 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 msgr2=0x7f93c4158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93c4158810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f93b8007110 tx=0x7f93b80075d0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 shutdown_connections 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93a806bbd0 0x7f93a806e090 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c4159d30 0x7f93c415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c409ebe0 0x7f93c4157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 --2- 172.21.15.82:0/282140884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93b4004800 0x7f93c4158810 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 >> 172.21.15.82:0/282140884 conn(0x7f93c409d030 msgr2=0x7f93c40b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 shutdown_connections 2024-03-14T12:15:24.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.270+0000 7f93caafc640 1 -- 172.21.15.82:0/282140884 wait complete. 2024-03-14T12:15:24.332 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:24.332 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:24.333 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.333 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.333 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.340 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.347 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.349 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:24.349 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.404 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:15:24.406 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.413 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.452 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.469 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.517 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.517 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.536 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.545+0000 7feb57d69640 1 Processor -- start 2024-03-14T12:15:24.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.545+0000 7feb57d69640 1 -- start start 2024-03-14T12:15:24.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb57d69640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb500d0760 0x7feb500d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb57d69640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 0x7feb500d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb57d69640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb500d1080 con 0x7feb500d1da0 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb57d69640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb500cf8c0 con 0x7feb500d0760 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb57d69640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb500cfba0 con 0x7feb500cf370 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb56306640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feb500d1da0 0x7feb500d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37966/0 (socket says 172.21.15.82:37966) 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb56306640 1 -- 172.21.15.82:0/2226904180 learned_addr learned my addr 172.21.15.82:0/2226904180 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb55b05640 1 --2- 172.21.15.82:0/2226904180 >> v2:172.21.15.164:3300/0 conn(0x7feb500d0760 0x7feb500d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb55304640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 0x7feb500d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764392927 0 0) 0x7feb500d1080 con 0x7feb500d1da0 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb3c003520 con 0x7feb500d1da0 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3561187059 0 0) 0x7feb3c003520 con 0x7feb500d1da0 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 msgr2=0x7feb500d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 0x7feb500d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 >> v2:172.21.15.164:3300/0 conn(0x7feb500d0760 msgr2=0x7feb500d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 --2- 172.21.15.82:0/2226904180 >> v2:172.21.15.164:3300/0 conn(0x7feb500d0760 0x7feb500d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500b6c40 con 0x7feb500d1da0 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7feb480019f0 con 0x7feb500d1da0 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 >> v1:172.21.15.82:6789/0 conn(0x7feb500d1da0 legacy=0x7feb500d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c0039b0 0x7feb3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 0x7feb3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb3c004d40 0x7feb3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500cf8c0 con 0x7feb3c004d40 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500cfba0 con 0x7feb3c0039b0 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.546+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c00b9c0 con 0x7feb500cf370 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb56306640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb3c004d40 0x7feb3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb55b05640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c0039b0 0x7feb3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb55304640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 0x7feb3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb56306640 1 -- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 msgr2=0x7feb3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb56306640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500cf370 0x7feb3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb56306640 1 -- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c0039b0 msgr2=0x7feb3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb56306640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c0039b0 0x7feb3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb56306640 1 -- 172.21.15.82:0/2226904180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c003520 con 0x7feb3c004d40 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.547+0000 7feb56306640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb3c004d40 0x7feb3c0051c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feb4800a870 tx=0x7feb3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e57cb84f29e3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800a4c0 con 0x7feb3c004d40 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48034630 con 0x7feb3c004d40 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb54b03640 1 -- 172.21.15.82:0/2226904180 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48034a20 con 0x7feb3c004d40 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb57d69640 1 -- 172.21.15.82:0/2226904180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb3c004d40 msgr2=0x7feb3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb57d69640 1 --2- 172.21.15.82:0/2226904180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb3c004d40 0x7feb3c0051c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feb4800a870 tx=0x7feb3c005830 comp rx=0 tx=0).stop 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb56306640 1 -- 172.21.15.82:0/2226904180 reap_dead start 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb57d69640 1 -- 172.21.15.82:0/2226904180 shutdown_connections 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb57d69640 1 -- 172.21.15.82:0/2226904180 >> 172.21.15.82:0/2226904180 conn(0x7feb5009d030 msgr2=0x7feb500d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb57d69640 1 -- 172.21.15.82:0/2226904180 shutdown_connections 2024-03-14T12:15:24.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.548+0000 7feb57d69640 1 -- 172.21.15.82:0/2226904180 wait complete. 2024-03-14T12:15:24.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.549+0000 7feb57d69640 1 Processor -- start 2024-03-14T12:15:24.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.549+0000 7feb57d69640 1 -- start start 2024-03-14T12:15:24.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb57d69640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c004d40 0x7feb50153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb57d69640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500c8e70 0x7feb500d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb57d69640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb50153c20 0x7feb500d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb57d69640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb501617a0 con 0x7feb50153c20 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb57d69640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500b7200 con 0x7feb3c004d40 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb57d69640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb50161920 con 0x7feb500c8e70 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb50153c20 0x7feb500d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb50153c20 0x7feb500d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41654/0 (socket says 172.21.15.82:41654) 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 -- 172.21.15.82:0/1917132393 learned_addr learned my addr 172.21.15.82:0/1917132393 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb55b05640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c004d40 0x7feb50153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb55304640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500c8e70 0x7feb500d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 -- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500c8e70 msgr2=0x7feb500d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500c8e70 0x7feb500d40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 -- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c004d40 msgr2=0x7feb50153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c004d40 0x7feb50153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.550+0000 7feb56306640 1 -- 172.21.15.82:0/1917132393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500c93e0 con 0x7feb50153c20 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.551+0000 7feb56306640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb50153c20 0x7feb500d45e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb4800a990 tx=0x7feb500ce7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.551+0000 7feb467fc640 1 -- 172.21.15.82:0/1917132393 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48039040 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.551+0000 7feb467fc640 1 -- 172.21.15.82:0/1917132393 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48043070 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.551+0000 7feb467fc640 1 -- 172.21.15.82:0/1917132393 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48002030 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.551+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb500c96c0 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.551+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb50153940 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.552+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5004f050 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.552+0000 7feb467fc640 1 -- 172.21.15.82:0/1917132393 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7feb48034630 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.553+0000 7feb467fc640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb3406bc00 0x7feb3406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.553+0000 7feb467fc640 1 -- 172.21.15.82:0/1917132393 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb480ac760 con 0x7feb50153c20 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.553+0000 7feb55b05640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb3406bc00 0x7feb3406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.553+0000 7feb55b05640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb3406bc00 0x7feb3406e0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009bc0 tx=0x7feb4c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.557+0000 7feb467fc640 1 -- 172.21.15.82:0/1917132393 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7feb4807b3a0 con 0x7feb50153c20 2024-03-14T12:15:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.601 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.631 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.631 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.666 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.678 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.728+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb500d0540 con 0x7feb50153c20 2024-03-14T12:15:24.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.729+0000 7feb467fc640 1 -- 172.21.15.82:0/1917132393 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16657 (secure 0 0 0) 0x7feb4807aca0 con 0x7feb50153c20 2024-03-14T12:15:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:24.729 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:50.791329+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb3406bc00 msgr2=0x7feb3406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb3406bc00 0x7feb3406e0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009bc0 tx=0x7feb4c009190 comp rx=0 tx=0).stop 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb50153c20 msgr2=0x7feb500d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb50153c20 0x7feb500d45e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb4800a990 tx=0x7feb500ce7b0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 shutdown_connections 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb50153c20 0x7feb500d45e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb3406bc00 0x7feb3406e0c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb500c8e70 0x7feb500d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 --2- 172.21.15.82:0/1917132393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb3c004d40 0x7feb50153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 >> 172.21.15.82:0/1917132393 conn(0x7feb5009d030 msgr2=0x7feb500b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 shutdown_connections 2024-03-14T12:15:24.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.731+0000 7feb57d69640 1 -- 172.21.15.82:0/1917132393 wait complete. 2024-03-14T12:15:24.732 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:14:47.564085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:14:48.095794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:15:24.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:15:24.743 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.743 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.790 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.809 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.846+0000 7f5438977640 1 Processor -- start 2024-03-14T12:15:24.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.846+0000 7f5438977640 1 -- start start 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5438977640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f543415dd70 0x7f543406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5438977640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54340d1b00 0x7f54340d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5438977640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5434070250 con 0x7f543415e150 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5438977640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54340d2490 con 0x7f54340d1b00 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5438977640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543406ea20 con 0x7f543415dd70 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5432d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54340d1b00 0x7f54340d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5432d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54340d1b00 0x7f54340d1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41668/0 (socket says 172.21.15.82:41668) 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5432d9d640 1 -- 172.21.15.82:0/2910979759 learned_addr learned my addr 172.21.15.82:0/2910979759 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f543259c640 1 --2- 172.21.15.82:0/2910979759 >> v2:172.21.15.164:3300/0 conn(0x7f543415dd70 0x7f543406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5432d9d640 1 -- 172.21.15.82:0/2910979759 >> v2:172.21.15.164:3300/0 conn(0x7f543415dd70 msgr2=0x7f543406fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5432d9d640 1 --2- 172.21.15.82:0/2910979759 >> v2:172.21.15.164:3300/0 conn(0x7f543415dd70 0x7f543406fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5432d9d640 1 -- 172.21.15.82:0/2910979759 >> v1:172.21.15.164:6789/0 conn(0x7f543415e150 legacy=0x7f54340b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.847+0000 7f5432d9d640 1 -- 172.21.15.82:0/2910979759 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543406ec70 con 0x7f54340d1b00 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.848+0000 7f5432d9d640 1 --2- 172.21.15.82:0/2910979759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54340d1b00 0x7f54340d1f50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f54340b7c80 tx=0x7f542c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aada56791e0e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.848+0000 7f5431d9b640 1 -- 172.21.15.82:0/2910979759 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c02ca80 con 0x7f54340d1b00 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.848+0000 7f5431d9b640 1 -- 172.21.15.82:0/2910979759 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c02cc20 con 0x7f54340d1b00 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.848+0000 7f5431d9b640 1 -- 172.21.15.82:0/2910979759 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0343f0 con 0x7f54340d1b00 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.848+0000 7f5438977640 1 -- 172.21.15.82:0/2910979759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54340d1b00 msgr2=0x7f54340d1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.849+0000 7f5438977640 1 --2- 172.21.15.82:0/2910979759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54340d1b00 0x7f54340d1f50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f54340b7c80 tx=0x7f542c02bee0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.849+0000 7f5438977640 1 -- 172.21.15.82:0/2910979759 shutdown_connections 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.849+0000 7f5438977640 1 --2- 172.21.15.82:0/2910979759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54340d1b00 0x7f54340d1f50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.849+0000 7f5438977640 1 --2- 172.21.15.82:0/2910979759 >> v2:172.21.15.164:3300/0 conn(0x7f543415dd70 0x7f543406fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.849+0000 7f5438977640 1 -- 172.21.15.82:0/2910979759 >> 172.21.15.82:0/2910979759 conn(0x7f5434072ec0 msgr2=0x7f543409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.849+0000 7f5438977640 1 -- 172.21.15.82:0/2910979759 shutdown_connections 2024-03-14T12:15:24.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.849+0000 7f5438977640 1 -- 172.21.15.82:0/2910979759 wait complete. 2024-03-14T12:15:24.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.850+0000 7f5438977640 1 Processor -- start 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.850+0000 7f5438977640 1 -- start start 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54340d1b00 0x7f54340cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f543415dd70 0x7f5434155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f543415e150 0x7f5434160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543406f230 con 0x7f543415e150 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543406f880 con 0x7f543415dd70 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543406f510 con 0x7f54340d1b00 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f543415e150 0x7f5434160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f543415e150 0x7f5434160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41678/0 (socket says 172.21.15.82:41678) 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 -- 172.21.15.82:0/1957791467 learned_addr learned my addr 172.21.15.82:0/1957791467 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f543259c640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54340d1b00 0x7f54340cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f542bfff640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f543415dd70 0x7f5434155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 -- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54340d1b00 msgr2=0x7f54340cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54340d1b00 0x7f54340cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 -- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f543415dd70 msgr2=0x7f5434155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f543415dd70 0x7f5434155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 -- 172.21.15.82:0/1957791467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54340cdcd0 con 0x7f543415e150 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f542bfff640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f543415dd70 0x7f5434155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5432d9d640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f543415e150 0x7f5434160e30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f542c036040 tx=0x7f542c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f542affd640 1 -- 172.21.15.82:0/1957791467 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c043020 con 0x7f543415e150 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f542affd640 1 -- 172.21.15.82:0/1957791467 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c03e450 con 0x7f543415e150 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5434155430 con 0x7f543415e150 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f542affd640 1 -- 172.21.15.82:0/1957791467 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c03e860 con 0x7f543415e150 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.851+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54340cd970 con 0x7f543415e150 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.853+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543404f050 con 0x7f543415e150 2024-03-14T12:15:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.853+0000 7f542affd640 1 -- 172.21.15.82:0/1957791467 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f542c046020 con 0x7f543415e150 2024-03-14T12:15:24.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.853+0000 7f542affd640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f541406bc00 0x7f541406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:24.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.853+0000 7f542affd640 1 -- 172.21.15.82:0/1957791467 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f542c0ad0c0 con 0x7f543415e150 2024-03-14T12:15:24.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.853+0000 7f543259c640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f541406bc00 0x7f541406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:24.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.854+0000 7f543259c640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f541406bc00 0x7f541406e0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f543415f060 tx=0x7f5424008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:24.856 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:24.857+0000 7f542affd640 1 -- 172.21.15.82:0/1957791467 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f542c04d070 con 0x7f543415e150 2024-03-14T12:15:24.857 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.873 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.910 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.939 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.969 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:24.974 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.004 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.021 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.027+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54340d29e0 con 0x7f543415e150 2024-03-14T12:15:25.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.028+0000 7f542affd640 1 -- 172.21.15.82:0/1957791467 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f542c07bbb0 con 0x7f543415e150 2024-03-14T12:15:25.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f541406bc00 msgr2=0x7f541406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f541406bc00 0x7f541406e0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f543415f060 tx=0x7f5424008040 comp rx=0 tx=0).stop 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f543415e150 msgr2=0x7f5434160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f543415e150 0x7f5434160e30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f542c036040 tx=0x7f542c004a40 comp rx=0 tx=0).stop 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 shutdown_connections 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f543415e150 0x7f5434160e30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f543415dd70 0x7f5434155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f541406bc00 0x7f541406e0c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 --2- 172.21.15.82:0/1957791467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54340d1b00 0x7f54340cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 >> 172.21.15.82:0/1957791467 conn(0x7f5434072ec0 msgr2=0x7f54340b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:25.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.029+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 shutdown_connections 2024-03-14T12:15:25.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.030+0000 7f5438977640 1 -- 172.21.15.82:0/1957791467 wait complete. 2024-03-14T12:15:25.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:15:25.068 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.083 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.088 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.133 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.143+0000 7f6f2c2c6640 1 Processor -- start 2024-03-14T12:15:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.143+0000 7f6f2c2c6640 1 -- start start 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2c2c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f2415c0e0 0x7f6f240d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2c2c6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f241608c0 0x7f6f240cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2c2c6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f2415fd00 0x7f6f241601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2c2c6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f240d0ba0 con 0x7f6f2415fd00 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2c2c6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f240d01d0 con 0x7f6f241608c0 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2c2c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240cf8a0 con 0x7f6f2415c0e0 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2a863640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f2415fd00 0x7f6f241601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2a863640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f2415fd00 0x7f6f241601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41688/0 (socket says 172.21.15.82:41688) 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.144+0000 7f6f2a863640 1 -- 172.21.15.82:0/3653765125 learned_addr learned my addr 172.21.15.82:0/3653765125 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f29861640 1 --2- 172.21.15.82:0/3653765125 >> v2:172.21.15.164:3300/0 conn(0x7f6f241608c0 0x7f6f240cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f2a062640 1 --2- 172.21.15.82:0/3653765125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f2415c0e0 0x7f6f240d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f2a863640 1 -- 172.21.15.82:0/3653765125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f2415c0e0 msgr2=0x7f6f240d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f2a863640 1 --2- 172.21.15.82:0/3653765125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f2415c0e0 0x7f6f240d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f2a863640 1 -- 172.21.15.82:0/3653765125 >> v2:172.21.15.164:3300/0 conn(0x7f6f241608c0 msgr2=0x7f6f240cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f2a863640 1 --2- 172.21.15.82:0/3653765125 >> v2:172.21.15.164:3300/0 conn(0x7f6f241608c0 0x7f6f240cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f2a863640 1 -- 172.21.15.82:0/3653765125 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240cfb80 con 0x7f6f2415fd00 2024-03-14T12:15:25.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.145+0000 7f6f2a863640 1 --2- 172.21.15.82:0/3653765125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415fd00 0x7f6f241601b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6f24069d00 tx=0x7f6f1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6d1d1fb77676c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f29060640 1 -- 172.21.15.82:0/3653765125 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c035020 con 0x7f6f2415fd00 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f29060640 1 -- 172.21.15.82:0/3653765125 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c031870 con 0x7f6f2415fd00 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f29060640 1 -- 172.21.15.82:0/3653765125 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c031c40 con 0x7f6f2415fd00 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3653765125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415fd00 msgr2=0x7f6f241601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3653765125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415fd00 0x7f6f241601b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6f24069d00 tx=0x7f6f1c02b910 comp rx=0 tx=0).stop 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3653765125 shutdown_connections 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3653765125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415fd00 0x7f6f241601b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3653765125 >> v2:172.21.15.164:3300/0 conn(0x7f6f241608c0 0x7f6f240cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3653765125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f2415c0e0 0x7f6f240d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3653765125 >> 172.21.15.82:0/3653765125 conn(0x7f6f240b3680 msgr2=0x7f6f240b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3653765125 shutdown_connections 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.146+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3653765125 wait complete. 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.147+0000 7f6f2c2c6640 1 Processor -- start 2024-03-14T12:15:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.147+0000 7f6f2c2c6640 1 -- start start 2024-03-14T12:15:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2c2c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415c0e0 0x7f6f24158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2c2c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f2415fd00 0x7f6f2406e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2c2c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f241608c0 0x7f6f2406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2c2c6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2415c640 con 0x7f6f2415c0e0 2024-03-14T12:15:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2c2c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2415c4c0 con 0x7f6f2415fd00 2024-03-14T12:15:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2c2c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240cd790 con 0x7f6f241608c0 2024-03-14T12:15:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415c0e0 0x7f6f24158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415c0e0 0x7f6f24158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41704/0 (socket says 172.21.15.82:41704) 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 -- 172.21.15.82:0/3174366446 learned_addr learned my addr 172.21.15.82:0/3174366446 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f29861640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f2415fd00 0x7f6f2406e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a863640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f241608c0 0x7f6f2406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 -- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f241608c0 msgr2=0x7f6f2406ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f241608c0 0x7f6f2406ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 -- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f2415fd00 msgr2=0x7f6f2406e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f2415fd00 0x7f6f2406e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.148+0000 7f6f2a062640 1 -- 172.21.15.82:0/3174366446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240ca650 con 0x7f6f2415c0e0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.149+0000 7f6f2a062640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415c0e0 0x7f6f24158350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6f20007e60 tx=0x7f6f2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.149+0000 7f6f1affd640 1 -- 172.21.15.82:0/3174366446 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20014620 con 0x7f6f2415c0e0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.149+0000 7f6f1affd640 1 -- 172.21.15.82:0/3174366446 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f20019070 con 0x7f6f2415c0e0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.149+0000 7f6f1affd640 1 -- 172.21.15.82:0/3174366446 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20014a00 con 0x7f6f2415c0e0 2024-03-14T12:15:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.149+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f240ca930 con 0x7f6f2415c0e0 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.149+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f240caef0 con 0x7f6f2415c0e0 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.150+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2404f050 con 0x7f6f2415c0e0 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.150+0000 7f6f1affd640 1 -- 172.21.15.82:0/3174366446 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6f20014ba0 con 0x7f6f2415c0e0 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.150+0000 7f6f1affd640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0806bbd0 0x7f6f0806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.151+0000 7f6f1affd640 1 -- 172.21.15.82:0/3174366446 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f2008b1d0 con 0x7f6f2415c0e0 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.151+0000 7f6f29861640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0806bbd0 0x7f6f0806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.151+0000 7f6f29861640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0806bbd0 0x7f6f0806e090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f2406f5a0 tx=0x7f6f14008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.152 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.155+0000 7f6f1affd640 1 -- 172.21.15.82:0/3174366446 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6f20059670 con 0x7f6f2415c0e0 2024-03-14T12:15:25.196 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.199 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.232 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.248 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.256 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.297 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.298 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.312 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.315 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.324+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f2406fd10 con 0x7f6f2415c0e0 2024-03-14T12:15:25.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.324+0000 7f6f1affd640 1 -- 172.21.15.82:0/3174366446 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f2005d710 con 0x7f6f2415c0e0 2024-03-14T12:15:25.324 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0806bbd0 msgr2=0x7f6f0806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0806bbd0 0x7f6f0806e090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f2406f5a0 tx=0x7f6f14008040 comp rx=0 tx=0).stop 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415c0e0 msgr2=0x7f6f24158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415c0e0 0x7f6f24158350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6f20007e60 tx=0x7f6f2000e4b0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 shutdown_connections 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0806bbd0 0x7f6f0806e090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f241608c0 0x7f6f2406ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f2415fd00 0x7f6f2406e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 --2- 172.21.15.82:0/3174366446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2415c0e0 0x7f6f24158350 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 >> 172.21.15.82:0/3174366446 conn(0x7f6f240b3680 msgr2=0x7f6f240b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:25.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 shutdown_connections 2024-03-14T12:15:25.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.326+0000 7f6f2c2c6640 1 -- 172.21.15.82:0/3174366446 wait complete. 2024-03-14T12:15:25.336 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:15:25.346 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.377 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.377 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.381 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.411 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.440+0000 7fa2c49bf640 1 Processor -- start 2024-03-14T12:15:25.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.440+0000 7fa2c49bf640 1 -- start start 2024-03-14T12:15:25.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2c49bf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015c2f0 0x7fa2c00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2c49bf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa2c00b1e00 0x7fa2c00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2c49bf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00670c0 con 0x7fa2c015bf10 2024-03-14T12:15:25.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2c49bf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c00b62e0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2c49bf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c006c740 con 0x7fa2c015c2f0 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2bed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa2c00b1e00 0x7fa2c00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2bed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa2c00b1e00 0x7fa2c00b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41710/0 (socket says 172.21.15.82:41710) 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2bed9d640 1 -- 172.21.15.82:0/4018712592 learned_addr learned my addr 172.21.15.82:0/4018712592 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2b7fff640 1 --2- 172.21.15.82:0/4018712592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015c2f0 0x7fa2c00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2bed9d640 1 -- 172.21.15.82:0/4018712592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015c2f0 msgr2=0x7fa2c00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2bed9d640 1 --2- 172.21.15.82:0/4018712592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015c2f0 0x7fa2c00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2bed9d640 1 -- 172.21.15.82:0/4018712592 >> v1:172.21.15.164:6789/0 conn(0x7fa2c015bf10 legacy=0x7fa2c00af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.441+0000 7fa2bed9d640 1 -- 172.21.15.82:0/4018712592 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c006c130 con 0x7fa2c00b1e00 2024-03-14T12:15:25.442 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2bed9d640 1 --2- 172.21.15.82:0/4018712592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c00b2280 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009e50 tx=0x7fa2b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8162329926493b7c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2bdd9b640 1 -- 172.21.15.82:0/4018712592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b802ca70 con 0x7fa2c00b1e00 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2bdd9b640 1 -- 172.21.15.82:0/4018712592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b802cc10 con 0x7fa2c00b1e00 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2bdd9b640 1 -- 172.21.15.82:0/4018712592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b80343f0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 -- 172.21.15.82:0/4018712592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 msgr2=0x7fa2c00b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/4018712592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c00b2280 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009e50 tx=0x7fa2b802bed0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 -- 172.21.15.82:0/4018712592 shutdown_connections 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/4018712592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c00b2280 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/4018712592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015c2f0 0x7fa2c00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 -- 172.21.15.82:0/4018712592 >> 172.21.15.82:0/4018712592 conn(0x7fa2c00b4fd0 msgr2=0x7fa2c00b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 -- 172.21.15.82:0/4018712592 shutdown_connections 2024-03-14T12:15:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.442+0000 7fa2c49bf640 1 -- 172.21.15.82:0/4018712592 wait complete. 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 Processor -- start 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 -- start start 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c01453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015bf10 0x7fa2c009a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2c015c2f0 0x7fa2c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0145b60 con 0x7fa2c00b1e00 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c006c3d0 con 0x7fa2c015c2f0 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.444+0000 7fa2c49bf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0066a20 con 0x7fa2c015bf10 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c01453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c01453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41720/0 (socket says 172.21.15.82:41720) 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 -- 172.21.15.82:0/1236389169 learned_addr learned my addr 172.21.15.82:0/1236389169 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 -- 172.21.15.82:0/1236389169 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015bf10 msgr2=0x7fa2c009a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015bf10 0x7fa2c009a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 -- 172.21.15.82:0/1236389169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2c015c2f0 msgr2=0x7fa2c006d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2c015c2f0 0x7fa2c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 -- 172.21.15.82:0/1236389169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c015e800 con 0x7fa2c00b1e00 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2be59c640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c01453d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c400 tx=0x7fa2b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2b6ffd640 1 -- 172.21.15.82:0/1236389169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00052d0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2b6ffd640 1 -- 172.21.15.82:0/1236389169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b0005470 con 0x7fa2c00b1e00 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c015eae0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2b6ffd640 1 -- 172.21.15.82:0/1236389169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0007260 con 0x7fa2c00b1e00 2024-03-14T12:15:25.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.445+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c015f0a0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.448 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.448 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.446+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c015f2a0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.446+0000 7fa2b6ffd640 1 -- 172.21.15.82:0/1236389169 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa2b0007400 con 0x7fa2c00b1e00 2024-03-14T12:15:25.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.447+0000 7fa2b6ffd640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa2a006bbd0 0x7fa2a006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.447+0000 7fa2b6ffd640 1 -- 172.21.15.82:0/1236389169 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2b0089fb0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.447+0000 7fa2b7fff640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa2a006bbd0 0x7fa2a006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.447+0000 7fa2b7fff640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa2a006bbd0 0x7fa2a006e090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006fd0 tx=0x7fa2a8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.451+0000 7fa2b6ffd640 1 -- 172.21.15.82:0/1236389169 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa2c015f2a0 con 0x7fa2c00b1e00 2024-03-14T12:15:25.460 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.506 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.523 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.572 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.573 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.575 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.588 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.623+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2c004f050 con 0x7fa2c00b1e00 2024-03-14T12:15:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.623+0000 7fa2b6ffd640 1 -- 172.21.15.82:0/1236389169 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2b0058b30 con 0x7fa2c00b1e00 2024-03-14T12:15:25.622 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:15:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa2a006bbd0 msgr2=0x7fa2a006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa2a006bbd0 0x7fa2a006e090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006fd0 tx=0x7fa2a8008040 comp rx=0 tx=0).stop 2024-03-14T12:15:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 msgr2=0x7fa2c01453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c01453d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c400 tx=0x7fa2b000c8c0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 shutdown_connections 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2c015c2f0 0x7fa2c006d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa2a006bbd0 0x7fa2a006e090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2c015bf10 0x7fa2c009a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 --2- 172.21.15.82:0/1236389169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c00b1e00 0x7fa2c01453d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 >> 172.21.15.82:0/1236389169 conn(0x7fa2c00b4fd0 msgr2=0x7fa2c00af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 shutdown_connections 2024-03-14T12:15:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.625+0000 7fa2c49bf640 1 -- 172.21.15.82:0/1236389169 wait complete. 2024-03-14T12:15:25.635 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:15:25.638 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.653 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.653 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.695 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.719 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.719 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.737+0000 7fc7657c4640 1 Processor -- start 2024-03-14T12:15:25.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.737+0000 7fc7657c4640 1 -- start start 2024-03-14T12:15:25.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc7657c4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc7600d0760 0x7fc7600d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc7657c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc7600d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc7657c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc7600d1da0 0x7fc7600d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc7657c4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7600d1080 con 0x7fc7600d0760 2024-03-14T12:15:25.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc7657c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7600cf8c0 con 0x7fc7600d1da0 2024-03-14T12:15:25.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc7657c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600cfba0 con 0x7fc7600cf370 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc7600d0760 0x7fc7600d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc7600d0760 0x7fc7600d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41734/0 (socket says 172.21.15.82:41734) 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 -- 172.21.15.82:0/1882317443 learned_addr learned my addr 172.21.15.82:0/1882317443 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc757fff640 1 --2- 172.21.15.82:0/1882317443 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc7600d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75f7fe640 1 --2- 172.21.15.82:0/1882317443 >> v2:172.21.15.164:3300/0 conn(0x7fc7600d1da0 0x7fc7600d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 -- 172.21.15.82:0/1882317443 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 msgr2=0x7fc7600d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 --2- 172.21.15.82:0/1882317443 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc7600d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 -- 172.21.15.82:0/1882317443 >> v2:172.21.15.164:3300/0 conn(0x7fc7600d1da0 msgr2=0x7fc7600d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 --2- 172.21.15.82:0/1882317443 >> v2:172.21.15.164:3300/0 conn(0x7fc7600d1da0 0x7fc7600d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.738+0000 7fc75effd640 1 -- 172.21.15.82:0/1882317443 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600b6c40 con 0x7fc7600d0760 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.739+0000 7fc75effd640 1 --2- 172.21.15.82:0/1882317443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600d0b40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc76008cbb0 tx=0x7fc75002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6853b5d7fe41cbb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.739+0000 7fc75e7fc640 1 -- 172.21.15.82:0/1882317443 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750034070 con 0x7fc7600d0760 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.739+0000 7fc75e7fc640 1 -- 172.21.15.82:0/1882317443 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750030860 con 0x7fc7600d0760 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.739+0000 7fc75e7fc640 1 -- 172.21.15.82:0/1882317443 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750030c30 con 0x7fc7600d0760 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 -- 172.21.15.82:0/1882317443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 msgr2=0x7fc7600d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 --2- 172.21.15.82:0/1882317443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600d0b40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc76008cbb0 tx=0x7fc75002b9a0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 -- 172.21.15.82:0/1882317443 shutdown_connections 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 --2- 172.21.15.82:0/1882317443 >> v2:172.21.15.164:3300/0 conn(0x7fc7600d1da0 0x7fc7600d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 --2- 172.21.15.82:0/1882317443 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc7600d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 --2- 172.21.15.82:0/1882317443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600d0b40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 -- 172.21.15.82:0/1882317443 >> 172.21.15.82:0/1882317443 conn(0x7fc76009d030 msgr2=0x7fc7600d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 -- 172.21.15.82:0/1882317443 shutdown_connections 2024-03-14T12:15:25.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.740+0000 7fc7657c4640 1 -- 172.21.15.82:0/1882317443 wait complete. 2024-03-14T12:15:25.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.741+0000 7fc7657c4640 1 Processor -- start 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.741+0000 7fc7657c4640 1 -- start start 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.741+0000 7fc7657c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc760154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.741+0000 7fc7657c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7657c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7600d1da0 0x7fc7600cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7657c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600d4ba0 con 0x7fc7600d0760 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7657c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600b7200 con 0x7fc7600d1da0 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7657c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7601617a0 con 0x7fc7600cf370 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41736/0 (socket says 172.21.15.82:41736) 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 -- 172.21.15.82:0/2605583758 learned_addr learned my addr 172.21.15.82:0/2605583758 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:25.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc75f7fe640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7600d1da0 0x7fc7600cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc75effd640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc760154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 -- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 msgr2=0x7fc760154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc760154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 -- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7600d1da0 msgr2=0x7fc7600cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7600d1da0 0x7fc7600cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 -- 172.21.15.82:0/2605583758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600a1230 con 0x7fc7600d0760 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc757fff640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600cc450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c5a0 tx=0x7fc74800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7577fe640 1 -- 172.21.15.82:0/2605583758 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748015070 con 0x7fc7600d0760 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7577fe640 1 -- 172.21.15.82:0/2605583758 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748004220 con 0x7fc7600d0760 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7600a1510 con 0x7fc7600d0760 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7577fe640 1 -- 172.21.15.82:0/2605583758 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748003610 con 0x7fc7600d0760 2024-03-14T12:15:25.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.742+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7600a04b0 con 0x7fc7600d0760 2024-03-14T12:15:25.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.743+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76004f050 con 0x7fc7600d0760 2024-03-14T12:15:25.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.744+0000 7fc7577fe640 1 -- 172.21.15.82:0/2605583758 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc7480037b0 con 0x7fc7600d0760 2024-03-14T12:15:25.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.744+0000 7fc7577fe640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc74006bbd0 0x7fc74006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:25.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.744+0000 7fc7577fe640 1 -- 172.21.15.82:0/2605583758 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc74808a6a0 con 0x7fc7600d0760 2024-03-14T12:15:25.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.744+0000 7fc75effd640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc74006bbd0 0x7fc74006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:25.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.745+0000 7fc75effd640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc74006bbd0 0x7fc74006e090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc750036000 tx=0x7fc750008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:25.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.749+0000 7fc7577fe640 1 -- 172.21.15.82:0/2605583758 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc748058bd0 con 0x7fc7600d0760 2024-03-14T12:15:25.751 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.812 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.850 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.853 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.917 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.918 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.933+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc7600b4f60 con 0x7fc7600d0760 2024-03-14T12:15:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.933+0000 7fc7577fe640 1 -- 172.21.15.82:0/2605583758 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc74805cc70 con 0x7fc7600d0760 2024-03-14T12:15:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc74006bbd0 msgr2=0x7fc74006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc74006bbd0 0x7fc74006e090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc750036000 tx=0x7fc750008030 comp rx=0 tx=0).stop 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 msgr2=0x7fc7600cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600cc450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c5a0 tx=0x7fc74800ca60 comp rx=0 tx=0).stop 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 shutdown_connections 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7600d1da0 0x7fc7600cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7600d0760 0x7fc7600cc450 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc74006bbd0 0x7fc74006e090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 --2- 172.21.15.82:0/2605583758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600cf370 0x7fc760154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 >> 172.21.15.82:0/2605583758 conn(0x7fc76009d030 msgr2=0x7fc7600ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 shutdown_connections 2024-03-14T12:15:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:25.935+0000 7fc7657c4640 1 -- 172.21.15.82:0/2605583758 wait complete. 2024-03-14T12:15:25.945 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T12:15:25.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:25.945 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T12:15:25.976 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.982 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:15:25.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:25.982 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T12:15:25.991 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:25.993 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.038 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:26 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:26.058 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.059 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.071 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.089 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.106 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.123 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.124 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.138 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.154 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.171 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.172 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.172 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.219 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.220 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.229 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.267 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.285 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.332 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.333 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.349 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.399 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.408 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.415 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.454 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.519 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.520 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.568 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.603 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.633 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.633 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.669 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.680 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.734 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.745 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.745 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.792 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.811 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.858 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.859 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.875 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.912 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.941 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.971 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:26.976 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.006 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.023 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.070 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.085 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.135 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.151 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.198 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.201 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.234 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:27.248 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:27.250 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:27.258 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.299 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.299 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.314 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.317 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.348 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:27.361 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:27.379 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.380 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:27.382 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.413 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.443 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.445 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.462 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:27.474 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:27.509 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.509 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.509 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:27.525 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.574 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.577 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:27.589 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:27 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:27.640 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.655 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.656 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.721 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.721 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.753 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.786 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.787 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.814 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.853 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.854 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.919 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.920 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.929 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.978 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.993 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:27.995 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.041 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.060 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.061 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.073 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.086 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.091 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.108 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.121 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.125 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.126 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.140 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.156 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.173 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.174 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.174 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.221 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.222 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.244 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.269 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.287 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.334 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.343 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.351 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.356 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.401 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.409 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.417 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.456 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.470 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.473 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.521 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.522 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.540 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.584 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.606 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.635 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.636 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.670 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.682 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.695 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.736 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.747 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.748 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.794 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.807 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.812 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.878 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.914 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:28.927 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:28.943 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:28 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:28.973 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.025 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.036 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.073 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.087 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.092 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.137 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.150 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.200 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.203 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.251 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.260 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.265 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.301 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.301 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.331 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.381 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.381 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.384 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.415 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.428 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.445 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.447 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.447 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.511 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.527 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.540 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.575 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.576 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.591 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.642 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.656 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.657 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.658 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.699 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.723 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.724 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.755 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.768 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.788 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.789 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.816 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.855 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.857 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.869 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.882 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.921 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.922 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.931 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.980 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:29.992 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:29.995 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:29.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:29 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:29.997 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.043 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.063 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.093 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.107 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.127 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.128 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.142 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.158 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.172 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.175 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.175 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.176 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.224 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.224 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.271 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.284 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.289 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.336 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.337 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.350 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.353 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.403 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.411 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.424 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.475 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.524 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.524 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.536 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.608 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.638 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.638 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.651 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.672 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.738 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.749 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.750 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.763 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.814 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.862 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.863 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.876 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:30.879 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:30.945 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.975 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.980 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:30.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:30.993 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.010 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.073 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.089 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.094 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.106 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.154 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.202 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.205 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.219 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.262 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.303 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.303 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.318 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.321 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.386 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.397 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.447 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.449 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.449 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.462 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.513 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.513 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.513 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.526 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.577 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.578 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.593 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.606 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.659 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.701 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.714 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.725 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.790 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.791 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.818 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.831 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.856 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.858 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.923 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.924 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.933 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:31.946 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:31.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:31 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:31.997 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:31.999 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.045 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:32.057 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:32.064 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:32 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:32.065 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.129 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.130 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.144 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:32.156 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:32.177 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.177 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.178 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:32 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:32.225 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.226 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.289 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.338 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.355 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.405 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.420 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.477 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.524 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.544 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.610 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.639 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.674 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.740 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.752 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.816 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.865 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.881 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.947 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:32.978 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.013 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.075 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.091 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.157 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.205 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.264 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.305 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.323 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.384 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.388 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.449 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.451 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.515 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.515 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.579 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.580 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.661 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.662 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.858 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.860 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.925 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:33.926 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.000 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.001 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.066 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.067 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.131 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.132 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.179 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.179 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.181 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:34.191 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:34.228 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.228 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:34 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:34.292 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.340 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.357 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.422 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.479 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.527 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.546 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.612 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.642 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.676 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.742 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.754 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.818 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.867 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.883 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.949 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:34.980 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.015 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.077 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.094 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.158 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.207 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.266 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.308 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.325 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.386 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.390 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.451 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.453 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.517 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.518 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.581 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.582 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.663 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.664 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.729 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.730 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.795 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.795 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.860 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.862 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.002 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.003 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.068 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.069 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.133 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.134 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.181 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.182 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.230 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.230 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.243 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.306 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.342 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.359 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.372 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.409 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.424 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.437 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.481 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.529 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.541 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.548 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.614 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.645 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.656 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.678 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.744 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.756 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.769 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.820 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.869 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.883 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:36.885 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:36.952 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.982 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:36.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:36.995 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.017 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.079 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.096 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.108 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.160 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.209 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.221 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.268 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.310 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.322 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.327 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.389 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.392 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.401 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.453 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.455 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.465 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.519 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.520 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.531 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.582 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.583 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.595 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.677 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.743 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.796 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.797 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.810 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.862 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.875 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:37.929 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.930 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:37.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:37.941 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:37.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:37 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:38.004 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.005 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:38.017 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:38.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:38 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:38.069 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.071 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:38.080 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:38.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:38 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:38.135 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.137 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:38.146 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:38.184 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.184 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:38 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:38.232 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.345 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.412 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.483 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.550 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.616 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.746 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.823 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.887 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.019 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.081 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.163 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.271 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.329 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.394 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.457 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.521 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.584 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.668 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.734 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.798 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.864 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:39.932 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.006 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.072 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.139 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.185 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.186 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:40.198 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:40.347 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.413 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.552 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.618 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.683 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.748 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.825 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.889 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:40.956 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.021 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.083 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.165 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.273 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.331 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.395 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.459 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.523 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.586 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.670 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.736 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.800 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.866 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:41.934 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.008 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.074 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.188 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.237 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.251 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.349 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.363 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.415 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.428 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.487 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.499 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.554 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.567 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.620 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.635 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.685 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.699 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.750 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.762 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.827 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.839 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.891 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.903 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:42.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:42.957 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:42.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:42.969 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.023 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.035 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.097 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.167 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.178 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.275 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.285 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.334 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.344 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.396 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.407 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.461 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.472 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.537 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.588 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.599 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.672 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.682 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.738 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.749 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.802 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.814 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.868 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.879 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:43.936 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:43.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:43.947 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:43.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:44.010 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:44.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:44.021 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:44.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:44.076 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:44.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:44.087 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:44.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:44.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:44.153 INFO:tasks.cephfs.mount:Terminating background process 2024-03-14T12:15:44.190 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:44.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-14T12:15:46.192 INFO:teuthology.orchestra.run.smithi164.stderr:open_background: keeping file open 2024-03-14T12:15:46.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-14T12:15:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-14T12:15:46.206 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:46.206 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:46.263 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:46.263 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:46.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:15:46.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.431+0000 7f0cbb3cd640 1 Processor -- start 2024-03-14T12:15:46.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.431+0000 7f0cbb3cd640 1 -- start start 2024-03-14T12:15:46.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cbb3cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb4071740 0x7f0cb4071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cbb3cd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0cb415e670 0x7f0cb40b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cbb3cd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb4072060 con 0x7f0cb4143f60 2024-03-14T12:15:46.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cbb3cd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb415ebe0 con 0x7f0cb415e670 2024-03-14T12:15:46.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cbb3cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb415eec0 con 0x7f0cb4071740 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cb8968640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0cb415e670 0x7f0cb40b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cb8968640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0cb415e670 0x7f0cb40b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45408/0 (socket says 172.21.15.82:45408) 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.432+0000 7f0cb8968640 1 -- 172.21.15.82:0/4166662564 learned_addr learned my addr 172.21.15.82:0/4166662564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.433+0000 7f0cb9169640 1 --2- 172.21.15.82:0/4166662564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb4071740 0x7f0cb4071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.433+0000 7f0cb8968640 1 -- 172.21.15.82:0/4166662564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb4071740 msgr2=0x7f0cb4071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.433+0000 7f0cb8968640 1 --2- 172.21.15.82:0/4166662564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb4071740 0x7f0cb4071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.433+0000 7f0cb8968640 1 -- 172.21.15.82:0/4166662564 >> v1:172.21.15.164:6789/0 conn(0x7f0cb4143f60 legacy=0x7f0cb415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.433+0000 7f0cb8968640 1 -- 172.21.15.82:0/4166662564 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb415f1a0 con 0x7f0cb415e670 2024-03-14T12:15:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.433+0000 7f0cb8968640 1 --2- 172.21.15.82:0/4166662564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb415e670 0x7f0cb40b2940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009870 tx=0x7f0ca402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24f22410969ede1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:46.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cabfff640 1 -- 172.21.15.82:0/4166662564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4035020 con 0x7f0cb415e670 2024-03-14T12:15:46.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cabfff640 1 -- 172.21.15.82:0/4166662564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca4031640 con 0x7f0cb415e670 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cabfff640 1 -- 172.21.15.82:0/4166662564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca402d040 con 0x7f0cb415e670 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/4166662564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb415e670 msgr2=0x7f0cb40b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/4166662564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb415e670 0x7f0cb40b2940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009870 tx=0x7f0ca402b8f0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/4166662564 shutdown_connections 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/4166662564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb415e670 0x7f0cb40b2940 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/4166662564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb4071740 0x7f0cb4071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/4166662564 >> 172.21.15.82:0/4166662564 conn(0x7f0cb40b1c70 msgr2=0x7f0cb40cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/4166662564 shutdown_connections 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.434+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/4166662564 wait complete. 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.435+0000 7f0cbb3cd640 1 Processor -- start 2024-03-14T12:15:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.435+0000 7f0cbb3cd640 1 -- start start 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cbb3cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb4071740 0x7f0cb4135730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cbb3cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cb4143f60 0x7f0cb4158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cbb3cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb415e670 0x7f0cb4159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cbb3cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb415b080 con 0x7f0cb4071740 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cbb3cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4143ce0 con 0x7f0cb4143f60 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cbb3cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4144670 con 0x7f0cb415e670 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cb9169640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb4071740 0x7f0cb4135730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cb9169640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb4071740 0x7f0cb4135730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45420/0 (socket says 172.21.15.82:45420) 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.436+0000 7f0cb9169640 1 -- 172.21.15.82:0/2323447145 learned_addr learned my addr 172.21.15.82:0/2323447145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb996a640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb415e670 0x7f0cb4159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb8968640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cb4143f60 0x7f0cb4158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb9169640 1 -- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb415e670 msgr2=0x7f0cb4159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb9169640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb415e670 0x7f0cb4159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb9169640 1 -- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cb4143f60 msgr2=0x7f0cb4158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb9169640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cb4143f60 0x7f0cb4158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb9169640 1 -- 172.21.15.82:0/2323447145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb41386e0 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cb9169640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb4071740 0x7f0cb4135730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000ccd0 tx=0x7f0cb000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0ca9ffb640 1 -- 172.21.15.82:0/2323447145 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0003990 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0ca9ffb640 1 -- 172.21.15.82:0/2323447145 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0003b30 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb4137640 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0ca9ffb640 1 -- 172.21.15.82:0/2323447145 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0004020 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.437+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb4137c00 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.438+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb404f050 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.438+0000 7f0ca9ffb640 1 -- 172.21.15.82:0/2323447145 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0cb00041c0 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.439+0000 7f0ca9ffb640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c9806bbd0 0x7f0c9806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.439+0000 7f0ca9ffb640 1 -- 172.21.15.82:0/2323447145 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cb008aec0 con 0x7f0cb4071740 2024-03-14T12:15:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.439+0000 7f0cb8968640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c9806bbd0 0x7f0c9806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.439+0000 7f0cb8968640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c9806bbd0 0x7f0c9806e090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036000 tx=0x7f0ca403c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.443+0000 7f0ca9ffb640 1 -- 172.21.15.82:0/2323447145 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0cb41386e0 con 0x7f0cb4071740 2024-03-14T12:15:46.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.615+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cb41388c0 con 0x7f0cb4071740 2024-03-14T12:15:46.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.616+0000 7f0ca9ffb640 1 -- 172.21.15.82:0/2323447145 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f0cb0014030 con 0x7f0cb4071740 2024-03-14T12:15:46.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:46.617 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:50.791329+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.618+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c9806bbd0 msgr2=0x7f0c9806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.618+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c9806bbd0 0x7f0c9806e090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4036000 tx=0x7f0ca403c040 comp rx=0 tx=0).stop 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.618+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb4071740 msgr2=0x7f0cb4135730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.618+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb4071740 0x7f0cb4135730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000ccd0 tx=0x7f0cb000a5c0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 shutdown_connections 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c9806bbd0 0x7f0c9806e090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0cb415e670 0x7f0cb4159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cb4143f60 0x7f0cb4158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 --2- 172.21.15.82:0/2323447145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0cb4071740 0x7f0cb4135730 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 >> 172.21.15.82:0/2323447145 conn(0x7f0cb40b1c70 msgr2=0x7f0cb413c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:46.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 shutdown_connections 2024-03-14T12:15:46.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.619+0000 7f0cbb3cd640 1 -- 172.21.15.82:0/2323447145 wait complete. 2024-03-14T12:15:46.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:14:47.564085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:14:48.095794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:15:46.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:15:46.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.733+0000 7fcb55080640 1 Processor -- start 2024-03-14T12:15:46.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.733+0000 7fcb55080640 1 -- start start 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb55080640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5015b9a0 0x7fcb5006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb55080640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcb5006e5f0 0x7fcb50071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb55080640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5015e740 con 0x7fcb50072070 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb55080640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5006ea50 con 0x7fcb5006e5f0 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb55080640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5015ea20 con 0x7fcb5015b9a0 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb47fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcb5006e5f0 0x7fcb50071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb4f59e640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcb50072070 0x7fcb501619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:59440/0 (socket says 172.21.15.82:59440) 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb4ed9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5015b9a0 0x7fcb5006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.734+0000 7fcb4f59e640 1 -- 172.21.15.82:0/2516929982 learned_addr learned my addr 172.21.15.82:0/2516929982 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:46.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.735+0000 7fcb4ed9d640 1 -- 172.21.15.82:0/2516929982 >> v2:172.21.15.164:3300/0 conn(0x7fcb5006e5f0 msgr2=0x7fcb50071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:46.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.735+0000 7fcb4ed9d640 1 --2- 172.21.15.82:0/2516929982 >> v2:172.21.15.164:3300/0 conn(0x7fcb5006e5f0 0x7fcb50071b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.735+0000 7fcb4ed9d640 1 -- 172.21.15.82:0/2516929982 >> v1:172.21.15.164:6789/0 conn(0x7fcb50072070 legacy=0x7fcb501619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.735+0000 7fcb4ed9d640 1 -- 172.21.15.82:0/2516929982 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5015ed00 con 0x7fcb5015b9a0 2024-03-14T12:15:46.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.735+0000 7fcb4e59c640 1 -- 172.21.15.82:0/2516929982 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978719506 0 0) 0x7fcb5015e740 con 0x7fcb50072070 2024-03-14T12:15:46.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.735+0000 7fcb47fff640 1 --2- 172.21.15.82:0/2516929982 >> v2:172.21.15.164:3300/0 conn(0x7fcb5006e5f0 0x7fcb50071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.735+0000 7fcb4ed9d640 1 --2- 172.21.15.82:0/2516929982 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5015b9a0 0x7fcb5006e0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb400097a0 tx=0x7fcb4002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=243c9a42fc7588d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb4e59c640 1 -- 172.21.15.82:0/2516929982 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400315a0 con 0x7fcb5015b9a0 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb4e59c640 1 -- 172.21.15.82:0/2516929982 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb40037070 con 0x7fcb5015b9a0 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb4e59c640 1 -- 172.21.15.82:0/2516929982 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40031960 con 0x7fcb5015b9a0 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 -- 172.21.15.82:0/2516929982 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5015b9a0 msgr2=0x7fcb5006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 --2- 172.21.15.82:0/2516929982 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5015b9a0 0x7fcb5006e0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb400097a0 tx=0x7fcb4002b840 comp rx=0 tx=0).stop 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 -- 172.21.15.82:0/2516929982 shutdown_connections 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 --2- 172.21.15.82:0/2516929982 >> v2:172.21.15.164:3300/0 conn(0x7fcb5006e5f0 0x7fcb50071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 --2- 172.21.15.82:0/2516929982 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5015b9a0 0x7fcb5006e0b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 -- 172.21.15.82:0/2516929982 >> 172.21.15.82:0/2516929982 conn(0x7fcb500b1d40 msgr2=0x7fcb500b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:46.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 -- 172.21.15.82:0/2516929982 shutdown_connections 2024-03-14T12:15:46.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.736+0000 7fcb55080640 1 -- 172.21.15.82:0/2516929982 wait complete. 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.737+0000 7fcb55080640 1 Processor -- start 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.737+0000 7fcb55080640 1 -- start start 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb55080640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5006e5f0 0x7fcb5014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb55080640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb50072070 0x7fcb50154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb55080640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb5015b9a0 0x7fcb500b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb55080640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500724e0 con 0x7fcb5015b9a0 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb55080640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5015e3f0 con 0x7fcb50072070 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb55080640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50072660 con 0x7fcb5006e5f0 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb4f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb5015b9a0 0x7fcb500b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb4f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb5015b9a0 0x7fcb500b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45436/0 (socket says 172.21.15.82:45436) 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb4f59e640 1 -- 172.21.15.82:0/3703621641 learned_addr learned my addr 172.21.15.82:0/3703621641 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:46.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb4ed9d640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5006e5f0 0x7fcb5014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.738+0000 7fcb47fff640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb50072070 0x7fcb50154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb4f59e640 1 -- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5006e5f0 msgr2=0x7fcb5014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb4f59e640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5006e5f0 0x7fcb5014f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb4f59e640 1 -- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb50072070 msgr2=0x7fcb50154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb4f59e640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb50072070 0x7fcb50154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb4f59e640 1 -- 172.21.15.82:0/3703621641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50154dc0 con 0x7fcb5015b9a0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb4ed9d640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5006e5f0 0x7fcb5014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb4f59e640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb5015b9a0 0x7fcb500b0d30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800caf0 tx=0x7fcb480021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb477fe640 1 -- 172.21.15.82:0/3703621641 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800ac00 con 0x7fcb5015b9a0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb477fe640 1 -- 172.21.15.82:0/3703621641 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4800ada0 con 0x7fcb5015b9a0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb50151cc0 con 0x7fcb5015b9a0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb477fe640 1 -- 172.21.15.82:0/3703621641 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48012640 con 0x7fcb5015b9a0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.739+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb50152280 con 0x7fcb5015b9a0 2024-03-14T12:15:46.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.740+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5004f050 con 0x7fcb5015b9a0 2024-03-14T12:15:46.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.740+0000 7fcb477fe640 1 -- 172.21.15.82:0/3703621641 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcb480127e0 con 0x7fcb5015b9a0 2024-03-14T12:15:46.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.741+0000 7fcb477fe640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcb3006bbd0 0x7fcb3006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:46.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.741+0000 7fcb4ed9d640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcb3006bbd0 0x7fcb3006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:46.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.741+0000 7fcb477fe640 1 -- 172.21.15.82:0/3703621641 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb4808b010 con 0x7fcb5015b9a0 2024-03-14T12:15:46.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.741+0000 7fcb4ed9d640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcb3006bbd0 0x7fcb3006e090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb40036000 tx=0x7fcb40008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:46.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.745+0000 7fcb477fe640 1 -- 172.21.15.82:0/3703621641 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fcb48059b10 con 0x7fcb5015b9a0 2024-03-14T12:15:46.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.918+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb50152450 con 0x7fcb5015b9a0 2024-03-14T12:15:46.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.918+0000 7fcb477fe640 1 -- 172.21.15.82:0/3703621641 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb48059410 con 0x7fcb5015b9a0 2024-03-14T12:15:46.919 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcb3006bbd0 msgr2=0x7fcb3006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcb3006bbd0 0x7fcb3006e090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb40036000 tx=0x7fcb40008030 comp rx=0 tx=0).stop 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb5015b9a0 msgr2=0x7fcb500b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb5015b9a0 0x7fcb500b0d30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800caf0 tx=0x7fcb480021c0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 shutdown_connections 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb5015b9a0 0x7fcb500b0d30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb50072070 0x7fcb50154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcb3006bbd0 0x7fcb3006e090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 --2- 172.21.15.82:0/3703621641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb5006e5f0 0x7fcb5014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 >> 172.21.15.82:0/3703621641 conn(0x7fcb500b1d40 msgr2=0x7fcb500b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:46.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 shutdown_connections 2024-03-14T12:15:46.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:46.920+0000 7fcb55080640 1 -- 172.21.15.82:0/3703621641 wait complete. 2024-03-14T12:15:46.932 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:15:47.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.035+0000 7f798fedc640 1 Processor -- start 2024-03-14T12:15:47.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.035+0000 7f798fedc640 1 -- start start 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798fedc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f798815e0c0 0x7f79880cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798fedc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79880b0bc0 0x7f79880ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798fedc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f79880d01b0 0x7f79880d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798fedc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880cf1e0 con 0x7f79880b0bc0 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798fedc640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880cf440 con 0x7f79880d01b0 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798fedc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880ccb60 con 0x7f798815e0c0 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798d477640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79880b0bc0 0x7f79880ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798d477640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79880b0bc0 0x7f79880ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45440/0 (socket says 172.21.15.82:45440) 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.036+0000 7f798d477640 1 -- 172.21.15.82:0/1748444798 learned_addr learned my addr 172.21.15.82:0/1748444798 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798dc78640 1 --2- 172.21.15.82:0/1748444798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f798815e0c0 0x7f79880cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798e479640 1 --2- 172.21.15.82:0/1748444798 >> v2:172.21.15.164:3300/0 conn(0x7f79880d01b0 0x7f79880d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798d477640 1 -- 172.21.15.82:0/1748444798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f798815e0c0 msgr2=0x7f79880cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798d477640 1 --2- 172.21.15.82:0/1748444798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f798815e0c0 0x7f79880cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798d477640 1 -- 172.21.15.82:0/1748444798 >> v2:172.21.15.164:3300/0 conn(0x7f79880d01b0 msgr2=0x7f79880d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798d477640 1 --2- 172.21.15.82:0/1748444798 >> v2:172.21.15.164:3300/0 conn(0x7f79880d01b0 0x7f79880d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798d477640 1 -- 172.21.15.82:0/1748444798 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880cce40 con 0x7f79880b0bc0 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.037+0000 7f798d477640 1 --2- 172.21.15.82:0/1748444798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79880b0bc0 0x7f79880ceca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f798806b770 tx=0x7f797802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9daf18de2e8dc97 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798cc76640 1 -- 172.21.15.82:0/1748444798 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978035020 con 0x7f79880b0bc0 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798cc76640 1 -- 172.21.15.82:0/1748444798 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978031870 con 0x7f79880b0bc0 2024-03-14T12:15:47.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798cc76640 1 -- 172.21.15.82:0/1748444798 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978031c40 con 0x7f79880b0bc0 2024-03-14T12:15:47.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 -- 172.21.15.82:0/1748444798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79880b0bc0 msgr2=0x7f79880ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 --2- 172.21.15.82:0/1748444798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79880b0bc0 0x7f79880ceca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f798806b770 tx=0x7f797802b910 comp rx=0 tx=0).stop 2024-03-14T12:15:47.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 -- 172.21.15.82:0/1748444798 shutdown_connections 2024-03-14T12:15:47.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 --2- 172.21.15.82:0/1748444798 >> v2:172.21.15.164:3300/0 conn(0x7f79880d01b0 0x7f79880d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 --2- 172.21.15.82:0/1748444798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79880b0bc0 0x7f79880ceca0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 --2- 172.21.15.82:0/1748444798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f798815e0c0 0x7f79880cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 -- 172.21.15.82:0/1748444798 >> 172.21.15.82:0/1748444798 conn(0x7f79880d1bd0 msgr2=0x7f79880d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 -- 172.21.15.82:0/1748444798 shutdown_connections 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.038+0000 7f798fedc640 1 -- 172.21.15.82:0/1748444798 wait complete. 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 Processor -- start 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 -- start start 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79880b0bc0 0x7f79880c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79880d01b0 0x7f79880b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798815e0c0 0x7f7988153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798815e4a0 con 0x7f798815e0c0 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880cd400 con 0x7f79880d01b0 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798fedc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798806bbc0 con 0x7f79880b0bc0 2024-03-14T12:15:47.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798e479640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798815e0c0 0x7f7988153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798e479640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798815e0c0 0x7f7988153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45450/0 (socket says 172.21.15.82:45450) 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.040+0000 7f798e479640 1 -- 172.21.15.82:0/3053311894 learned_addr learned my addr 172.21.15.82:0/3053311894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798d477640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79880d01b0 0x7f79880b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798dc78640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79880b0bc0 0x7f79880c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798e479640 1 -- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79880b0bc0 msgr2=0x7f79880c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798e479640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79880b0bc0 0x7f79880c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798e479640 1 -- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79880d01b0 msgr2=0x7f79880b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798e479640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79880d01b0 0x7f79880b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798e479640 1 -- 172.21.15.82:0/3053311894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880c5700 con 0x7f798815e0c0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798e479640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798815e0c0 0x7f7988153c60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f798000d800 tx=0x7f798000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f797e7fc640 1 -- 172.21.15.82:0/3053311894 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980018020 con 0x7f798815e0c0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f797e7fc640 1 -- 172.21.15.82:0/3053311894 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79800096c0 con 0x7f798815e0c0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79880c59e0 con 0x7f798815e0c0 2024-03-14T12:15:47.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f797e7fc640 1 -- 172.21.15.82:0/3053311894 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980009a90 con 0x7f798815e0c0 2024-03-14T12:15:47.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.041+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79880ca3b0 con 0x7f798815e0c0 2024-03-14T12:15:47.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.042+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798804f050 con 0x7f798815e0c0 2024-03-14T12:15:47.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.042+0000 7f797e7fc640 1 -- 172.21.15.82:0/3053311894 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f798001b020 con 0x7f798815e0c0 2024-03-14T12:15:47.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.043+0000 7f797e7fc640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f796c06bc00 0x7f796c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.043+0000 7f798dc78640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f796c06bc00 0x7f796c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.043+0000 7f797e7fc640 1 -- 172.21.15.82:0/3053311894 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f798008ac00 con 0x7f798815e0c0 2024-03-14T12:15:47.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.043+0000 7f798dc78640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f796c06bc00 0x7f796c06e0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79840099c0 tx=0x7f7984003990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.047+0000 7f797e7fc640 1 -- 172.21.15.82:0/3053311894 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7980091080 con 0x7f798815e0c0 2024-03-14T12:15:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.217+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79880b7b50 con 0x7f798815e0c0 2024-03-14T12:15:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.217+0000 7f797e7fc640 1 -- 172.21.15.82:0/3053311894 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7980022030 con 0x7f798815e0c0 2024-03-14T12:15:47.217 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f796c06bc00 msgr2=0x7f796c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f796c06bc00 0x7f796c06e0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79840099c0 tx=0x7f7984003990 comp rx=0 tx=0).stop 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798815e0c0 msgr2=0x7f7988153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798815e0c0 0x7f7988153c60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f798000d800 tx=0x7f798000dcc0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 shutdown_connections 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798815e0c0 0x7f7988153c60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79880d01b0 0x7f79880b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f796c06bc00 0x7f796c06e0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 --2- 172.21.15.82:0/3053311894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79880b0bc0 0x7f79880c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 >> 172.21.15.82:0/3053311894 conn(0x7f79880d1bd0 msgr2=0x7f79880cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 shutdown_connections 2024-03-14T12:15:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.219+0000 7f798fedc640 1 -- 172.21.15.82:0/3053311894 wait complete. 2024-03-14T12:15:47.230 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:15:47.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.332+0000 7feae659e640 1 Processor -- start 2024-03-14T12:15:47.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.332+0000 7feae659e640 1 -- start start 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae659e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae015dff0 0x7feae00d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae659e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feae00d0e10 0x7feae00cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae659e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae00d25a0 con 0x7feae00d12a0 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae659e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae00d2350 con 0x7feae00d0e10 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae659e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae00d2880 con 0x7feae015dff0 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feadffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae015dff0 0x7feae00d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae4b3b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feae00d0e10 0x7feae00cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae4b3b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feae00d0e10 0x7feae00cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:42520/0 (socket says 172.21.15.82:42520) 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feadffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae015dff0 0x7feae00d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:47440/0 (socket says 172.21.15.82:47440) 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feae4b3b640 1 -- 172.21.15.82:0/1121361740 learned_addr learned my addr 172.21.15.82:0/1121361740 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.333+0000 7feadf7fe640 1 --1- 172.21.15.82:0/1121361740 >> v1:172.21.15.164:6789/0 conn(0x7feae00d12a0 0x7feae00d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:59454/0 (socket says 172.21.15.82:59454) 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.334+0000 7feae4b3b640 1 -- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae015dff0 msgr2=0x7feae00d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.334+0000 7feae4b3b640 1 --2- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae015dff0 0x7feae00d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.334+0000 7feae4b3b640 1 -- 172.21.15.82:0/1121361740 >> v1:172.21.15.164:6789/0 conn(0x7feae00d12a0 legacy=0x7feae00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.334+0000 7feae4b3b640 1 -- 172.21.15.82:0/1121361740 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae00d2b60 con 0x7feae00d0e10 2024-03-14T12:15:47.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.334+0000 7feadffff640 1 --2- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae015dff0 0x7feae00d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:15:47.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae4b3b640 1 --2- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d0e10 0x7feae00cb320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e50 tx=0x7fead802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91c11dbe740d759e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feadeffd640 1 -- 172.21.15.82:0/1121361740 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead802cb00 con 0x7feae00d0e10 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feadeffd640 1 -- 172.21.15.82:0/1121361740 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead802cca0 con 0x7feae00d0e10 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feadeffd640 1 -- 172.21.15.82:0/1121361740 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8034610 con 0x7feae00d0e10 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 -- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d0e10 msgr2=0x7feae00cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 --2- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d0e10 0x7feae00cb320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e50 tx=0x7fead802bf60 comp rx=0 tx=0).stop 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 -- 172.21.15.82:0/1121361740 shutdown_connections 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 --2- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d0e10 0x7feae00cb320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 --2- 172.21.15.82:0/1121361740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae015dff0 0x7feae00d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 -- 172.21.15.82:0/1121361740 >> 172.21.15.82:0/1121361740 conn(0x7feae009d0d0 msgr2=0x7feae00d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 -- 172.21.15.82:0/1121361740 shutdown_connections 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.335+0000 7feae659e640 1 -- 172.21.15.82:0/1121361740 wait complete. 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.336+0000 7feae659e640 1 Processor -- start 2024-03-14T12:15:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.337+0000 7feae659e640 1 -- start start 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.337+0000 7feae659e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d0e10 0x7feae01572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.337+0000 7feae659e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d12a0 0x7feae01f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.337+0000 7feae659e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae015dff0 0x7feae014e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.337+0000 7feae659e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae015e550 con 0x7feae015dff0 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.337+0000 7feae659e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae015e3d0 con 0x7feae00d12a0 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.337+0000 7feae659e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae00cb860 con 0x7feae00d0e10 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae015dff0 0x7feae014e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae015dff0 0x7feae014e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45462/0 (socket says 172.21.15.82:45462) 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 -- 172.21.15.82:0/2497308203 learned_addr learned my addr 172.21.15.82:0/2497308203 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feadffff640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d0e10 0x7feae01572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feadf7fe640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d12a0 0x7feae01f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 -- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d0e10 msgr2=0x7feae01572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d0e10 0x7feae01572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 -- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d12a0 msgr2=0x7feae01f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d12a0 0x7feae01f3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 -- 172.21.15.82:0/2497308203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae014e760 con 0x7feae015dff0 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feadffff640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d0e10 0x7feae01572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feadf7fe640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d12a0 0x7feae01f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae4b3b640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae015dff0 0x7feae014e220 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fead8036040 tx=0x7fead8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feadcff9640 1 -- 172.21.15.82:0/2497308203 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8043020 con 0x7feae015dff0 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feadcff9640 1 -- 172.21.15.82:0/2497308203 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead802ce50 con 0x7feae015dff0 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae00b7a40 con 0x7feae015dff0 2024-03-14T12:15:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feadcff9640 1 -- 172.21.15.82:0/2497308203 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8009510 con 0x7feae015dff0 2024-03-14T12:15:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.338+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae00b8000 con 0x7feae015dff0 2024-03-14T12:15:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.339+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae004f050 con 0x7feae015dff0 2024-03-14T12:15:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.340+0000 7feadcff9640 1 -- 172.21.15.82:0/2497308203 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fead8046020 con 0x7feae015dff0 2024-03-14T12:15:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.340+0000 7feadcff9640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feacc06bbd0 0x7feacc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.340+0000 7feadcff9640 1 -- 172.21.15.82:0/2497308203 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fead80ad040 con 0x7feae015dff0 2024-03-14T12:15:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.340+0000 7feadffff640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feacc06bbd0 0x7feacc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.340+0000 7feadffff640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feacc06bbd0 0x7feacc06e090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feae01f70e0 tx=0x7fead0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.344+0000 7feadcff9640 1 -- 172.21.15.82:0/2497308203 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fead804d030 con 0x7feae015dff0 2024-03-14T12:15:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.516+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feae0160c90 con 0x7feae015dff0 2024-03-14T12:15:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.516+0000 7feadcff9640 1 -- 172.21.15.82:0/2497308203 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fead803c350 con 0x7feae015dff0 2024-03-14T12:15:47.517 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feacc06bbd0 msgr2=0x7feacc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feacc06bbd0 0x7feacc06e090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feae01f70e0 tx=0x7fead0008040 comp rx=0 tx=0).stop 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae015dff0 msgr2=0x7feae014e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae015dff0 0x7feae014e220 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fead8036040 tx=0x7fead8004a40 comp rx=0 tx=0).stop 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 shutdown_connections 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feae015dff0 0x7feae014e220 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feae00d12a0 0x7feae01f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feacc06bbd0 0x7feacc06e090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 --2- 172.21.15.82:0/2497308203 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feae00d0e10 0x7feae01572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 >> 172.21.15.82:0/2497308203 conn(0x7feae009d0d0 msgr2=0x7feae00cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.518+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 shutdown_connections 2024-03-14T12:15:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.519+0000 7feae659e640 1 -- 172.21.15.82:0/2497308203 wait complete. 2024-03-14T12:15:47.529 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:15:47.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.630+0000 7f98e351b640 1 Processor -- start 2024-03-14T12:15:47.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.631+0000 7f98e351b640 1 -- start start 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.631+0000 7f98e351b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98dc15b9e0 0x7f98dc0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.631+0000 7f98e351b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc15c250 con 0x7f98dc0cd0d0 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.631+0000 7f98e351b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0cd810 con 0x7f98dc15bdc0 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.631+0000 7f98e351b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0cf730 con 0x7f98dc15b9e0 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98e1ab8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f98dc0cd0d0 0x7f98dc15f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38130/0 (socket says 172.21.15.82:38130) 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98e1ab8640 1 -- 172.21.15.82:0/1752902731 learned_addr learned my addr 172.21.15.82:0/1752902731 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98e12b7640 1 --2- 172.21.15.82:0/1752902731 >> v2:172.21.15.164:3300/0 conn(0x7f98dc15b9e0 0x7f98dc0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734033797 0 0) 0x7f98dc15c250 con 0x7f98dc0cd0d0 2024-03-14T12:15:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c8003520 con 0x7f98dc0cd0d0 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3484753661 0 0) 0x7f98c8003520 con 0x7f98dc0cd0d0 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 >> v2:172.21.15.164:3300/0 conn(0x7f98dc15b9e0 msgr2=0x7f98dc0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 --2- 172.21.15.82:0/1752902731 >> v2:172.21.15.164:3300/0 conn(0x7f98dc15b9e0 0x7f98dc0b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 >> v1:172.21.15.164:6789/0 conn(0x7f98dc15bdc0 legacy=0x7f98dc0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0cf980 con 0x7f98dc0cd0d0 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f98d40019f0 con 0x7f98dc0cd0d0 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 >> v1:172.21.15.82:6789/0 conn(0x7f98dc0cd0d0 legacy=0x7f98dc15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98c80039b0 0x7f98c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98c8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98c8005120 0x7f98c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0cf730 con 0x7f98c80043b0 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c800c060 con 0x7f98c80039b0 2024-03-14T12:15:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c800c260 con 0x7f98c8005120 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98e0ab6640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98c8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98e12b7640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98c80039b0 0x7f98c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.632+0000 7f98e1ab8640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98c8005120 0x7f98c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98e0ab6640 1 -- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98c8005120 msgr2=0x7f98c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98e0ab6640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98c8005120 0x7f98c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98e0ab6640 1 -- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98c80039b0 msgr2=0x7f98c8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98e0ab6640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98c80039b0 0x7f98c8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98e0ab6640 1 -- 172.21.15.82:0/1752902731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c8003520 con 0x7f98c80043b0 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98e1ab8640 1 -- 172.21.15.82:0/1752902731 reap_dead start 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98e0ab6640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98c8004880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f98d8001910 tx=0x7f98c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0d44e3da17ec91 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80014e0 con 0x7f98c80043b0 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d8037070 con 0x7f98c80043b0 2024-03-14T12:15:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.633+0000 7f98cffff640 1 -- 172.21.15.82:0/1752902731 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8032880 con 0x7f98c80043b0 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.634+0000 7f98e351b640 1 -- 172.21.15.82:0/1752902731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 msgr2=0x7f98c8004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.634+0000 7f98e351b640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98c8004880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f98d8001910 tx=0x7f98c8005830 comp rx=0 tx=0).stop 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.634+0000 7f98e351b640 1 -- 172.21.15.82:0/1752902731 shutdown_connections 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.634+0000 7f98e351b640 1 --2- 172.21.15.82:0/1752902731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98c8004880 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.634+0000 7f98e351b640 1 -- 172.21.15.82:0/1752902731 >> 172.21.15.82:0/1752902731 conn(0x7f98dc0b3670 msgr2=0x7f98dc0b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.634+0000 7f98e351b640 1 -- 172.21.15.82:0/1752902731 shutdown_connections 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.634+0000 7f98e351b640 1 -- 172.21.15.82:0/1752902731 wait complete. 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.635+0000 7f98e351b640 1 Processor -- start 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.635+0000 7f98e351b640 1 -- start start 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e351b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98dc0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e351b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98dc0ca410 0x7f98dc155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e351b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98dc156460 0x7f98dc06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e351b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0cd520 con 0x7f98c80043b0 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e351b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0cff40 con 0x7f98dc156460 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e351b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0b4190 con 0x7f98dc0ca410 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98dc0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98dc0b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45484/0 (socket says 172.21.15.82:45484) 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 -- 172.21.15.82:0/2840257611 learned_addr learned my addr 172.21.15.82:0/2840257611 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e1ab8640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98dc156460 0x7f98dc06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e0ab6640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98dc0ca410 0x7f98dc155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 -- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98dc0ca410 msgr2=0x7f98dc155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98dc0ca410 0x7f98dc155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 -- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98dc156460 msgr2=0x7f98dc06f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98dc156460 0x7f98dc06f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 -- 172.21.15.82:0/2840257611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc15ecf0 con 0x7f98c80043b0 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98e12b7640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98dc0b03b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f98d0011a70 tx=0x7f98d0011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.636+0000 7f98cdffb640 1 -- 172.21.15.82:0/2840257611 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000fb50 con 0x7f98c80043b0 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.637+0000 7f98cdffb640 1 -- 172.21.15.82:0/2840257611 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d000fcf0 con 0x7f98c80043b0 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.637+0000 7f98cdffb640 1 -- 172.21.15.82:0/2840257611 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000fea0 con 0x7f98c80043b0 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.637+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc0b08f0 con 0x7f98c80043b0 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.637+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc0cabb0 con 0x7f98c80043b0 2024-03-14T12:15:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.638+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc04f050 con 0x7f98c80043b0 2024-03-14T12:15:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.638+0000 7f98cdffb640 1 -- 172.21.15.82:0/2840257611 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f98d0012040 con 0x7f98c80043b0 2024-03-14T12:15:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.639+0000 7f98cdffb640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f98c006bbd0 0x7f98c006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.639+0000 7f98cdffb640 1 -- 172.21.15.82:0/2840257611 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98d008b170 con 0x7f98c80043b0 2024-03-14T12:15:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.639+0000 7f98e0ab6640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f98c006bbd0 0x7f98c006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.639+0000 7f98e0ab6640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f98c006bbd0 0x7f98c006e090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f98d8036000 tx=0x7f98d8009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.643+0000 7f98cdffb640 1 -- 172.21.15.82:0/2840257611 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f98d0019030 con 0x7f98c80043b0 2024-03-14T12:15:47.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.826+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f98dc0b7ae0 con 0x7f98c80043b0 2024-03-14T12:15:47.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.826+0000 7f98cdffb640 1 -- 172.21.15.82:0/2840257611 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f98d0059cf0 con 0x7f98c80043b0 2024-03-14T12:15:47.827 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f98c006bbd0 msgr2=0x7f98c006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f98c006bbd0 0x7f98c006e090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f98d8036000 tx=0x7f98d8009030 comp rx=0 tx=0).stop 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 msgr2=0x7f98dc0b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98dc0b03b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f98d0011a70 tx=0x7f98d0011f30 comp rx=0 tx=0).stop 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 shutdown_connections 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98dc156460 0x7f98dc06f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f98c006bbd0 0x7f98c006e090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98dc0ca410 0x7f98dc155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 --2- 172.21.15.82:0/2840257611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98c80043b0 0x7f98dc0b03b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 >> 172.21.15.82:0/2840257611 conn(0x7f98dc0b3670 msgr2=0x7f98dc15f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 shutdown_connections 2024-03-14T12:15:47.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:47.828+0000 7f98e351b640 1 -- 172.21.15.82:0/2840257611 wait complete. 2024-03-14T12:15:47.839 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T12:15:47.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:47.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T12:15:47.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:15:47.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:15:47.877 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T12:15:47.935 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:15:47.935 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:15:47.980 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:15:48.033 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:15:48.033 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:48.033 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:48.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:15:48.048 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:48.048 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:48.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:15:48.064 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:48.065 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:48.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:15:48.106 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:48.107 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:48.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:15:48.123 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:15:48.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.246+0000 7fdeb637e640 1 Processor -- start 2024-03-14T12:15:48.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.246+0000 7fdeb637e640 1 -- start start 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.246+0000 7fdeb637e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdeb0143f60 0x7fdeb015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.246+0000 7fdeb637e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb00b3050 con 0x7fdeb0071740 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.246+0000 7fdeb637e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb015ebe0 con 0x7fdeb015e670 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.246+0000 7fdeb637e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb015eec0 con 0x7fdeb0143f60 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdeb0071740 0x7fdeb0071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38138/0 (socket says 172.21.15.82:38138) 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaffff640 1 -- 172.21.15.82:0/2377996636 learned_addr learned my addr 172.21.15.82:0/2377996636 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeb491b640 1 --2- 172.21.15.82:0/2377996636 >> v2:172.21.15.164:3300/0 conn(0x7fdeb0143f60 0x7fdeb015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968329775 0 0) 0x7fdeb00b3050 con 0x7fdeb0071740 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde98003520 con 0x7fdeb0071740 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1997545432 0 0) 0x7fde98003520 con 0x7fdeb0071740 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 >> v2:172.21.15.164:3300/0 conn(0x7fdeb0143f60 msgr2=0x7fdeb015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 --2- 172.21.15.82:0/2377996636 >> v2:172.21.15.164:3300/0 conn(0x7fdeb0143f60 0x7fdeb015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 >> v1:172.21.15.164:6789/0 conn(0x7fdeb015e670 legacy=0x7fdeb00b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb015f1a0 con 0x7fdeb0071740 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fdea0001670 con 0x7fdeb0071740 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 >> v1:172.21.15.82:6789/0 conn(0x7fdeb0071740 legacy=0x7fdeb0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde980039b0 0x7fde98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde98004800 0x7fde98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde980051c0 0x7fde9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb015eec0 con 0x7fde980051c0 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9800c060 con 0x7fde980039b0 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9800c260 con 0x7fde98004800 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeb491b640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde980051c0 0x7fde9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaffff640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde980039b0 0x7fde98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeaf7fe640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde98004800 0x7fde98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeb491b640 1 -- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde98004800 msgr2=0x7fde98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeb491b640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde98004800 0x7fde98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.247+0000 7fdeb491b640 1 -- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde980039b0 msgr2=0x7fde98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.248+0000 7fdeb491b640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde980039b0 0x7fde98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.248+0000 7fdeb491b640 1 -- 172.21.15.82:0/2377996636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98003520 con 0x7fde980051c0 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.248+0000 7fdeb491b640 1 -- 172.21.15.82:0/2377996636 reap_dead start 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.248+0000 7fdeb491b640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde980051c0 0x7fde9800b9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009fc0 tx=0x7fde98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc5ed00838189ad server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.248+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8009c10 con 0x7fde980051c0 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.248+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea802cca0 con 0x7fde980051c0 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.248+0000 7fdeaeffd640 1 -- 172.21.15.82:0/2377996636 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8003bd0 con 0x7fde980051c0 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.249+0000 7fdeb637e640 1 -- 172.21.15.82:0/2377996636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde980051c0 msgr2=0x7fde9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.249+0000 7fdeb637e640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde980051c0 0x7fde9800b9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009fc0 tx=0x7fde98005830 comp rx=0 tx=0).stop 2024-03-14T12:15:48.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.249+0000 7fdeb637e640 1 -- 172.21.15.82:0/2377996636 shutdown_connections 2024-03-14T12:15:48.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.249+0000 7fdeb637e640 1 --2- 172.21.15.82:0/2377996636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde980051c0 0x7fde9800b9c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.249+0000 7fdeb637e640 1 -- 172.21.15.82:0/2377996636 >> 172.21.15.82:0/2377996636 conn(0x7fdeb00b1c70 msgr2=0x7fdeb00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:48.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.249+0000 7fdeb637e640 1 -- 172.21.15.82:0/2377996636 shutdown_connections 2024-03-14T12:15:48.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.249+0000 7fdeb637e640 1 -- 172.21.15.82:0/2377996636 wait complete. 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.250+0000 7fdeb637e640 1 Processor -- start 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.250+0000 7fdeb637e640 1 -- start start 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeb637e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde980051c0 0x7fdeb0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeb637e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeb013d190 0x7fdeb00ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeb637e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb00b0220 0x7fdeb00b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeb637e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb015b080 con 0x7fdeb013d190 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeb637e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb0143ce0 con 0x7fdeb00b0220 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeb637e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb0144670 con 0x7fde980051c0 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeb013d190 0x7fdeb00ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeb013d190 0x7fdeb00ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45500/0 (socket says 172.21.15.82:45500) 2024-03-14T12:15:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 -- 172.21.15.82:0/742263110 learned_addr learned my addr 172.21.15.82:0/742263110 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaffff640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde980051c0 0x7fdeb0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeb491b640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb00b0220 0x7fdeb00b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 -- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde980051c0 msgr2=0x7fdeb0158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde980051c0 0x7fdeb0158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 -- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb00b0220 msgr2=0x7fdeb00b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb00b0220 0x7fdeb00b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 -- 172.21.15.82:0/742263110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb015b680 con 0x7fdeb013d190 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.251+0000 7fdeaf7fe640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeb013d190 0x7fdeb00ad850 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e4b0 tx=0x7fdea400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.252+0000 7fdeacff9640 1 -- 172.21.15.82:0/742263110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40052e0 con 0x7fdeb013d190 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.252+0000 7fdeacff9640 1 -- 172.21.15.82:0/742263110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4018070 con 0x7fdeb013d190 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.252+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb00ae0a0 con 0x7fdeb013d190 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.252+0000 7fdeacff9640 1 -- 172.21.15.82:0/742263110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4003830 con 0x7fdeb013d190 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.252+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb00ae660 con 0x7fdeb013d190 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.252+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb004f050 con 0x7fdeb013d190 2024-03-14T12:15:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.253+0000 7fdeacff9640 1 -- 172.21.15.82:0/742263110 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdea40039d0 con 0x7fdeb013d190 2024-03-14T12:15:48.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.253+0000 7fdeacff9640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde9c06bbd0 0x7fde9c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.253+0000 7fdeaffff640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde9c06bbd0 0x7fde9c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.254+0000 7fdeacff9640 1 -- 172.21.15.82:0/742263110 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdea408a5f0 con 0x7fdeb013d190 2024-03-14T12:15:48.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.254+0000 7fdeaffff640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde9c06bbd0 0x7fde9c06e090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdea0001790 tx=0x7fdea0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.258+0000 7fdeacff9640 1 -- 172.21.15.82:0/742263110 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdea4059170 con 0x7fdeb013d190 2024-03-14T12:15:48.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.465+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdeb00d07b0 con 0x7fdeb013d190 2024-03-14T12:15:48.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.466+0000 7fdeacff9640 1 -- 172.21.15.82:0/742263110 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fdea4058a70 con 0x7fdeb013d190 2024-03-14T12:15:48.466 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:48.467 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2478084830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2478084830},{"type":"v1","addr":"172.21.15.82:6835","nonce":2478084830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4834,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1811632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1811632800},{"type":"v1","addr":"172.21.15.164:6834","nonce":1811632800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4176962755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4176962755},{"type":"v1","addr":"172.21.15.164:6835","nonce":4176962755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:14:49.135548+0000","modified":"2024-03-14T12:14:51.133632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/682556190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":682556190},{"type":"v1","addr":"172.21.15.82:6837","nonce":682556190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T12:15:48.467 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 15 2024-03-14T12:15:48.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde9c06bbd0 msgr2=0x7fde9c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde9c06bbd0 0x7fde9c06e090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdea0001790 tx=0x7fdea0009040 comp rx=0 tx=0).stop 2024-03-14T12:15:48.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeb013d190 msgr2=0x7fdeb00ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeb013d190 0x7fdeb00ad850 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e4b0 tx=0x7fdea400e970 comp rx=0 tx=0).stop 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 shutdown_connections 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb00b0220 0x7fdeb00b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeb013d190 0x7fdeb00ad850 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde9c06bbd0 0x7fde9c06e090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 --2- 172.21.15.82:0/742263110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde980051c0 0x7fdeb0158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 >> 172.21.15.82:0/742263110 conn(0x7fdeb00b1c70 msgr2=0x7fdeb00b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 shutdown_connections 2024-03-14T12:15:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.468+0000 7fdeb637e640 1 -- 172.21.15.82:0/742263110 wait complete. 2024-03-14T12:15:48.479 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:15:48.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.579+0000 7fefdf9c4640 1 Processor -- start 2024-03-14T12:15:48.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.580+0000 7fefdf9c4640 1 -- start start 2024-03-14T12:15:48.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.580+0000 7fefdf9c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 0x7fefd80d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.580+0000 7fefdf9c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd80cf8c0 con 0x7fefd80d0760 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.580+0000 7fefdf9c4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd80cfba0 con 0x7fefd80b7720 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.580+0000 7fefdf9c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80b6c40 con 0x7fefd80cf370 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.580+0000 7fefdd760640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fefd80d0760 0x7fefd80d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38150/0 (socket says 172.21.15.82:38150) 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefdd760640 1 -- 172.21.15.82:0/2248765344 learned_addr learned my addr 172.21.15.82:0/2248765344 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755408261 0 0) 0x7fefd80cf8c0 con 0x7fefd80d0760 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefdcf5f640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 0x7fefd80d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefc4003520 con 0x7fefd80d0760 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640957128 0 0) 0x7fefc4003520 con 0x7fefd80d0760 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 msgr2=0x7fefd80d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 0x7fefd80d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 >> v1:172.21.15.164:6789/0 conn(0x7fefd80b7720 legacy=0x7fefd80b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80b6f20 con 0x7fefd80d0760 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fefd0001670 con 0x7fefd80d0760 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 >> v1:172.21.15.82:6789/0 conn(0x7fefd80d0760 legacy=0x7fefd80d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc40039e0 0x7fefc4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 0x7fefc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc4004d40 0x7fefc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80b6c40 con 0x7fefc4004d40 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc400bb30 con 0x7fefc40039e0 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc400bd30 con 0x7fefd80cf370 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefddf61640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc4004d40 0x7fefc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefdd760640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc40039e0 0x7fefc4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefdcf5f640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 0x7fefc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefddf61640 1 -- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 msgr2=0x7fefc4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefddf61640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cf370 0x7fefc4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefddf61640 1 -- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc40039e0 msgr2=0x7fefc4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefddf61640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc40039e0 0x7fefc4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefddf61640 1 -- 172.21.15.82:0/2248765344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4003520 con 0x7fefc4004d40 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.581+0000 7fefdd760640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefc40039e0 0x7fefc4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.582+0000 7fefddf61640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc4004d40 0x7fefc40051c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefd4001e20 tx=0x7fefc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab923d776dd0673 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.582+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4001a70 con 0x7fefc4004d40 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.582+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4003b70 con 0x7fefc4004d40 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.582+0000 7fefcffff640 1 -- 172.21.15.82:0/2248765344 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd40050d0 con 0x7fefc4004d40 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.583+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2248765344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc4004d40 msgr2=0x7fefc40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.583+0000 7fefdf9c4640 1 --2- 172.21.15.82:0/2248765344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc4004d40 0x7fefc40051c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefd4001e20 tx=0x7fefc4005830 comp rx=0 tx=0).stop 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.583+0000 7fefddf61640 1 -- 172.21.15.82:0/2248765344 reap_dead start 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.583+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2248765344 shutdown_connections 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.583+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2248765344 >> 172.21.15.82:0/2248765344 conn(0x7fefd809d030 msgr2=0x7fefd80d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:48.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.583+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2248765344 shutdown_connections 2024-03-14T12:15:48.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.583+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2248765344 wait complete. 2024-03-14T12:15:48.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.584+0000 7fefdf9c4640 1 Processor -- start 2024-03-14T12:15:48.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.584+0000 7fefdf9c4640 1 -- start start 2024-03-14T12:15:48.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdf9c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd815a130 0x7fefd8159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdf9c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cc680 0x7fefd80cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdf9c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefd815a510 0x7fefd8154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdf9c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8161920 con 0x7fefd815a510 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdf9c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd81617a0 con 0x7fefd815a130 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdf9c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80d4660 con 0x7fefd80cc680 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefd815a510 0x7fefd8154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefd815a510 0x7fefd8154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45518/0 (socket says 172.21.15.82:45518) 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 -- 172.21.15.82:0/2158022372 learned_addr learned my addr 172.21.15.82:0/2158022372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdcf5f640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cc680 0x7fefd80cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdd760640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd815a130 0x7fefd8159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 -- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cc680 msgr2=0x7fefd80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cc680 0x7fefd80cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 -- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd815a130 msgr2=0x7fefd8159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd815a130 0x7fefd8159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 -- 172.21.15.82:0/2158022372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80cb360 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefdd760640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd815a130 0x7fefd8159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.585+0000 7fefddf61640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefd815a510 0x7fefd8154080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fefd4001f40 tx=0x7fefd40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.586+0000 7fefcdffb640 1 -- 172.21.15.82:0/2158022372 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4007db0 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.586+0000 7fefcdffb640 1 -- 172.21.15.82:0/2158022372 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4003970 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.586+0000 7fefcdffb640 1 -- 172.21.15.82:0/2158022372 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4006d90 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.586+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd80cb5e0 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.586+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd80cbba0 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.587+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd804f050 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.587+0000 7fefcdffb640 1 -- 172.21.15.82:0/2158022372 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fefd4041030 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.588+0000 7fefcdffb640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fefbc06bc00 0x7fefbc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.588+0000 7fefcdffb640 1 -- 172.21.15.82:0/2158022372 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefd40ab8e0 con 0x7fefd815a510 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.588+0000 7fefdd760640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fefbc06bc00 0x7fefbc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.588+0000 7fefdd760640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fefbc06bc00 0x7fefbc06e0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fefd0001eb0 tx=0x7fefd0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.592+0000 7fefcdffb640 1 -- 172.21.15.82:0/2158022372 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fefd80cb360 con 0x7fefd815a510 2024-03-14T12:15:48.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.764+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefd80cde50 con 0x7fefd815a510 2024-03-14T12:15:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.765+0000 7fefcdffb640 1 -- 172.21.15.82:0/2158022372 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16657 (secure 0 0 0) 0x7fefd4033020 con 0x7fefd815a510 2024-03-14T12:15:48.765 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:50.791329+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:15:48.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fefbc06bc00 msgr2=0x7fefbc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fefbc06bc00 0x7fefbc06e0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fefd0001eb0 tx=0x7fefd0009040 comp rx=0 tx=0).stop 2024-03-14T12:15:48.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefd815a510 msgr2=0x7fefd8154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefd815a510 0x7fefd8154080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fefd4001f40 tx=0x7fefd40014a0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 shutdown_connections 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefd815a510 0x7fefd8154080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fefbc06bc00 0x7fefbc06e0c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fefd80cc680 0x7fefd80cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 --2- 172.21.15.82:0/2158022372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd815a130 0x7fefd8159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 >> 172.21.15.82:0/2158022372 conn(0x7fefd809d030 msgr2=0x7fefd80ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 shutdown_connections 2024-03-14T12:15:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.767+0000 7fefdf9c4640 1 -- 172.21.15.82:0/2158022372 wait complete. 2024-03-14T12:15:48.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:14:47.564085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:14:48.095794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:15:48.778 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:15:48.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.882+0000 7f6774a81640 1 Processor -- start 2024-03-14T12:15:48.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.882+0000 7f6774a81640 1 -- start start 2024-03-14T12:15:48.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6774a81640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f67700d0db0 0x7f6770162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6774a81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 0x7f677015e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6774a81640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677006e8e0 con 0x7f6770147360 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6774a81640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677006eb80 con 0x7f67700d0db0 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6774a81640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677006f100 con 0x7f67700b2510 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6767fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6770147360 0x7f67700b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38156/0 (socket says 172.21.15.82:38156) 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6767fff640 1 -- 172.21.15.82:0/2013157191 learned_addr learned my addr 172.21.15.82:0/2013157191 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387008041 0 0) 0x7f677006e8e0 con 0x7f6770147360 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676ed9d640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 0x7f677015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676e59c640 1 --2- 172.21.15.82:0/2013157191 >> v2:172.21.15.164:3300/0 conn(0x7f67700d0db0 0x7f6770162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f675c003520 con 0x7f6770147360 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660093596 0 0) 0x7f675c003520 con 0x7f6770147360 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 msgr2=0x7f677015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 0x7f677015e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 >> v2:172.21.15.164:3300/0 conn(0x7f67700d0db0 msgr2=0x7f6770162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 --2- 172.21.15.82:0/2013157191 >> v2:172.21.15.164:3300/0 conn(0x7f67700d0db0 0x7f6770162560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677006ee20 con 0x7f6770147360 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6758001670 con 0x7f6770147360 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 >> v1:172.21.15.82:6789/0 conn(0x7f6770147360 legacy=0x7f67700b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c0039b0 0x7f675c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f675c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 0x7f675c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677006eb80 con 0x7f675c004800 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677006f100 con 0x7f675c0039b0 2024-03-14T12:15:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c00b9f0 con 0x7f67700b2510 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.883+0000 7f6767fff640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f675c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f676e59c640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c0039b0 0x7f675c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f676ed9d640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 0x7f675c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f6767fff640 1 -- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 msgr2=0x7f675c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f6767fff640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 0x7f675c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f6767fff640 1 -- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c0039b0 msgr2=0x7f675c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f6767fff640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c0039b0 0x7f675c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f6767fff640 1 -- 172.21.15.82:0/2013157191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c003520 con 0x7f675c004800 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f676ed9d640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67700b2510 0x7f675c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.884+0000 7f6767fff640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f675c004c80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f675800c290 tx=0x7f675c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e352b9950c1f47 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758007d50 con 0x7f675c004800 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6758030070 con 0x7f675c004800 2024-03-14T12:15:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f676dd9b640 1 -- 172.21.15.82:0/2013157191 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675802cd50 con 0x7f675c004800 2024-03-14T12:15:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f6774a81640 1 -- 172.21.15.82:0/2013157191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 msgr2=0x7f675c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f6774a81640 1 --2- 172.21.15.82:0/2013157191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f675c004c80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f675800c290 tx=0x7f675c005830 comp rx=0 tx=0).stop 2024-03-14T12:15:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f676ed9d640 1 -- 172.21.15.82:0/2013157191 reap_dead start 2024-03-14T12:15:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f6774a81640 1 -- 172.21.15.82:0/2013157191 shutdown_connections 2024-03-14T12:15:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f6774a81640 1 -- 172.21.15.82:0/2013157191 >> 172.21.15.82:0/2013157191 conn(0x7f67700b4fd0 msgr2=0x7f67700b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f6774a81640 1 -- 172.21.15.82:0/2013157191 shutdown_connections 2024-03-14T12:15:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.885+0000 7f6774a81640 1 -- 172.21.15.82:0/2013157191 wait complete. 2024-03-14T12:15:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.886+0000 7f6774a81640 1 Processor -- start 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.886+0000 7f6774a81640 1 -- start start 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f6774a81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f677009c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f6774a81640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f677009d860 0x7f6770146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f6774a81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f677009af80 0x7f677015c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f6774a81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770162aa0 con 0x7f675c004800 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f6774a81640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770162040 con 0x7f677009d860 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f6774a81640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677006d330 con 0x7f677009af80 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f676e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f677009c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f676e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f677009c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45534/0 (socket says 172.21.15.82:45534) 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f676e59c640 1 -- 172.21.15.82:0/2960842712 learned_addr learned my addr 172.21.15.82:0/2960842712 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f6767fff640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f677009d860 0x7f6770146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f676e59c640 1 -- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f677009af80 msgr2=0x7f677015c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f676ed9d640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f677009af80 0x7f677015c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.887+0000 7f676e59c640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f677009af80 0x7f677015c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f676e59c640 1 -- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f677009d860 msgr2=0x7f6770146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f676e59c640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f677009d860 0x7f6770146260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f676e59c640 1 -- 172.21.15.82:0/2960842712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677009b8f0 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f6767fff640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f677009d860 0x7f6770146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f676ed9d640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f677009af80 0x7f677015c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f676e59c640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f677009c990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6760011960 tx=0x7f6760011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f6766ffd640 1 -- 172.21.15.82:0/2960842712 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000faf0 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f6766ffd640 1 -- 172.21.15.82:0/2960842712 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676000fc90 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f6766ffd640 1 -- 172.21.15.82:0/2960842712 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760014440 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677009bbd0 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.888+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677009c190 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.889+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677004f050 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.889+0000 7f6766ffd640 1 -- 172.21.15.82:0/2960842712 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6760012040 con 0x7f675c004800 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.890+0000 7f6766ffd640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f675006bc00 0x7f675006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.890+0000 7f6767fff640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f675006bc00 0x7f675006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:48.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.890+0000 7f6766ffd640 1 -- 172.21.15.82:0/2960842712 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f676008b0e0 con 0x7f675c004800 2024-03-14T12:15:48.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.890+0000 7f6767fff640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f675006bc00 0x7f675006e0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6758034000 tx=0x7f6758007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:48.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:48.894+0000 7f6766ffd640 1 -- 172.21.15.82:0/2960842712 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6760019030 con 0x7f675c004800 2024-03-14T12:15:49.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.066+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f677015de30 con 0x7f675c004800 2024-03-14T12:15:49.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.067+0000 7f6766ffd640 1 -- 172.21.15.82:0/2960842712 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6760059bd0 con 0x7f675c004800 2024-03-14T12:15:49.068 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:15:49.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f675006bc00 msgr2=0x7f675006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f675006bc00 0x7f675006e0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6758034000 tx=0x7f6758007000 comp rx=0 tx=0).stop 2024-03-14T12:15:49.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 msgr2=0x7f677009c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f677009c990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6760011960 tx=0x7f6760011e20 comp rx=0 tx=0).stop 2024-03-14T12:15:49.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 shutdown_connections 2024-03-14T12:15:49.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f675006bc00 0x7f675006e0c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f677009af80 0x7f677015c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f677009d860 0x7f6770146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 --2- 172.21.15.82:0/2960842712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f675c004800 0x7f677009c990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 >> 172.21.15.82:0/2960842712 conn(0x7f67700b4fd0 msgr2=0x7f67701607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:49.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 shutdown_connections 2024-03-14T12:15:49.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.069+0000 7f6774a81640 1 -- 172.21.15.82:0/2960842712 wait complete. 2024-03-14T12:15:49.080 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:15:49.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.182+0000 7f9d39612640 1 Processor -- start 2024-03-14T12:15:49.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.182+0000 7f9d39612640 1 -- start start 2024-03-14T12:15:49.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d39612640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d3415dff0 0x7f9d340d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d39612640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9d340d12a0 0x7f9d340d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d39612640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d340d25a0 con 0x7f9d340d0c40 2024-03-14T12:15:49.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d39612640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d340d2350 con 0x7f9d3415dff0 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d39612640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d340d2880 con 0x7f9d340d12a0 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d32ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d3415dff0 0x7f9d340d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d32ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d3415dff0 0x7f9d340d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45538/0 (socket says 172.21.15.82:45538) 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d32ffd640 1 -- 172.21.15.82:0/1352507034 learned_addr learned my addr 172.21.15.82:0/1352507034 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d327fc640 1 --2- 172.21.15.82:0/1352507034 >> v2:172.21.15.164:3300/0 conn(0x7f9d340d12a0 0x7f9d340d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d32ffd640 1 -- 172.21.15.82:0/1352507034 >> v2:172.21.15.164:3300/0 conn(0x7f9d340d12a0 msgr2=0x7f9d340d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d32ffd640 1 --2- 172.21.15.82:0/1352507034 >> v2:172.21.15.164:3300/0 conn(0x7f9d340d12a0 0x7f9d340d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d32ffd640 1 -- 172.21.15.82:0/1352507034 >> v1:172.21.15.164:6789/0 conn(0x7f9d340d0c40 legacy=0x7f9d340ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.183+0000 7f9d32ffd640 1 -- 172.21.15.82:0/1352507034 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d340d2b60 con 0x7f9d3415dff0 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.184+0000 7f9d32ffd640 1 --2- 172.21.15.82:0/1352507034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d340d4480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9d340d3390 tx=0x7f9d2c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718bee636d7296a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.184+0000 7f9d31ffb640 1 -- 172.21.15.82:0/1352507034 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c02ca80 con 0x7f9d3415dff0 2024-03-14T12:15:49.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.184+0000 7f9d31ffb640 1 -- 172.21.15.82:0/1352507034 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c02cc20 con 0x7f9d3415dff0 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.184+0000 7f9d31ffb640 1 -- 172.21.15.82:0/1352507034 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0343f0 con 0x7f9d3415dff0 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 -- 172.21.15.82:0/1352507034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 msgr2=0x7f9d340d4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 --2- 172.21.15.82:0/1352507034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d340d4480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9d340d3390 tx=0x7f9d2c02bee0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 -- 172.21.15.82:0/1352507034 shutdown_connections 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 --2- 172.21.15.82:0/1352507034 >> v2:172.21.15.164:3300/0 conn(0x7f9d340d12a0 0x7f9d340d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 --2- 172.21.15.82:0/1352507034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d340d4480 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 -- 172.21.15.82:0/1352507034 >> 172.21.15.82:0/1352507034 conn(0x7f9d3409d0d0 msgr2=0x7f9d340d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 -- 172.21.15.82:0/1352507034 shutdown_connections 2024-03-14T12:15:49.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.185+0000 7f9d39612640 1 -- 172.21.15.82:0/1352507034 wait complete. 2024-03-14T12:15:49.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.186+0000 7f9d39612640 1 Processor -- start 2024-03-14T12:15:49.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.186+0000 7f9d39612640 1 -- start start 2024-03-14T12:15:49.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d39612640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d340d0c40 0x7f9d3415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d39612640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d340d12a0 0x7f9d340a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d39612640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d3415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d39612640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3415e550 con 0x7f9d3415dff0 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d39612640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3415e3d0 con 0x7f9d340d0c40 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d39612640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340cc330 con 0x7f9d340d12a0 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d3415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d3415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45540/0 (socket says 172.21.15.82:45540) 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 -- 172.21.15.82:0/3999771901 learned_addr learned my addr 172.21.15.82:0/3999771901 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d32ffd640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d340d0c40 0x7f9d3415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d327fc640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d340d12a0 0x7f9d340a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 -- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d340d12a0 msgr2=0x7f9d340a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d340d12a0 0x7f9d340a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 -- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d340d0c40 msgr2=0x7f9d3415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d340d0c40 0x7f9d3415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 -- 172.21.15.82:0/3999771901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3415c8d0 con 0x7f9d3415dff0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d337fe640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d3415edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c440 tx=0x7f9d2400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d2b7fe640 1 -- 172.21.15.82:0/3999771901 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24011790 con 0x7f9d3415dff0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d2b7fe640 1 -- 172.21.15.82:0/3999771901 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24017070 con 0x7f9d3415dff0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d340c8590 con 0x7f9d3415dff0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.187+0000 7f9d2b7fe640 1 -- 172.21.15.82:0/3999771901 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24011b70 con 0x7f9d3415dff0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.188+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d340a0be0 con 0x7f9d3415dff0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.188+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3404f050 con 0x7f9d3415dff0 2024-03-14T12:15:49.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.189+0000 7f9d2b7fe640 1 -- 172.21.15.82:0/3999771901 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9d24011d10 con 0x7f9d3415dff0 2024-03-14T12:15:49.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.189+0000 7f9d2b7fe640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d1406bbd0 0x7f9d1406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.189+0000 7f9d2b7fe640 1 -- 172.21.15.82:0/3999771901 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d2408a840 con 0x7f9d3415dff0 2024-03-14T12:15:49.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.189+0000 7f9d32ffd640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d1406bbd0 0x7f9d1406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.190+0000 7f9d32ffd640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d1406bbd0 0x7f9d1406e090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c035000 tx=0x7f9d2c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.194+0000 7f9d2b7fe640 1 -- 172.21.15.82:0/3999771901 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9d3415c8d0 con 0x7f9d3415dff0 2024-03-14T12:15:49.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.362+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d340cf910 con 0x7f9d3415dff0 2024-03-14T12:15:49.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.362+0000 7f9d2b7fe640 1 -- 172.21.15.82:0/3999771901 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d240593c0 con 0x7f9d3415dff0 2024-03-14T12:15:49.363 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d1406bbd0 msgr2=0x7f9d1406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d1406bbd0 0x7f9d1406e090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c035000 tx=0x7f9d2c030000 comp rx=0 tx=0).stop 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 msgr2=0x7f9d3415edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d3415edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c440 tx=0x7f9d2400c900 comp rx=0 tx=0).stop 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 shutdown_connections 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3415dff0 0x7f9d3415edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d1406bbd0 0x7f9d1406e090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d340d12a0 0x7f9d340a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 --2- 172.21.15.82:0/3999771901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d340d0c40 0x7f9d3415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 >> 172.21.15.82:0/3999771901 conn(0x7f9d3409d0d0 msgr2=0x7f9d340ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 shutdown_connections 2024-03-14T12:15:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.364+0000 7f9d39612640 1 -- 172.21.15.82:0/3999771901 wait complete. 2024-03-14T12:15:49.375 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:15:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.478+0000 7ff1b6fd4640 1 Processor -- start 2024-03-14T12:15:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.478+0000 7ff1b6fd4640 1 -- start start 2024-03-14T12:15:49.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1b6fd4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1b0147360 0x7ff1b00b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1b6fd4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b006e8e0 con 0x7ff1b00d0db0 2024-03-14T12:15:49.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1b6fd4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b006eb80 con 0x7ff1b01478b0 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1b6fd4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b006f100 con 0x7ff1b0147360 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1affff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1b0147360 0x7ff1b00b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1b4d70640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff1b00d0db0 0x7ff1b0162560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38160/0 (socket says 172.21.15.82:38160) 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1b4d70640 1 -- 172.21.15.82:0/2264168950 learned_addr learned my addr 172.21.15.82:0/2264168950 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1affff640 1 -- 172.21.15.82:0/2264168950 >> v1:172.21.15.164:6789/0 conn(0x7ff1b01478b0 legacy=0x7ff1b015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1affff640 1 -- 172.21.15.82:0/2264168950 >> v1:172.21.15.82:6789/0 conn(0x7ff1b00d0db0 legacy=0x7ff1b0162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1affff640 1 -- 172.21.15.82:0/2264168950 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b006ee20 con 0x7ff1b0147360 2024-03-14T12:15:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.479+0000 7ff1af7fe640 1 -- 172.21.15.82:0/2264168950 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159471662 0 0) 0x7ff1b006e8e0 con 0x7ff1b00d0db0 2024-03-14T12:15:49.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.480+0000 7ff1affff640 1 --2- 172.21.15.82:0/2264168950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b0147360 0x7ff1b00b1e00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006ed50 tx=0x7ff1a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16acc61f7753bc84 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.480+0000 7ff1af7fe640 1 -- 172.21.15.82:0/2264168950 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40315a0 con 0x7ff1b0147360 2024-03-14T12:15:49.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.480+0000 7ff1af7fe640 1 -- 172.21.15.82:0/2264168950 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4037070 con 0x7ff1b0147360 2024-03-14T12:15:49.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.480+0000 7ff1af7fe640 1 -- 172.21.15.82:0/2264168950 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4031960 con 0x7ff1b0147360 2024-03-14T12:15:49.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.481+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/2264168950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b0147360 msgr2=0x7ff1b00b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.481+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/2264168950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b0147360 0x7ff1b00b1e00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006ed50 tx=0x7ff1a402b840 comp rx=0 tx=0).stop 2024-03-14T12:15:49.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.481+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/2264168950 shutdown_connections 2024-03-14T12:15:49.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.481+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/2264168950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b0147360 0x7ff1b00b1e00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.481+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/2264168950 >> 172.21.15.82:0/2264168950 conn(0x7ff1b00b4fd0 msgr2=0x7ff1b00b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:49.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.481+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/2264168950 shutdown_connections 2024-03-14T12:15:49.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.481+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/2264168950 wait complete. 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.482+0000 7ff1b6fd4640 1 Processor -- start 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.482+0000 7ff1b6fd4640 1 -- start start 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b00d0db0 0x7ff1b0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b0147360 0x7ff1b015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1b01478b0 0x7ff1b01590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b006d330 con 0x7ff1b00d0db0 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0162040 con 0x7ff1b0147360 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b006d4b0 con 0x7ff1b01478b0 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b00d0db0 0x7ff1b0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b00d0db0 0x7ff1b0099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45560/0 (socket says 172.21.15.82:45560) 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 -- 172.21.15.82:0/838088935 learned_addr learned my addr 172.21.15.82:0/838088935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 -- 172.21.15.82:0/838088935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1b01478b0 msgr2=0x7ff1b01590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1b01478b0 0x7ff1b01590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 -- 172.21.15.82:0/838088935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b0147360 msgr2=0x7ff1b015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b0147360 0x7ff1b015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 -- 172.21.15.82:0/838088935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b01f3cc0 con 0x7ff1b00d0db0 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b4d70640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b00d0db0 0x7ff1b0099a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000c4d0 tx=0x7ff1a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1ad7fa640 1 -- 172.21.15.82:0/838088935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0018070 con 0x7ff1b00d0db0 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1ad7fa640 1 -- 172.21.15.82:0/838088935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a00074e0 con 0x7ff1b00d0db0 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1ad7fa640 1 -- 172.21.15.82:0/838088935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0004a20 con 0x7ff1b00d0db0 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b0138960 con 0x7ff1b00d0db0 2024-03-14T12:15:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.483+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b0138f20 con 0x7ff1b00d0db0 2024-03-14T12:15:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.484+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b004f050 con 0x7ff1b00d0db0 2024-03-14T12:15:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.485+0000 7ff1ad7fa640 1 -- 172.21.15.82:0/838088935 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff1a001b070 con 0x7ff1b00d0db0 2024-03-14T12:15:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.485+0000 7ff1ad7fa640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff19c06bbd0 0x7ff19c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.485+0000 7ff1affff640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff19c06bbd0 0x7ff19c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.485+0000 7ff1ad7fa640 1 -- 172.21.15.82:0/838088935 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1a0089d90 con 0x7ff1b00d0db0 2024-03-14T12:15:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.486+0000 7ff1affff640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff19c06bbd0 0x7ff19c06e090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4036000 tx=0x7ff1a4030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.491+0000 7ff1ad7fa640 1 -- 172.21.15.82:0/838088935 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff1a0058880 con 0x7ff1b00d0db0 2024-03-14T12:15:49.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.659+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1b006dbe0 con 0x7ff1b00d0db0 2024-03-14T12:15:49.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.660+0000 7ff1ad7fa640 1 -- 172.21.15.82:0/838088935 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff1a0058180 con 0x7ff1b00d0db0 2024-03-14T12:15:49.660 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:15:49.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff19c06bbd0 msgr2=0x7ff19c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff19c06bbd0 0x7ff19c06e090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4036000 tx=0x7ff1a4030000 comp rx=0 tx=0).stop 2024-03-14T12:15:49.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b00d0db0 msgr2=0x7ff1b0099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b00d0db0 0x7ff1b0099a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000c4d0 tx=0x7ff1a000c990 comp rx=0 tx=0).stop 2024-03-14T12:15:49.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 shutdown_connections 2024-03-14T12:15:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff19c06bbd0 0x7ff19c06e090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1b01478b0 0x7ff1b01590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1b0147360 0x7ff1b015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 --2- 172.21.15.82:0/838088935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b00d0db0 0x7ff1b0099a50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 >> 172.21.15.82:0/838088935 conn(0x7ff1b00b4fd0 msgr2=0x7ff1b015e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 shutdown_connections 2024-03-14T12:15:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.662+0000 7ff1b6fd4640 1 -- 172.21.15.82:0/838088935 wait complete. 2024-03-14T12:15:49.672 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:15:49.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.772+0000 7f33ea709640 1 Processor -- start 2024-03-14T12:15:49.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.772+0000 7f33ea709640 1 -- start start 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33ea709640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33ea709640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33e40afff0 0x7f33e40d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33ea709640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40b1340 con 0x7f33e40b0f60 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33ea709640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e40d0ee0 con 0x7f33e40afff0 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33ea709640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40b4ee0 con 0x7f33e406b420 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33e3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f33e40b0f60 0x7f33e406ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38174/0 (socket says 172.21.15.82:38174) 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33e3fff640 1 -- 172.21.15.82:0/3016355682 learned_addr learned my addr 172.21.15.82:0/3016355682 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33e8ca6640 1 --2- 172.21.15.82:0/3016355682 >> v2:172.21.15.82:3300/0 conn(0x7f33e40afff0 0x7f33e40d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33e37fe640 1 --2- 172.21.15.82:0/3016355682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33e8ca6640 1 -- 172.21.15.82:0/3016355682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 msgr2=0x7f33e40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33e8ca6640 1 --2- 172.21.15.82:0/3016355682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e40cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.773+0000 7f33e8ca6640 1 -- 172.21.15.82:0/3016355682 >> v1:172.21.15.82:6789/0 conn(0x7f33e40b0f60 legacy=0x7f33e406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.774+0000 7f33e8ca6640 1 -- 172.21.15.82:0/3016355682 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40b0480 con 0x7f33e40afff0 2024-03-14T12:15:49.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.774+0000 7f33e8ca6640 1 --2- 172.21.15.82:0/3016355682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e40d09a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009d90 tx=0x7f33dc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ff4d0df43dcdaf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.774+0000 7f33e2ffd640 1 -- 172.21.15.82:0/3016355682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc039070 con 0x7f33e40afff0 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.774+0000 7f33e2ffd640 1 -- 172.21.15.82:0/3016355682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc02cc50 con 0x7f33e40afff0 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.774+0000 7f33e2ffd640 1 -- 172.21.15.82:0/3016355682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc034440 con 0x7f33e40afff0 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 -- 172.21.15.82:0/3016355682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 msgr2=0x7f33e40d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 --2- 172.21.15.82:0/3016355682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e40d09a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009d90 tx=0x7f33dc02bea0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 -- 172.21.15.82:0/3016355682 shutdown_connections 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 --2- 172.21.15.82:0/3016355682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e40d09a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 --2- 172.21.15.82:0/3016355682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 -- 172.21.15.82:0/3016355682 >> 172.21.15.82:0/3016355682 conn(0x7f33e4072310 msgr2=0x7f33e4072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 -- 172.21.15.82:0/3016355682 shutdown_connections 2024-03-14T12:15:49.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.775+0000 7f33ea709640 1 -- 172.21.15.82:0/3016355682 wait complete. 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.776+0000 7f33ea709640 1 Processor -- start 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.776+0000 7f33ea709640 1 -- start start 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33ea709640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e4158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33ea709640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e415c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33ea709640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33e40b0f60 0x7f33e40c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33ea709640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40cd5f0 con 0x7f33e40afff0 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33ea709640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40cd310 con 0x7f33e40b0f60 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33ea709640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40cd8d0 con 0x7f33e406b420 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e415c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e415c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45586/0 (socket says 172.21.15.82:45586) 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 -- 172.21.15.82:0/1997176768 learned_addr learned my addr 172.21.15.82:0/1997176768 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e3fff640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e8ca6640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33e40b0f60 0x7f33e40c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 -- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 msgr2=0x7f33e4158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e4158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 -- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33e40b0f60 msgr2=0x7f33e40c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33e40b0f60 0x7f33e40c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.777+0000 7f33e37fe640 1 -- 172.21.15.82:0/1997176768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40c5550 con 0x7f33e40afff0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.778+0000 7f33e37fe640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e415c3d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f33d800d800 tx=0x7f33d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.778+0000 7f33e0ff9640 1 -- 172.21.15.82:0/1997176768 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d800b6c0 con 0x7f33e40afff0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.778+0000 7f33e0ff9640 1 -- 172.21.15.82:0/1997176768 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d8019070 con 0x7f33e40afff0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.778+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e40c5830 con 0x7f33e40afff0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.778+0000 7f33e0ff9640 1 -- 172.21.15.82:0/1997176768 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d800ba80 con 0x7f33e40afff0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.778+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e40c5df0 con 0x7f33e40afff0 2024-03-14T12:15:49.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.779+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e404f050 con 0x7f33e40afff0 2024-03-14T12:15:49.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.779+0000 7f33e0ff9640 1 -- 172.21.15.82:0/1997176768 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f33d800bc20 con 0x7f33e40afff0 2024-03-14T12:15:49.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.780+0000 7f33e0ff9640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33d006bbd0 0x7f33d006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:49.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.780+0000 7f33e0ff9640 1 -- 172.21.15.82:0/1997176768 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33d808ade0 con 0x7f33e40afff0 2024-03-14T12:15:49.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.780+0000 7f33e3fff640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33d006bbd0 0x7f33d006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:49.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.780+0000 7f33e3fff640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33d006bbd0 0x7f33d006e090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f33e406e1d0 tx=0x7f33d4007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:49.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.785+0000 7f33e0ff9640 1 -- 172.21.15.82:0/1997176768 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f33d8059280 con 0x7f33e40afff0 2024-03-14T12:15:49.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.993+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33e4071200 con 0x7f33e40afff0 2024-03-14T12:15:49.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.993+0000 7f33e0ff9640 1 -- 172.21.15.82:0/1997176768 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f33d805d320 con 0x7f33e40afff0 2024-03-14T12:15:49.994 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:49.994 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2478084830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2478084830},{"type":"v1","addr":"172.21.15.82:6835","nonce":2478084830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4834,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1811632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1811632800},{"type":"v1","addr":"172.21.15.164:6834","nonce":1811632800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4176962755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4176962755},{"type":"v1","addr":"172.21.15.164:6835","nonce":4176962755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:14:49.135548+0000","modified":"2024-03-14T12:14:51.133632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/682556190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":682556190},{"type":"v1","addr":"172.21.15.82:6837","nonce":682556190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T12:15:49.994 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 15 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33d006bbd0 msgr2=0x7f33d006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33d006bbd0 0x7f33d006e090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f33e406e1d0 tx=0x7f33d4007e80 comp rx=0 tx=0).stop 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 msgr2=0x7f33e415c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e415c3d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f33d800d800 tx=0x7f33d800dcc0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 shutdown_connections 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33e40b0f60 0x7f33e40c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33e40afff0 0x7f33e415c3d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f33d006bbd0 0x7f33d006e090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 --2- 172.21.15.82:0/1997176768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f33e406b420 0x7f33e4158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 >> 172.21.15.82:0/1997176768 conn(0x7f33e4072310 msgr2=0x7f33e40acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 shutdown_connections 2024-03-14T12:15:49.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:49.995+0000 7f33ea709640 1 -- 172.21.15.82:0/1997176768 wait complete. 2024-03-14T12:15:50.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:15:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.107+0000 7febe5d6f640 1 Processor -- start 2024-03-14T12:15:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.107+0000 7febe5d6f640 1 -- start start 2024-03-14T12:15:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febe5d6f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe015e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febe5d6f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febe006b420 0x7febe015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febe5d6f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00d1590 con 0x7febe015df30 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febe5d6f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe0161b70 con 0x7febe006b420 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febe5d6f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe0161e50 con 0x7febe006f7f0 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febe006b420 0x7febe015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdeffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febe006b420 0x7febe015e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:42678/0 (socket says 172.21.15.82:42678) 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdffff640 1 -- 172.21.15.82:0/1673465610 learned_addr learned my addr 172.21.15.82:0/1673465610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdf7fe640 1 --1- 172.21.15.82:0/1673465610 >> v1:172.21.15.164:6789/0 conn(0x7febe015df30 0x7febe006ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:59514/0 (socket says 172.21.15.82:59514) 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdeffd640 1 -- 172.21.15.82:0/1673465610 >> v2:172.21.15.164:3300/0 conn(0x7febe006b420 msgr2=0x7febe015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdeffd640 1 --2- 172.21.15.82:0/1673465610 >> v2:172.21.15.164:3300/0 conn(0x7febe006b420 0x7febe015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdeffd640 1 -- 172.21.15.82:0/1673465610 >> v1:172.21.15.164:6789/0 conn(0x7febe015df30 legacy=0x7febe006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdeffd640 1 -- 172.21.15.82:0/1673465610 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0162130 con 0x7febe006f7f0 2024-03-14T12:15:50.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.108+0000 7febdffff640 1 --2- 172.21.15.82:0/1673465610 >> v2:172.21.15.164:3300/0 conn(0x7febe006b420 0x7febe015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:15:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.109+0000 7febdeffd640 1 --2- 172.21.15.82:0/1673465610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe015e310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7febd40097a0 tx=0x7febd402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b354f54a501cb22 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.109+0000 7febde7fc640 1 -- 172.21.15.82:0/1673465610 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4035020 con 0x7febe006f7f0 2024-03-14T12:15:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.109+0000 7febde7fc640 1 -- 172.21.15.82:0/1673465610 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd4031830 con 0x7febe006f7f0 2024-03-14T12:15:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.109+0000 7febde7fc640 1 -- 172.21.15.82:0/1673465610 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4031c00 con 0x7febe006f7f0 2024-03-14T12:15:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 -- 172.21.15.82:0/1673465610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 msgr2=0x7febe015e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 --2- 172.21.15.82:0/1673465610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe015e310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7febd40097a0 tx=0x7febd402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 -- 172.21.15.82:0/1673465610 shutdown_connections 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 --2- 172.21.15.82:0/1673465610 >> v2:172.21.15.164:3300/0 conn(0x7febe006b420 0x7febe015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 --2- 172.21.15.82:0/1673465610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe015e310 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 -- 172.21.15.82:0/1673465610 >> 172.21.15.82:0/1673465610 conn(0x7febe00d09d0 msgr2=0x7febe00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 -- 172.21.15.82:0/1673465610 shutdown_connections 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.110+0000 7febe5d6f640 1 -- 172.21.15.82:0/1673465610 wait complete. 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.111+0000 7febe5d6f640 1 Processor -- start 2024-03-14T12:15:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.111+0000 7febe5d6f640 1 -- start start 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febe5d6f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febe006b420 0x7febe0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febe5d6f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe0149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febe5d6f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febe015df30 0x7febe0159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febe5d6f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe0162650 con 0x7febe015df30 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febe5d6f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00b3c10 con 0x7febe006b420 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febe5d6f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe01627d0 con 0x7febe006f7f0 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febe015df30 0x7febe0159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febe015df30 0x7febe0159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45594/0 (socket says 172.21.15.82:45594) 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 -- 172.21.15.82:0/3968299188 learned_addr learned my addr 172.21.15.82:0/3968299188 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:50.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdeffd640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe0149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdf7fe640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febe006b420 0x7febe0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 -- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 msgr2=0x7febe0149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe0149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 -- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febe006b420 msgr2=0x7febe0154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febe006b420 0x7febe0154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 -- 172.21.15.82:0/3968299188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0153760 con 0x7febe015df30 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdeffd640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe0149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdf7fe640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febe006b420 0x7febe0154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febdffff640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febe015df30 0x7febe0159a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7febd800ef90 tx=0x7febd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febc7fff640 1 -- 172.21.15.82:0/3968299188 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800ccc0 con 0x7febe015df30 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febc7fff640 1 -- 172.21.15.82:0/3968299188 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd8003990 con 0x7febe015df30 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe01583c0 con 0x7febe015df30 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.112+0000 7febc7fff640 1 -- 172.21.15.82:0/3968299188 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8003d80 con 0x7febe015df30 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.113+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe00af660 con 0x7febe015df30 2024-03-14T12:15:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.113+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe00af170 con 0x7febe015df30 2024-03-14T12:15:50.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.114+0000 7febc7fff640 1 -- 172.21.15.82:0/3968299188 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7febd8012050 con 0x7febe015df30 2024-03-14T12:15:50.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.114+0000 7febc7fff640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7febcc06bbd0 0x7febcc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.114+0000 7febc7fff640 1 -- 172.21.15.82:0/3968299188 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febd808ae80 con 0x7febe015df30 2024-03-14T12:15:50.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.114+0000 7febdf7fe640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7febcc06bbd0 0x7febcc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.115+0000 7febdf7fe640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7febcc06bbd0 0x7febcc06e090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febd0000f60 tx=0x7febd0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.120+0000 7febc7fff640 1 -- 172.21.15.82:0/3968299188 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7febd8010040 con 0x7febe015df30 2024-03-14T12:15:50.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.289+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febe004f050 con 0x7febe015df30 2024-03-14T12:15:50.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.291+0000 7febc7fff640 1 -- 172.21.15.82:0/3968299188 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16657 (secure 0 0 0) 0x7febd8059970 con 0x7febe015df30 2024-03-14T12:15:50.291 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:50.791329+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7febcc06bbd0 msgr2=0x7febcc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7febcc06bbd0 0x7febcc06e090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febd0000f60 tx=0x7febd0009040 comp rx=0 tx=0).stop 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febe015df30 msgr2=0x7febe0159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febe015df30 0x7febe0159a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7febd800ef90 tx=0x7febd800c490 comp rx=0 tx=0).stop 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 shutdown_connections 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febe015df30 0x7febe0159a90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7febcc06bbd0 0x7febcc06e090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febe006f7f0 0x7febe0149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 --2- 172.21.15.82:0/3968299188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febe006b420 0x7febe0154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 >> 172.21.15.82:0/3968299188 conn(0x7febe00d09d0 msgr2=0x7febe00d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 shutdown_connections 2024-03-14T12:15:50.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.293+0000 7febe5d6f640 1 -- 172.21.15.82:0/3968299188 wait complete. 2024-03-14T12:15:50.304 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T12:15:50.305 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:15:50.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.405+0000 7f8eb1589640 1 Processor -- start 2024-03-14T12:15:50.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.405+0000 7f8eb1589640 1 -- start start 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eb1589640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8eac0af150 0x7f8eac0af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eb1589640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8eac06b420 0x7f8eac157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eb1589640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac0b01d0 con 0x7f8eac15f1f0 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eb1589640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac15e710 con 0x7f8eac06b420 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eb1589640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac15e9f0 con 0x7f8eac0af150 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eab7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8eac06b420 0x7f8eac157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eaa7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8eac15f1f0 0x7f8eac06ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38176/0 (socket says 172.21.15.82:38176) 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.406+0000 7f8eaa7fc640 1 -- 172.21.15.82:0/754416085 learned_addr learned my addr 172.21.15.82:0/754416085 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8eaaffd640 1 --2- 172.21.15.82:0/754416085 >> v2:172.21.15.164:3300/0 conn(0x7f8eac0af150 0x7f8eac0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8eab7fe640 1 -- 172.21.15.82:0/754416085 >> v2:172.21.15.164:3300/0 conn(0x7f8eac0af150 msgr2=0x7f8eac0af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:50.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8eab7fe640 1 --2- 172.21.15.82:0/754416085 >> v2:172.21.15.164:3300/0 conn(0x7f8eac0af150 0x7f8eac0af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8eab7fe640 1 -- 172.21.15.82:0/754416085 >> v1:172.21.15.82:6789/0 conn(0x7f8eac15f1f0 legacy=0x7f8eac06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8eab7fe640 1 -- 172.21.15.82:0/754416085 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac15ecd0 con 0x7f8eac06b420 2024-03-14T12:15:50.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8ea9ffb640 1 -- 172.21.15.82:0/754416085 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296939576 0 0) 0x7f8eac0b01d0 con 0x7f8eac15f1f0 2024-03-14T12:15:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8eab7fe640 1 --2- 172.21.15.82:0/754416085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac06b420 0x7f8eac157cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8eac04f5e0 tx=0x7f8ea402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193f3079a212ccd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8ea9ffb640 1 -- 172.21.15.82:0/754416085 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40315a0 con 0x7f8eac06b420 2024-03-14T12:15:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8ea9ffb640 1 -- 172.21.15.82:0/754416085 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4037070 con 0x7f8eac06b420 2024-03-14T12:15:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.407+0000 7f8ea9ffb640 1 -- 172.21.15.82:0/754416085 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4031960 con 0x7f8eac06b420 2024-03-14T12:15:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 -- 172.21.15.82:0/754416085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac06b420 msgr2=0x7f8eac157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 --2- 172.21.15.82:0/754416085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac06b420 0x7f8eac157cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8eac04f5e0 tx=0x7f8ea402b840 comp rx=0 tx=0).stop 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 -- 172.21.15.82:0/754416085 shutdown_connections 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 --2- 172.21.15.82:0/754416085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac06b420 0x7f8eac157cd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 --2- 172.21.15.82:0/754416085 >> v2:172.21.15.164:3300/0 conn(0x7f8eac0af150 0x7f8eac0af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 -- 172.21.15.82:0/754416085 >> 172.21.15.82:0/754416085 conn(0x7f8eac0d0a20 msgr2=0x7f8eac0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 -- 172.21.15.82:0/754416085 shutdown_connections 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.408+0000 7f8eb1589640 1 -- 172.21.15.82:0/754416085 wait complete. 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.409+0000 7f8eb1589640 1 Processor -- start 2024-03-14T12:15:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.409+0000 7f8eb1589640 1 -- start start 2024-03-14T12:15:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eb1589640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8eac06b420 0x7f8eac148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eb1589640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac0af150 0x7f8eac0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eb1589640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15f1f0 0x7f8eac15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eb1589640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0aff40 con 0x7f8eac0af150 2024-03-14T12:15:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eb1589640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0afdc0 con 0x7f8eac06b420 2024-03-14T12:15:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eb1589640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac0b58f0 con 0x7f8eac15f1f0 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac0af150 0x7f8eac0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac0af150 0x7f8eac0ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45614/0 (socket says 172.21.15.82:45614) 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 -- 172.21.15.82:0/3144624592 learned_addr learned my addr 172.21.15.82:0/3144624592 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eab7fe640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15f1f0 0x7f8eac15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 -- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15f1f0 msgr2=0x7f8eac15ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaaffd640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8eac06b420 0x7f8eac148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15f1f0 0x7f8eac15ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 -- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8eac06b420 msgr2=0x7f8eac148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8eac06b420 0x7f8eac148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaa7fc640 1 -- 172.21.15.82:0/3144624592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac148a70 con 0x7f8eac0af150 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eab7fe640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15f1f0 0x7f8eac15ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.410+0000 7f8eaaffd640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8eac06b420 0x7f8eac148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.411+0000 7f8eaa7fc640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac0af150 0x7f8eac0ce320 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000c3f0 tx=0x7f8ea000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.411+0000 7f8e937fe640 1 -- 172.21.15.82:0/3144624592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00074e0 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.411+0000 7f8e937fe640 1 -- 172.21.15.82:0/3144624592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea00047e0 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.411+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac158aa0 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.411+0000 7f8e937fe640 1 -- 172.21.15.82:0/3144624592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0004c10 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.411+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac159060 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.412+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac04f050 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.412+0000 7f8e937fe640 1 -- 172.21.15.82:0/3144624592 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8ea0011070 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.413+0000 7f8e937fe640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9806bbd0 0x7f8e9806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.413+0000 7f8eaaffd640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9806bbd0 0x7f8e9806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.413+0000 7f8e937fe640 1 -- 172.21.15.82:0/3144624592 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ea008a620 con 0x7f8eac0af150 2024-03-14T12:15:50.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.413+0000 7f8eaaffd640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9806bbd0 0x7f8e9806e090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c007f50 tx=0x7f8e9c0074a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.417+0000 7f8e937fe640 1 -- 172.21.15.82:0/3144624592 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8ea001a2b0 con 0x7f8eac0af150 2024-03-14T12:15:50.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.625+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8eac06b980 con 0x7f8eac0af150 2024-03-14T12:15:50.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.626+0000 7f8e937fe640 1 -- 172.21.15.82:0/3144624592 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f8ea0059110 con 0x7f8eac0af150 2024-03-14T12:15:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:50.625 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2478084830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2478084830},{"type":"v1","addr":"172.21.15.82:6835","nonce":2478084830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4834,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1811632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1811632800},{"type":"v1","addr":"172.21.15.164:6834","nonce":1811632800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4176962755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4176962755},{"type":"v1","addr":"172.21.15.164:6835","nonce":4176962755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:14:49.135548+0000","modified":"2024-03-14T12:14:51.133632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/682556190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":682556190},{"type":"v1","addr":"172.21.15.82:6837","nonce":682556190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T12:15:50.626 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 15 2024-03-14T12:15:50.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9806bbd0 msgr2=0x7f8e9806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9806bbd0 0x7f8e9806e090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c007f50 tx=0x7f8e9c0074a0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac0af150 msgr2=0x7f8eac0ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac0af150 0x7f8eac0ce320 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000c3f0 tx=0x7f8ea000c8b0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 shutdown_connections 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9806bbd0 0x7f8e9806e090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15f1f0 0x7f8eac15ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac0af150 0x7f8eac0ce320 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 --2- 172.21.15.82:0/3144624592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8eac06b420 0x7f8eac148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 >> 172.21.15.82:0/3144624592 conn(0x7f8eac0d0a20 msgr2=0x7f8eac071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 shutdown_connections 2024-03-14T12:15:50.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.628+0000 7f8eb1589640 1 -- 172.21.15.82:0/3144624592 wait complete. 2024-03-14T12:15:50.638 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:15:50.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.739+0000 7f2e7b643640 1 Processor -- start 2024-03-14T12:15:50.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.739+0000 7f2e7b643640 1 -- start start 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.740+0000 7f2e7b643640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2e740b4220 0x7f2e74161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.740+0000 7f2e7b643640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 0x7f2e7406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.740+0000 7f2e7b643640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7406fa90 con 0x7f2e7415e150 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.740+0000 7f2e7b643640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e7406c750 con 0x7f2e740b4220 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.740+0000 7f2e7b643640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406be20 con 0x7f2e74162010 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.740+0000 7f2e793df640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2e7415e150 0x7f2e7406f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38192/0 (socket says 172.21.15.82:38192) 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.740+0000 7f2e793df640 1 -- 172.21.15.82:0/491895605 learned_addr learned my addr 172.21.15.82:0/491895605 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:50.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e78bde640 1 --2- 172.21.15.82:0/491895605 >> v2:172.21.15.164:3300/0 conn(0x7f2e740b4220 0x7f2e74161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e79be0640 1 --2- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 0x7f2e7406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e6bfff640 1 -- 172.21.15.82:0/491895605 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715511641 0 0) 0x7f2e7406fa90 con 0x7f2e7415e150 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e6bfff640 1 -- 172.21.15.82:0/491895605 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e60003520 con 0x7f2e7415e150 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e6bfff640 1 -- 172.21.15.82:0/491895605 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1062134251 0 0) 0x7f2e60003520 con 0x7f2e7415e150 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e78bde640 1 -- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 msgr2=0x7f2e7406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e78bde640 1 --2- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 0x7f2e7406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e78bde640 1 -- 172.21.15.82:0/491895605 >> v1:172.21.15.82:6789/0 conn(0x7f2e7415e150 legacy=0x7f2e7406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.741+0000 7f2e78bde640 1 -- 172.21.15.82:0/491895605 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7406c100 con 0x7f2e740b4220 2024-03-14T12:15:50.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e78bde640 1 --2- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 0x7f2e74161ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e640097d0 tx=0x7f2e6402b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=697b67a323b6128f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e6bfff640 1 -- 172.21.15.82:0/491895605 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64037070 con 0x7f2e740b4220 2024-03-14T12:15:50.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e6bfff640 1 -- 172.21.15.82:0/491895605 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e64030510 con 0x7f2e740b4220 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e6bfff640 1 -- 172.21.15.82:0/491895605 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70001670 con 0x7f2e740b4220 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e7b643640 1 -- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 msgr2=0x7f2e74161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e7b643640 1 --2- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 0x7f2e74161ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e640097d0 tx=0x7f2e6402b770 comp rx=0 tx=0).stop 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e7b643640 1 -- 172.21.15.82:0/491895605 shutdown_connections 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e7b643640 1 --2- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 0x7f2e7406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e7b643640 1 --2- 172.21.15.82:0/491895605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 0x7f2e74161ad0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.742+0000 7f2e7b643640 1 -- 172.21.15.82:0/491895605 >> 172.21.15.82:0/491895605 conn(0x7f2e740b3670 msgr2=0x7f2e740b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.743+0000 7f2e7b643640 1 -- 172.21.15.82:0/491895605 shutdown_connections 2024-03-14T12:15:50.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.743+0000 7f2e7b643640 1 -- 172.21.15.82:0/491895605 wait complete. 2024-03-14T12:15:50.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 Processor -- start 2024-03-14T12:15:50.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 -- start start 2024-03-14T12:15:50.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 0x7f2e740afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7415e150 0x7f2e740b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 0x7f2e740c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406d2a0 con 0x7f2e7415e150 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406d120 con 0x7f2e740b4220 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.744+0000 7f2e7b643640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406d420 con 0x7f2e74162010 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7415e150 0x7f2e740b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7415e150 0x7f2e740b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45620/0 (socket says 172.21.15.82:45620) 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 -- 172.21.15.82:0/1794286661 learned_addr learned my addr 172.21.15.82:0/1794286661 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 -- 172.21.15.82:0/1794286661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 msgr2=0x7f2e740c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 0x7f2e740c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 -- 172.21.15.82:0/1794286661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 msgr2=0x7f2e740afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 0x7f2e740afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 -- 172.21.15.82:0/1794286661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740c6550 con 0x7f2e7415e150 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e78bde640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7415e150 0x7f2e740b0250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2e64000c00 tx=0x7f2e640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e69ffb640 1 -- 172.21.15.82:0/1794286661 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64040020 con 0x7f2e7415e150 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e69ffb640 1 -- 172.21.15.82:0/1794286661 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e64038040 con 0x7f2e7415e150 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e740c6830 con 0x7f2e7415e150 2024-03-14T12:15:50.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e69ffb640 1 -- 172.21.15.82:0/1794286661 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e64037070 con 0x7f2e7415e150 2024-03-14T12:15:50.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.745+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e740c6df0 con 0x7f2e7415e150 2024-03-14T12:15:50.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.746+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7404f050 con 0x7f2e7415e150 2024-03-14T12:15:50.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.747+0000 7f2e69ffb640 1 -- 172.21.15.82:0/1794286661 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2e64043020 con 0x7f2e7415e150 2024-03-14T12:15:50.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.747+0000 7f2e69ffb640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e5806bc00 0x7f2e5806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:50.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.747+0000 7f2e69ffb640 1 -- 172.21.15.82:0/1794286661 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e640abd80 con 0x7f2e7415e150 2024-03-14T12:15:50.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.747+0000 7f2e793df640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e5806bc00 0x7f2e5806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:50.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.747+0000 7f2e793df640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e5806bc00 0x7f2e5806e0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e70001eb0 tx=0x7f2e70001a40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:50.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.751+0000 7f2e69ffb640 1 -- 172.21.15.82:0/1794286661 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2e6404a030 con 0x7f2e7415e150 2024-03-14T12:15:50.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.963+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e740b7ae0 con 0x7f2e7415e150 2024-03-14T12:15:50.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.964+0000 7f2e69ffb640 1 -- 172.21.15.82:0/1794286661 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f2e6407a900 con 0x7f2e7415e150 2024-03-14T12:15:50.963 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:50.963 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2478084830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2478084830},{"type":"v1","addr":"172.21.15.82:6835","nonce":2478084830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4834,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1811632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1811632800},{"type":"v1","addr":"172.21.15.164:6834","nonce":1811632800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4176962755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4176962755},{"type":"v1","addr":"172.21.15.164:6835","nonce":4176962755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:14:49.135548+0000","modified":"2024-03-14T12:14:51.133632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/682556190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":682556190},{"type":"v1","addr":"172.21.15.82:6837","nonce":682556190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T12:15:50.963 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 15 2024-03-14T12:15:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e5806bc00 msgr2=0x7f2e5806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e5806bc00 0x7f2e5806e0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e70001eb0 tx=0x7f2e70001a40 comp rx=0 tx=0).stop 2024-03-14T12:15:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7415e150 msgr2=0x7f2e740b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7415e150 0x7f2e740b0250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2e64000c00 tx=0x7f2e640046c0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 shutdown_connections 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e5806bc00 0x7f2e5806e0c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e74162010 0x7f2e740c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7415e150 0x7f2e740b0250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 --2- 172.21.15.82:0/1794286661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e740b4220 0x7f2e740afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 >> 172.21.15.82:0/1794286661 conn(0x7f2e740b3670 msgr2=0x7f2e740b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 shutdown_connections 2024-03-14T12:15:50.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:50.966+0000 7f2e7b643640 1 -- 172.21.15.82:0/1794286661 wait complete. 2024-03-14T12:15:50.976 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:15:51.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.076+0000 7f42978ea640 1 Processor -- start 2024-03-14T12:15:51.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f42978ea640 1 -- start start 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f42978ea640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f429015e180 0x7f42900b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f42978ea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f42900d07b0 0x7f4290072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f42978ea640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42900b5080 con 0x7f4290066a20 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f42978ea640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900b5360 con 0x7f42900d07b0 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f42978ea640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900b5640 con 0x7f429015e180 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f4295e87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f42900d07b0 0x7f4290072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f4295e87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f42900d07b0 0x7f4290072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45634/0 (socket says 172.21.15.82:45634) 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.077+0000 7f4295e87640 1 -- 172.21.15.82:0/3900215799 learned_addr learned my addr 172.21.15.82:0/3900215799 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.078+0000 7f4295686640 1 --2- 172.21.15.82:0/3900215799 >> v2:172.21.15.164:3300/0 conn(0x7f429015e180 0x7f42900b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.078+0000 7f4295e87640 1 -- 172.21.15.82:0/3900215799 >> v2:172.21.15.164:3300/0 conn(0x7f429015e180 msgr2=0x7f42900b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.078+0000 7f4295e87640 1 --2- 172.21.15.82:0/3900215799 >> v2:172.21.15.164:3300/0 conn(0x7f429015e180 0x7f42900b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.078+0000 7f4295e87640 1 -- 172.21.15.82:0/3900215799 >> v1:172.21.15.164:6789/0 conn(0x7f4290066a20 legacy=0x7f42900d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.078+0000 7f4295e87640 1 -- 172.21.15.82:0/3900215799 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900b5920 con 0x7f42900d07b0 2024-03-14T12:15:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.078+0000 7f4295e87640 1 --2- 172.21.15.82:0/3900215799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42900d07b0 0x7f4290072cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4288009870 tx=0x7f428802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a19063d996f4d69 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f4287fff640 1 -- 172.21.15.82:0/3900215799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288035020 con 0x7f42900d07b0 2024-03-14T12:15:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f4287fff640 1 -- 172.21.15.82:0/3900215799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4288031870 con 0x7f42900d07b0 2024-03-14T12:15:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f4287fff640 1 -- 172.21.15.82:0/3900215799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288031c40 con 0x7f42900d07b0 2024-03-14T12:15:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 -- 172.21.15.82:0/3900215799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42900d07b0 msgr2=0x7f4290072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 --2- 172.21.15.82:0/3900215799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42900d07b0 0x7f4290072cc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4288009870 tx=0x7f428802b910 comp rx=0 tx=0).stop 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 -- 172.21.15.82:0/3900215799 shutdown_connections 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 --2- 172.21.15.82:0/3900215799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42900d07b0 0x7f4290072cc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 --2- 172.21.15.82:0/3900215799 >> v2:172.21.15.164:3300/0 conn(0x7f429015e180 0x7f42900b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 -- 172.21.15.82:0/3900215799 >> 172.21.15.82:0/3900215799 conn(0x7f42900721a0 msgr2=0x7f42900725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 -- 172.21.15.82:0/3900215799 shutdown_connections 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.079+0000 7f42978ea640 1 -- 172.21.15.82:0/3900215799 wait complete. 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.080+0000 7f42978ea640 1 Processor -- start 2024-03-14T12:15:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.080+0000 7f42978ea640 1 -- start start 2024-03-14T12:15:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f42978ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290066a20 0x7f42900c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f42978ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42900d07b0 0x7f42901f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f42978ea640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f429015e180 0x7f42901f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f42978ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42900afa50 con 0x7f4290066a20 2024-03-14T12:15:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f42978ea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42900b00b0 con 0x7f42900d07b0 2024-03-14T12:15:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f42978ea640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42901f4f70 con 0x7f429015e180 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290066a20 0x7f42900c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290066a20 0x7f42900c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45644/0 (socket says 172.21.15.82:45644) 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 -- 172.21.15.82:0/358067111 learned_addr learned my addr 172.21.15.82:0/358067111 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4294e85640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42900d07b0 0x7f42901f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 -- 172.21.15.82:0/358067111 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f429015e180 msgr2=0x7f42901f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f429015e180 0x7f42901f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 -- 172.21.15.82:0/358067111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42900d07b0 msgr2=0x7f42901f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42900d07b0 0x7f42901f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.081+0000 7f4295686640 1 -- 172.21.15.82:0/358067111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42901f3dd0 con 0x7f4290066a20 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.082+0000 7f4295686640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290066a20 0x7f42900c9250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f428c007e40 tx=0x7f428c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.082+0000 7f4285ffb640 1 -- 172.21.15.82:0/358067111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c014620 con 0x7f4290066a20 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.082+0000 7f4285ffb640 1 -- 172.21.15.82:0/358067111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c019070 con 0x7f4290066a20 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.082+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42900c9930 con 0x7f4290066a20 2024-03-14T12:15:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.082+0000 7f4285ffb640 1 -- 172.21.15.82:0/358067111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c014a00 con 0x7f4290066a20 2024-03-14T12:15:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.082+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42900c9ef0 con 0x7f4290066a20 2024-03-14T12:15:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.083+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42900ca230 con 0x7f4290066a20 2024-03-14T12:15:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.083+0000 7f4285ffb640 1 -- 172.21.15.82:0/358067111 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f428c014ba0 con 0x7f4290066a20 2024-03-14T12:15:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.084+0000 7f4285ffb640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f427406bbd0 0x7f427406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.084+0000 7f4285ffb640 1 -- 172.21.15.82:0/358067111 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f428c08b330 con 0x7f4290066a20 2024-03-14T12:15:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.084+0000 7f4294e85640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f427406bbd0 0x7f427406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.084+0000 7f4294e85640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f427406bbd0 0x7f427406e090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f429015a9c0 tx=0x7f4280009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.088+0000 7f4285ffb640 1 -- 172.21.15.82:0/358067111 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f42900ca230 con 0x7f4290066a20 2024-03-14T12:15:51.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.259+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f429004f050 con 0x7f4290066a20 2024-03-14T12:15:51.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.260+0000 7f4285ffb640 1 -- 172.21.15.82:0/358067111 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f428c059e20 con 0x7f4290066a20 2024-03-14T12:15:51.259 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:51.260 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:14:50.791329+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:14:47.564085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T12:14:48.095794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f427406bbd0 msgr2=0x7f427406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f427406bbd0 0x7f427406e090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f429015a9c0 tx=0x7f4280009040 comp rx=0 tx=0).stop 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290066a20 msgr2=0x7f42900c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290066a20 0x7f42900c9250 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f428c007e40 tx=0x7f428c00e4b0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 shutdown_connections 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f427406bbd0 0x7f427406e090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f429015e180 0x7f42901f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42900d07b0 0x7f42901f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 --2- 172.21.15.82:0/358067111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290066a20 0x7f42900c9250 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 >> 172.21.15.82:0/358067111 conn(0x7f42900721a0 msgr2=0x7f42900cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 shutdown_connections 2024-03-14T12:15:51.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.262+0000 7f42978ea640 1 -- 172.21.15.82:0/358067111 wait complete. 2024-03-14T12:15:51.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T12:15:51.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.375+0000 7f3965812640 1 Processor -- start 2024-03-14T12:15:51.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.375+0000 7f3965812640 1 -- start start 2024-03-14T12:15:51.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.375+0000 7f3965812640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600cf370 0x7f39600d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.375+0000 7f3965812640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f39600b7720 0x7f39600b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.375+0000 7f3965812640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600cf8c0 con 0x7f39600d0760 2024-03-14T12:15:51.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.375+0000 7f3965812640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39600cfba0 con 0x7f39600b7720 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.375+0000 7f3965812640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b6c40 con 0x7f39600cf370 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395f7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f39600b7720 0x7f39600b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f3957fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600cf370 0x7f39600d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395f7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f39600b7720 0x7f39600b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:42770/0 (socket says 172.21.15.82:42770) 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395effd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f39600d0760 0x7f39600d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:59530/0 (socket says 172.21.15.82:59530) 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395f7fe640 1 -- 172.21.15.82:0/3587105860 learned_addr learned my addr 172.21.15.82:0/3587105860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395f7fe640 1 -- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600cf370 msgr2=0x7f39600d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395f7fe640 1 --2- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600cf370 0x7f39600d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395f7fe640 1 -- 172.21.15.82:0/3587105860 >> v1:172.21.15.164:6789/0 conn(0x7f39600d0760 legacy=0x7f39600d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395e7fc640 1 -- 172.21.15.82:0/3587105860 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493209675 0 0) 0x7f39600cf8c0 con 0x7f39600d0760 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f395f7fe640 1 -- 172.21.15.82:0/3587105860 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600b6f20 con 0x7f39600b7720 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.376+0000 7f3957fff640 1 --2- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600cf370 0x7f39600d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.377+0000 7f395f7fe640 1 --2- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 0x7f39600b7bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3958009e50 tx=0x7f395802bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f476f22788518cc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.377+0000 7f395e7fc640 1 -- 172.21.15.82:0/3587105860 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395802cb90 con 0x7f39600b7720 2024-03-14T12:15:51.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.377+0000 7f395e7fc640 1 -- 172.21.15.82:0/3587105860 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395802cd30 con 0x7f39600b7720 2024-03-14T12:15:51.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.377+0000 7f395e7fc640 1 -- 172.21.15.82:0/3587105860 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958034610 con 0x7f39600b7720 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 -- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 msgr2=0x7f39600b7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 --2- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 0x7f39600b7bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3958009e50 tx=0x7f395802bff0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 -- 172.21.15.82:0/3587105860 shutdown_connections 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 --2- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 0x7f39600b7bb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 --2- 172.21.15.82:0/3587105860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600cf370 0x7f39600d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 -- 172.21.15.82:0/3587105860 >> 172.21.15.82:0/3587105860 conn(0x7f396009d030 msgr2=0x7f39600d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 -- 172.21.15.82:0/3587105860 shutdown_connections 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.378+0000 7f3965812640 1 -- 172.21.15.82:0/3587105860 wait complete. 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.379+0000 7f3965812640 1 Processor -- start 2024-03-14T12:15:51.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.379+0000 7f3965812640 1 -- start start 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3965812640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 0x7f3960155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3965812640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39600cf370 0x7f396009fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3965812640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600d0760 0x7f396009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3965812640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601617a0 con 0x7f39600cf370 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3965812640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600d4ba0 con 0x7f39600b7720 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3965812640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3960161920 con 0x7f39600d0760 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39600cf370 0x7f396009fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39600cf370 0x7f396009fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45648/0 (socket says 172.21.15.82:45648) 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 -- 172.21.15.82:0/2422527500 learned_addr learned my addr 172.21.15.82:0/2422527500 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f395effd640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 0x7f3960155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f395f7fe640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600d0760 0x7f396009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 -- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600d0760 msgr2=0x7f396009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600d0760 0x7f396009daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 -- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 msgr2=0x7f3960155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 0x7f3960155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 -- 172.21.15.82:0/2422527500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600c86c0 con 0x7f39600cf370 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3957fff640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39600cf370 0x7f396009fe10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f394800ebe0 tx=0x7f394800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f395cff9640 1 -- 172.21.15.82:0/2422527500 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948017070 con 0x7f39600cf370 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f395cff9640 1 -- 172.21.15.82:0/2422527500 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394800cc30 con 0x7f39600cf370 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39600c8980 con 0x7f39600cf370 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.380+0000 7f395cff9640 1 -- 172.21.15.82:0/2422527500 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948013470 con 0x7f39600cf370 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.381+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39600c9220 con 0x7f39600cf370 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.382+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3960072dc0 con 0x7f39600cf370 2024-03-14T12:15:51.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.382+0000 7f395cff9640 1 -- 172.21.15.82:0/2422527500 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f394801a070 con 0x7f39600cf370 2024-03-14T12:15:51.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.382+0000 7f395cff9640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f394006bc00 0x7f394006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.382+0000 7f395cff9640 1 -- 172.21.15.82:0/2422527500 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f394808a8e0 con 0x7f39600cf370 2024-03-14T12:15:51.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.382+0000 7f395effd640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f394006bc00 0x7f394006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.383+0000 7f395effd640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f394006bc00 0x7f394006e0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3950007fd0 tx=0x7f3950007de0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.387+0000 7f395cff9640 1 -- 172.21.15.82:0/2422527500 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3960072dc0 con 0x7f39600cf370 2024-03-14T12:15:51.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.593+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f396004f050 con 0x7f39600cf370 2024-03-14T12:15:51.592 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-14T12:15:51.593+0000 7f8354c85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T12:15:51.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.595+0000 7f395cff9640 1 -- 172.21.15.82:0/2422527500 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3948058d80 con 0x7f39600cf370 2024-03-14T12:15:51.594 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T12:15:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f394006bc00 msgr2=0x7f394006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f394006bc00 0x7f394006e0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3950007fd0 tx=0x7f3950007de0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39600cf370 msgr2=0x7f396009fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39600cf370 0x7f396009fe10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f394800ebe0 tx=0x7f394800c520 comp rx=0 tx=0).stop 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 shutdown_connections 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f394006bc00 0x7f394006e0c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f39600d0760 0x7f396009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39600cf370 0x7f396009fe10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 --2- 172.21.15.82:0/2422527500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39600b7720 0x7f3960155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 >> 172.21.15.82:0/2422527500 conn(0x7f396009d030 msgr2=0x7f3960160630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 shutdown_connections 2024-03-14T12:15:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.598+0000 7f3965812640 1 -- 172.21.15.82:0/2422527500 wait complete. 2024-03-14T12:15:51.608 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T12:15:51.634 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-14T12:15:51.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.712+0000 7fe9deb69640 1 Processor -- start 2024-03-14T12:15:51.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.712+0000 7fe9deb69640 1 -- start start 2024-03-14T12:15:51.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.713+0000 7fe9deb69640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe9d80b41d0 0x7fe9d815ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.713+0000 7fe9deb69640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80b3450 con 0x7fe9d80cf3f0 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.713+0000 7fe9deb69640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d815f4b0 con 0x7fe9d80d02d0 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.713+0000 7fe9deb69640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d80cf7d0 con 0x7fe9d80b41d0 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.713+0000 7fe9dc905640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe9d80cf3f0 0x7fe9d80cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46932/0 (socket says 172.21.15.82:46932) 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.713+0000 7fe9dc905640 1 -- 172.21.15.82:0/4232158261 learned_addr learned my addr 172.21.15.82:0/4232158261 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.713+0000 7fe9dd106640 1 --2- 172.21.15.82:0/4232158261 >> v2:172.21.15.164:3300/0 conn(0x7fe9d80b41d0 0x7fe9d815ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.714+0000 7fe9d77fe640 1 -- 172.21.15.82:0/4232158261 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013018966 0 0) 0x7fe9d80b3450 con 0x7fe9d80cf3f0 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.714+0000 7fe9d77fe640 1 -- 172.21.15.82:0/4232158261 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c0003520 con 0x7fe9d80cf3f0 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.714+0000 7fe9dd106640 1 -- 172.21.15.82:0/4232158261 >> v1:172.21.15.164:6789/0 conn(0x7fe9d80d02d0 legacy=0x7fe9d815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.714+0000 7fe9dd106640 1 -- 172.21.15.82:0/4232158261 >> v1:172.21.15.82:6789/0 conn(0x7fe9d80cf3f0 legacy=0x7fe9d80cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.714+0000 7fe9dd106640 1 -- 172.21.15.82:0/4232158261 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80b36f0 con 0x7fe9d80b41d0 2024-03-14T12:15:51.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.714+0000 7fe9dd106640 1 --2- 172.21.15.82:0/4232158261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 0x7fe9d815ee90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009d90 tx=0x7fe9d002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2e2509784bf55f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9d77fe640 1 -- 172.21.15.82:0/4232158261 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0039070 con 0x7fe9d80b41d0 2024-03-14T12:15:51.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9d77fe640 1 -- 172.21.15.82:0/4232158261 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d002cc60 con 0x7fe9d80b41d0 2024-03-14T12:15:51.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9d77fe640 1 -- 172.21.15.82:0/4232158261 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0034660 con 0x7fe9d80b41d0 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9deb69640 1 -- 172.21.15.82:0/4232158261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 msgr2=0x7fe9d815ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9deb69640 1 --2- 172.21.15.82:0/4232158261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 0x7fe9d815ee90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009d90 tx=0x7fe9d002beb0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9deb69640 1 -- 172.21.15.82:0/4232158261 shutdown_connections 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9deb69640 1 --2- 172.21.15.82:0/4232158261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 0x7fe9d815ee90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9deb69640 1 -- 172.21.15.82:0/4232158261 >> 172.21.15.82:0/4232158261 conn(0x7fe9d806aef0 msgr2=0x7fe9d806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9deb69640 1 -- 172.21.15.82:0/4232158261 shutdown_connections 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.715+0000 7fe9deb69640 1 -- 172.21.15.82:0/4232158261 wait complete. 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.716+0000 7fe9deb69640 1 Processor -- start 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.716+0000 7fe9deb69640 1 -- start start 2024-03-14T12:15:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9deb69640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 0x7fe9d80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9deb69640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9d80cf3f0 0x7fe9d80cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9deb69640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d80d02d0 0x7fe9d80ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9deb69640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80707a0 con 0x7fe9d80d02d0 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9deb69640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80b3cb0 con 0x7fe9d80b41d0 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9deb69640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8070920 con 0x7fe9d80cf3f0 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dd106640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d80d02d0 0x7fe9d80ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dd106640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d80d02d0 0x7fe9d80ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57538/0 (socket says 172.21.15.82:57538) 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dd106640 1 -- 172.21.15.82:0/2777966320 learned_addr learned my addr 172.21.15.82:0/2777966320 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dc905640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 0x7fe9d80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9d7fff640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9d80cf3f0 0x7fe9d80cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dd106640 1 -- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9d80cf3f0 msgr2=0x7fe9d80cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dd106640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9d80cf3f0 0x7fe9d80cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dd106640 1 -- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 msgr2=0x7fe9d80ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.717+0000 7fe9dd106640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 0x7fe9d80ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:51.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.718+0000 7fe9dd106640 1 -- 172.21.15.82:0/2777966320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80ab110 con 0x7fe9d80d02d0 2024-03-14T12:15:51.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.718+0000 7fe9dd106640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d80d02d0 0x7fe9d80ad0d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009d90 tx=0x7fe9d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.718+0000 7fe9d57fa640 1 -- 172.21.15.82:0/2777966320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0039050 con 0x7fe9d80d02d0 2024-03-14T12:15:51.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.718+0000 7fe9d57fa640 1 -- 172.21.15.82:0/2777966320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0045070 con 0x7fe9d80d02d0 2024-03-14T12:15:51.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.718+0000 7fe9d57fa640 1 -- 172.21.15.82:0/2777966320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0009450 con 0x7fe9d80d02d0 2024-03-14T12:15:51.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.718+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d80ab3e0 con 0x7fe9d80d02d0 2024-03-14T12:15:51.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.718+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d80ab9a0 con 0x7fe9d80d02d0 2024-03-14T12:15:51.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.719+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d804f050 con 0x7fe9d80d02d0 2024-03-14T12:15:51.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.719+0000 7fe9d57fa640 1 -- 172.21.15.82:0/2777966320 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe9d003e850 con 0x7fe9d80d02d0 2024-03-14T12:15:51.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.719+0000 7fe9d57fa640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9c406bbd0 0x7fe9c406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:51.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.720+0000 7fe9d57fa640 1 -- 172.21.15.82:0/2777966320 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe9d00ad950 con 0x7fe9d80d02d0 2024-03-14T12:15:51.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.720+0000 7fe9dc905640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9c406bbd0 0x7fe9c406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:51.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.720+0000 7fe9dc905640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9c406bbd0 0x7fe9c406e090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80011e0 tx=0x7fe9c8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:51.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.724+0000 7fe9d57fa640 1 -- 172.21.15.82:0/2777966320 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe9d0032e20 con 0x7fe9d80d02d0 2024-03-14T12:15:51.918 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:15:51.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:15:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:51.932+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9d806d320 con 0x7fe9d80d02d0 2024-03-14T12:15:52.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.599+0000 7fe9d57fa640 1 -- 172.21.15.82:0/2777966320 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9d007c500 con 0x7fe9d80d02d0 2024-03-14T12:15:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.601+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9c406bbd0 msgr2=0x7fe9c406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.601+0000 7fe9deb69640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9c406bbd0 0x7fe9c406e090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80011e0 tx=0x7fe9c8009040 comp rx=0 tx=0).stop 2024-03-14T12:15:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.601+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d80d02d0 msgr2=0x7fe9d80ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.601+0000 7fe9deb69640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d80d02d0 0x7fe9d80ad0d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009d90 tx=0x7fe9d0004a40 comp rx=0 tx=0).stop 2024-03-14T12:15:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 shutdown_connections 2024-03-14T12:15:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d80d02d0 0x7fe9d80ad0d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9c406bbd0 0x7fe9c406e090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9d80cf3f0 0x7fe9d80cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 --2- 172.21.15.82:0/2777966320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d80b41d0 0x7fe9d80ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 >> 172.21.15.82:0/2777966320 conn(0x7fe9d806aef0 msgr2=0x7fe9d80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:52.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 shutdown_connections 2024-03-14T12:15:52.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.602+0000 7fe9deb69640 1 -- 172.21.15.82:0/2777966320 wait complete. 2024-03-14T12:15:52.614 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T12:15:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.721+0000 7fd1f4644640 1 Processor -- start 2024-03-14T12:15:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.722+0000 7fd1f4644640 1 -- start start 2024-03-14T12:15:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.722+0000 7fd1f4644640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1ec15dfd0 0x7fd1ec0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.722+0000 7fd1f4644640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec06cf60 0x7fd1ec06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.722+0000 7fd1f4644640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0cfef0 con 0x7fd1ec0cfaa0 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.722+0000 7fd1f4644640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec06ad10 con 0x7fd1ec15dfd0 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.722+0000 7fd1f4644640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0ae4f0 con 0x7fd1ec06cf60 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f23e0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1ec15dfd0 0x7fd1ec0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f2be1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec06cf60 0x7fd1ec06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f2be1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec06cf60 0x7fd1ec06d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:49714/0 (socket says 172.21.15.82:49714) 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f23e0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd1ec15dfd0 0x7fd1ec0ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:60330/0 (socket says 172.21.15.82:60330) 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f2be1640 1 -- 172.21.15.82:0/627021308 learned_addr learned my addr 172.21.15.82:0/627021308 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f1bdf640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd1ec0cfaa0 0x7fd1ec06ba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:55556/0 (socket says 172.21.15.82:55556) 2024-03-14T12:15:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f23e0640 1 -- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec06cf60 msgr2=0x7fd1ec06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f23e0640 1 --2- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec06cf60 0x7fd1ec06d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f23e0640 1 -- 172.21.15.82:0/627021308 >> v1:172.21.15.164:6789/0 conn(0x7fd1ec0cfaa0 legacy=0x7fd1ec06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f23e0640 1 -- 172.21.15.82:0/627021308 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec06af60 con 0x7fd1ec15dfd0 2024-03-14T12:15:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f13de640 1 -- 172.21.15.82:0/627021308 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333191443 0 0) 0x7fd1ec0cfef0 con 0x7fd1ec0cfaa0 2024-03-14T12:15:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.723+0000 7fd1f2be1640 1 --2- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec06cf60 0x7fd1ec06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:15:52.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.724+0000 7fd1f23e0640 1 --2- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 0x7fd1ec0ade30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80097a0 tx=0x7fd1e802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b9b322d65bcee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.724+0000 7fd1f13de640 1 -- 172.21.15.82:0/627021308 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8035020 con 0x7fd1ec15dfd0 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.724+0000 7fd1f13de640 1 -- 172.21.15.82:0/627021308 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8031830 con 0x7fd1ec15dfd0 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.724+0000 7fd1f13de640 1 -- 172.21.15.82:0/627021308 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8031c00 con 0x7fd1ec15dfd0 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 -- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 msgr2=0x7fd1ec0ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 --2- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 0x7fd1ec0ade30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80097a0 tx=0x7fd1e802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 -- 172.21.15.82:0/627021308 shutdown_connections 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 --2- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec06cf60 0x7fd1ec06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 --2- 172.21.15.82:0/627021308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 0x7fd1ec0ade30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 -- 172.21.15.82:0/627021308 >> 172.21.15.82:0/627021308 conn(0x7fd1ec0d0110 msgr2=0x7fd1ec0d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 -- 172.21.15.82:0/627021308 shutdown_connections 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.725+0000 7fd1f4644640 1 -- 172.21.15.82:0/627021308 wait complete. 2024-03-14T12:15:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.726+0000 7fd1f4644640 1 Processor -- start 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.726+0000 7fd1f4644640 1 -- start start 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f4644640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1ec06cf60 0x7fd1ec15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f4644640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec0cfaa0 0x7fd1ec0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f4644640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 0x7fd1ec1f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f4644640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06c150 con 0x7fd1ec06cf60 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f4644640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06b520 con 0x7fd1ec15dfd0 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f4644640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec15e3b0 con 0x7fd1ec0cfaa0 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1ec06cf60 0x7fd1ec15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1ec06cf60 0x7fd1ec15d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57542/0 (socket says 172.21.15.82:57542) 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 -- 172.21.15.82:0/2406065021 learned_addr learned my addr 172.21.15.82:0/2406065021 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:52.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f1bdf640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec0cfaa0 0x7fd1ec0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f2be1640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 0x7fd1ec1f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 -- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec0cfaa0 msgr2=0x7fd1ec0c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec0cfaa0 0x7fd1ec0c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 -- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 msgr2=0x7fd1ec1f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 0x7fd1ec1f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 -- 172.21.15.82:0/2406065021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0b4880 con 0x7fd1ec06cf60 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.727+0000 7fd1f23e0640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1ec06cf60 0x7fd1ec15d550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80097a0 tx=0x7fd1e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.728+0000 7fd1e2ffd640 1 -- 172.21.15.82:0/2406065021 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8035040 con 0x7fd1ec06cf60 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.728+0000 7fd1e2ffd640 1 -- 172.21.15.82:0/2406065021 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8004230 con 0x7fd1ec06cf60 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.728+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec0b4b60 con 0x7fd1ec06cf60 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.728+0000 7fd1e2ffd640 1 -- 172.21.15.82:0/2406065021 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e803a640 con 0x7fd1ec06cf60 2024-03-14T12:15:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.728+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec0b5120 con 0x7fd1ec06cf60 2024-03-14T12:15:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.729+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec15da90 con 0x7fd1ec06cf60 2024-03-14T12:15:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.729+0000 7fd1e2ffd640 1 -- 172.21.15.82:0/2406065021 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd1e8038020 con 0x7fd1ec06cf60 2024-03-14T12:15:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.730+0000 7fd1e2ffd640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd1d006bbd0 0x7fd1d006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.730+0000 7fd1e2ffd640 1 -- 172.21.15.82:0/2406065021 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd1e80ac7e0 con 0x7fd1ec06cf60 2024-03-14T12:15:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.730+0000 7fd1f1bdf640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd1d006bbd0 0x7fd1d006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.730+0000 7fd1f1bdf640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd1d006bbd0 0x7fd1d006e090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec09cf90 tx=0x7fd1dc007c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:52.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.734+0000 7fd1e2ffd640 1 -- 172.21.15.82:0/2406065021 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd1e807b300 con 0x7fd1ec06cf60 2024-03-14T12:15:52.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:52.908+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd1ec04f050 con 0x7fd1ec06cf60 2024-03-14T12:15:53.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.601+0000 7fd1e2ffd640 1 -- 172.21.15.82:0/2406065021 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd1e807ac00 con 0x7fd1ec06cf60 2024-03-14T12:15:53.600 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-03-14T12:15:53.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd1d006bbd0 msgr2=0x7fd1d006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd1d006bbd0 0x7fd1d006e090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec09cf90 tx=0x7fd1dc007c00 comp rx=0 tx=0).stop 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1ec06cf60 msgr2=0x7fd1ec15d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1ec06cf60 0x7fd1ec15d550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80097a0 tx=0x7fd1e80046c0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 shutdown_connections 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec15dfd0 0x7fd1ec1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd1d006bbd0 0x7fd1d006e090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd1ec0cfaa0 0x7fd1ec0c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 --2- 172.21.15.82:0/2406065021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1ec06cf60 0x7fd1ec15d550 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 >> 172.21.15.82:0/2406065021 conn(0x7fd1ec0d0110 msgr2=0x7fd1ec0afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 shutdown_connections 2024-03-14T12:15:53.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.604+0000 7fd1f4644640 1 -- 172.21.15.82:0/2406065021 wait complete. 2024-03-14T12:15:53.616 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T12:15:53.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.725+0000 7f57d5f05640 1 Processor -- start 2024-03-14T12:15:53.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.725+0000 7f57d5f05640 1 -- start start 2024-03-14T12:15:53.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57d5f05640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f57d01617d0 0x7f57d00d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57d5f05640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f57d00b4cd0 0x7f57d00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57d5f05640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d00cfa50 0x7f57d00d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57d5f05640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d0161bb0 con 0x7f57d00b4cd0 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57d5f05640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d00d4840 con 0x7f57d01617d0 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57d5f05640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00d4b20 con 0x7f57d00cfa50 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57ceffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f57d00b4cd0 0x7f57d00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57ceffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f57d00b4cd0 0x7f57d00cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57554/0 (socket says 172.21.15.82:57554) 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.726+0000 7f57ceffd640 1 -- 172.21.15.82:0/2889264508 learned_addr learned my addr 172.21.15.82:0/2889264508 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.727+0000 7f57ceffd640 1 -- 172.21.15.82:0/2889264508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d00cfa50 msgr2=0x7f57d00d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.727+0000 7f57ceffd640 1 --2- 172.21.15.82:0/2889264508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d00cfa50 0x7f57d00d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.727+0000 7f57ceffd640 1 -- 172.21.15.82:0/2889264508 >> v2:172.21.15.164:3300/0 conn(0x7f57d01617d0 msgr2=0x7f57d00d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:53.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.727+0000 7f57ceffd640 1 --2- 172.21.15.82:0/2889264508 >> v2:172.21.15.164:3300/0 conn(0x7f57d01617d0 0x7f57d00d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.727+0000 7f57ceffd640 1 -- 172.21.15.82:0/2889264508 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d00cea30 con 0x7f57d00b4cd0 2024-03-14T12:15:53.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.727+0000 7f57ceffd640 1 --2- 172.21.15.82:0/2889264508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00b4cd0 0x7f57d00cf510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f57d00b6870 tx=0x7f57c402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15914703938f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:53.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.727+0000 7f57ce7fc640 1 -- 172.21.15.82:0/2889264508 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4035020 con 0x7f57d00b4cd0 2024-03-14T12:15:53.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57ce7fc640 1 -- 172.21.15.82:0/2889264508 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c4031890 con 0x7f57d00b4cd0 2024-03-14T12:15:53.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57ce7fc640 1 -- 172.21.15.82:0/2889264508 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4031c60 con 0x7f57d00b4cd0 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 -- 172.21.15.82:0/2889264508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00b4cd0 msgr2=0x7f57d00cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 --2- 172.21.15.82:0/2889264508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00b4cd0 0x7f57d00cf510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f57d00b6870 tx=0x7f57c402b930 comp rx=0 tx=0).stop 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 -- 172.21.15.82:0/2889264508 shutdown_connections 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 --2- 172.21.15.82:0/2889264508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d00cfa50 0x7f57d00d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 --2- 172.21.15.82:0/2889264508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00b4cd0 0x7f57d00cf510 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 --2- 172.21.15.82:0/2889264508 >> v2:172.21.15.164:3300/0 conn(0x7f57d01617d0 0x7f57d00d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 -- 172.21.15.82:0/2889264508 >> 172.21.15.82:0/2889264508 conn(0x7f57d00d3c80 msgr2=0x7f57d00d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:53.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 -- 172.21.15.82:0/2889264508 shutdown_connections 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.728+0000 7f57d5f05640 1 -- 172.21.15.82:0/2889264508 wait complete. 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.729+0000 7f57d5f05640 1 Processor -- start 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.729+0000 7f57d5f05640 1 -- start start 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57d5f05640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57d00b4cd0 0x7f57d015e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57d5f05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00cfa50 0x7f57d00ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57d5f05640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d01617d0 0x7f57d00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57d5f05640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00d0fe0 con 0x7f57d00cfa50 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57d5f05640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00ceff0 con 0x7f57d00b4cd0 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57d5f05640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00d27a0 con 0x7f57d01617d0 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57ceffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00cfa50 0x7f57d00ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57ceffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00cfa50 0x7f57d00ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57556/0 (socket says 172.21.15.82:57556) 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57ceffd640 1 -- 172.21.15.82:0/3482035156 learned_addr learned my addr 172.21.15.82:0/3482035156 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57cf7fe640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57d00b4cd0 0x7f57d015e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.730+0000 7f57cffff640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d01617d0 0x7f57d00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:53.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57ceffd640 1 -- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d01617d0 msgr2=0x7f57d00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57ceffd640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d01617d0 0x7f57d00cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57ceffd640 1 -- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57d00b4cd0 msgr2=0x7f57d015e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57ceffd640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57d00b4cd0 0x7f57d015e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57ceffd640 1 -- 172.21.15.82:0/3482035156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d01f4f30 con 0x7f57d00cfa50 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57cf7fe640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57d00b4cd0 0x7f57d015e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57cffff640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d01617d0 0x7f57d00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57ceffd640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00cfa50 0x7f57d00ccfc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000c00 tx=0x7f57c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57b7fff640 1 -- 172.21.15.82:0/3482035156 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c403ad60 con 0x7f57d00cfa50 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57b7fff640 1 -- 172.21.15.82:0/3482035156 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c40443f0 con 0x7f57d00cfa50 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d01f4170 con 0x7f57d00cfa50 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57b7fff640 1 -- 172.21.15.82:0/3482035156 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40447c0 con 0x7f57d00cfa50 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.731+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d01f4730 con 0x7f57d00cfa50 2024-03-14T12:15:53.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.732+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d004f050 con 0x7f57d00cfa50 2024-03-14T12:15:53.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.732+0000 7f57b7fff640 1 -- 172.21.15.82:0/3482035156 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f57c404b060 con 0x7f57d00cfa50 2024-03-14T12:15:53.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.733+0000 7f57b7fff640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57bc06bc00 0x7f57bc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:53.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.733+0000 7f57b7fff640 1 -- 172.21.15.82:0/3482035156 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f57c40acb90 con 0x7f57d00cfa50 2024-03-14T12:15:53.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.733+0000 7f57cf7fe640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57bc06bc00 0x7f57bc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:53.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.733+0000 7f57cf7fe640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57bc06bc00 0x7f57bc06e0c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57c000a870 tx=0x7f57c0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:53.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.737+0000 7f57b7fff640 1 -- 172.21.15.82:0/3482035156 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f57c403f070 con 0x7f57d00cfa50 2024-03-14T12:15:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:53.911+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57d00d2040 con 0x7f57d00cfa50 2024-03-14T12:15:54.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.602+0000 7f57b7fff640 1 -- 172.21.15.82:0/3482035156 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f57c402d040 con 0x7f57d00cfa50 2024-03-14T12:15:54.602 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-03-14T12:15:54.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.605+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57bc06bc00 msgr2=0x7f57bc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.605+0000 7f57d5f05640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57bc06bc00 0x7f57bc06e0c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57c000a870 tx=0x7f57c0008040 comp rx=0 tx=0).stop 2024-03-14T12:15:54.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.605+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00cfa50 msgr2=0x7f57d00ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.605+0000 7f57d5f05640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00cfa50 0x7f57d00ccfc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000c00 tx=0x7f57c40046c0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 shutdown_connections 2024-03-14T12:15:54.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f57bc06bc00 0x7f57bc06e0c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f57d01617d0 0x7f57d00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57d00cfa50 0x7f57d00ccfc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 --2- 172.21.15.82:0/3482035156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57d00b4cd0 0x7f57d015e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 >> 172.21.15.82:0/3482035156 conn(0x7f57d00d3c80 msgr2=0x7f57d006be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:54.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 shutdown_connections 2024-03-14T12:15:54.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.606+0000 7f57d5f05640 1 -- 172.21.15.82:0/3482035156 wait complete. 2024-03-14T12:15:54.620 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-14T12:15:54.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.723+0000 7f60805df640 1 Processor -- start 2024-03-14T12:15:54.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.723+0000 7f60805df640 1 -- start start 2024-03-14T12:15:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.724+0000 7f60805df640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f60780d12a0 0x7f60780d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.724+0000 7f60805df640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 0x7f60780cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.724+0000 7f60805df640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60780d25a0 con 0x7f607815dff0 2024-03-14T12:15:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.724+0000 7f60805df640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60780d2350 con 0x7f60780d12a0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.724+0000 7f60805df640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780d2880 con 0x7f60780d0c40 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.724+0000 7f607e37b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f607815dff0 0x7f60780d4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46940/0 (socket says 172.21.15.82:46940) 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.724+0000 7f607e37b640 1 -- 172.21.15.82:0/3790076696 learned_addr learned my addr 172.21.15.82:0/3790076696 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607db7a640 1 --2- 172.21.15.82:0/3790076696 >> v2:172.21.15.164:3300/0 conn(0x7f60780d12a0 0x7f60780d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607eb7c640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 0x7f60780cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694987503 0 0) 0x7f60780d25a0 con 0x7f607815dff0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6064003520 con 0x7f607815dff0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177999080 0 0) 0x7f6064003520 con 0x7f607815dff0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 msgr2=0x7f60780cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 0x7f60780cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 >> v2:172.21.15.164:3300/0 conn(0x7f60780d12a0 msgr2=0x7f60780d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 --2- 172.21.15.82:0/3790076696 >> v2:172.21.15.164:3300/0 conn(0x7f60780d12a0 0x7f60780d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60780d2b60 con 0x7f607815dff0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f60740019f0 con 0x7f607815dff0 2024-03-14T12:15:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 >> v1:172.21.15.82:6789/0 conn(0x7f607815dff0 legacy=0x7f60780d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60640039b0 0x7f6064003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 0x7f6064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6064004d40 0x7f60640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780d2350 con 0x7f60640039b0 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780d2880 con 0x7f6064004d40 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606400b9c0 con 0x7f60780d0c40 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607e37b640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60640039b0 0x7f6064003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607db7a640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 0x7f6064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607eb7c640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6064004d40 0x7f60640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607e37b640 1 -- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 msgr2=0x7f6064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607e37b640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60780d0c40 0x7f6064004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607e37b640 1 -- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6064004d40 msgr2=0x7f60640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607e37b640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6064004d40 0x7f60640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.725+0000 7f607e37b640 1 -- 172.21.15.82:0/3790076696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6064003520 con 0x7f60640039b0 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.726+0000 7f607e37b640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60640039b0 0x7f6064003dd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f607400a890 tx=0x7f6064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8a3175ad43129b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.726+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400a4e0 con 0x7f60640039b0 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.726+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6074034650 con 0x7f60640039b0 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.726+0000 7f607d379640 1 -- 172.21.15.82:0/3790076696 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074034a40 con 0x7f60640039b0 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.727+0000 7f60805df640 1 -- 172.21.15.82:0/3790076696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60640039b0 msgr2=0x7f6064003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.727+0000 7f60805df640 1 --2- 172.21.15.82:0/3790076696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60640039b0 0x7f6064003dd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f607400a890 tx=0x7f6064005830 comp rx=0 tx=0).stop 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.727+0000 7f607eb7c640 1 -- 172.21.15.82:0/3790076696 reap_dead start 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.727+0000 7f60805df640 1 -- 172.21.15.82:0/3790076696 shutdown_connections 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.727+0000 7f60805df640 1 -- 172.21.15.82:0/3790076696 >> 172.21.15.82:0/3790076696 conn(0x7f607809d0d0 msgr2=0x7f60780d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.727+0000 7f60805df640 1 -- 172.21.15.82:0/3790076696 shutdown_connections 2024-03-14T12:15:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.727+0000 7f60805df640 1 -- 172.21.15.82:0/3790076696 wait complete. 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.728+0000 7f60805df640 1 Processor -- start 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.728+0000 7f60805df640 1 -- start start 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60640039b0 0x7f60781f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f607814e140 0x7f60780b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60780cf000 0x7f60780cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780cb4a0 con 0x7f60780cf000 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780cb320 con 0x7f60640039b0 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780cb620 con 0x7f607814e140 2024-03-14T12:15:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60780cf000 0x7f60780cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60780cf000 0x7f60780cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57566/0 (socket says 172.21.15.82:57566) 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 -- 172.21.15.82:0/3816278078 learned_addr learned my addr 172.21.15.82:0/3816278078 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607db7a640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f607814e140 0x7f60780b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607e37b640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60640039b0 0x7f60781f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 -- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f607814e140 msgr2=0x7f60780b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f607814e140 0x7f60780b9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 -- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60640039b0 msgr2=0x7f60781f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60640039b0 0x7f60781f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 -- 172.21.15.82:0/3816278078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60781f4640 con 0x7f60780cf000 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607e37b640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60640039b0 0x7f60781f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f607eb7c640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60780cf000 0x7f60780cf480 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f60780cff30 tx=0x7f6070011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f606effd640 1 -- 172.21.15.82:0/3816278078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070004240 con 0x7f60780cf000 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f606effd640 1 -- 172.21.15.82:0/3816278078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607001b070 con 0x7f60780cf000 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60781f4920 con 0x7f60780cf000 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f606effd640 1 -- 172.21.15.82:0/3816278078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000f770 con 0x7f60780cf000 2024-03-14T12:15:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.729+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60781f5230 con 0x7f60780cf000 2024-03-14T12:15:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.730+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60780cf9c0 con 0x7f60780cf000 2024-03-14T12:15:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.731+0000 7f606effd640 1 -- 172.21.15.82:0/3816278078 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6070012040 con 0x7f60780cf000 2024-03-14T12:15:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.731+0000 7f606effd640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f605c06bc00 0x7f605c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.731+0000 7f606effd640 1 -- 172.21.15.82:0/3816278078 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f607008a500 con 0x7f60780cf000 2024-03-14T12:15:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.731+0000 7f607e37b640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f605c06bc00 0x7f605c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.732+0000 7f607e37b640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f605c06bc00 0x7f605c06e0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6074034000 tx=0x7f6074009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.736+0000 7f606effd640 1 -- 172.21.15.82:0/3816278078 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f60700592f0 con 0x7f60780cf000 2024-03-14T12:15:54.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.909+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f607804f050 con 0x7f60780cf000 2024-03-14T12:15:54.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.912+0000 7f606effd640 1 -- 172.21.15.82:0/3816278078 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6070058bf0 con 0x7f60780cf000 2024-03-14T12:15:54.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f605c06bc00 msgr2=0x7f605c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f605c06bc00 0x7f605c06e0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6074034000 tx=0x7f6074009030 comp rx=0 tx=0).stop 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60780cf000 msgr2=0x7f60780cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60780cf000 0x7f60780cf480 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f60780cff30 tx=0x7f6070011cf0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 shutdown_connections 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60780cf000 0x7f60780cf480 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f605c06bc00 0x7f605c06e0c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f607814e140 0x7f60780b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 --2- 172.21.15.82:0/3816278078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60640039b0 0x7f60781f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 >> 172.21.15.82:0/3816278078 conn(0x7f607809d0d0 msgr2=0x7f60780cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 shutdown_connections 2024-03-14T12:15:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:54.914+0000 7f60805df640 1 -- 172.21.15.82:0/3816278078 wait complete. 2024-03-14T12:15:54.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io' 2024-03-14T12:15:55.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.025+0000 7ff06c774640 1 Processor -- start 2024-03-14T12:15:55.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.025+0000 7ff06c774640 1 -- start start 2024-03-14T12:15:55.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff06c774640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff0640cdd60 0x7ff0640ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff06c774640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640ce8b0 con 0x7ff064144ac0 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff06c774640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640ceb90 con 0x7ff06415f7a0 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff06c774640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640cf150 con 0x7ff0640cdd60 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff069d0f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff0640cdd60 0x7ff0640ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff06ad11640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff064144ac0 0x7ff064144ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46950/0 (socket says 172.21.15.82:46950) 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff06ad11640 1 -- 172.21.15.82:0/927474913 learned_addr learned my addr 172.21.15.82:0/927474913 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff069d0f640 1 -- 172.21.15.82:0/927474913 >> v1:172.21.15.164:6789/0 conn(0x7ff06415f7a0 legacy=0x7ff06415fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff069d0f640 1 -- 172.21.15.82:0/927474913 >> v1:172.21.15.82:6789/0 conn(0x7ff064144ac0 legacy=0x7ff064144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff069d0f640 1 -- 172.21.15.82:0/927474913 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640cee70 con 0x7ff0640cdd60 2024-03-14T12:15:55.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.026+0000 7ff06950e640 1 -- 172.21.15.82:0/927474913 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822786911 0 0) 0x7ff0640ce8b0 con 0x7ff064144ac0 2024-03-14T12:15:55.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.027+0000 7ff069d0f640 1 --2- 172.21.15.82:0/927474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0640cdd60 0x7ff0640ce210 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0540097a0 tx=0x7ff05402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d81d29d164dea5c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:55.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.027+0000 7ff06950e640 1 -- 172.21.15.82:0/927474913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0540315a0 con 0x7ff0640cdd60 2024-03-14T12:15:55.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.027+0000 7ff06950e640 1 -- 172.21.15.82:0/927474913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054037070 con 0x7ff0640cdd60 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.027+0000 7ff06950e640 1 -- 172.21.15.82:0/927474913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054031960 con 0x7ff0640cdd60 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.028+0000 7ff06c774640 1 -- 172.21.15.82:0/927474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0640cdd60 msgr2=0x7ff0640ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.028+0000 7ff06c774640 1 --2- 172.21.15.82:0/927474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0640cdd60 0x7ff0640ce210 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0540097a0 tx=0x7ff05402b840 comp rx=0 tx=0).stop 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.028+0000 7ff06c774640 1 -- 172.21.15.82:0/927474913 shutdown_connections 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.028+0000 7ff06c774640 1 --2- 172.21.15.82:0/927474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0640cdd60 0x7ff0640ce210 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.028+0000 7ff06c774640 1 -- 172.21.15.82:0/927474913 >> 172.21.15.82:0/927474913 conn(0x7ff0640b1c70 msgr2=0x7ff0640cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.028+0000 7ff06c774640 1 -- 172.21.15.82:0/927474913 shutdown_connections 2024-03-14T12:15:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.028+0000 7ff06c774640 1 -- 172.21.15.82:0/927474913 wait complete. 2024-03-14T12:15:55.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.029+0000 7ff06c774640 1 Processor -- start 2024-03-14T12:15:55.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.029+0000 7ff06c774640 1 -- start start 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.029+0000 7ff06c774640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0640cdd60 0x7ff064157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.029+0000 7ff06c774640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff064144ac0 0x7ff06413a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06c774640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06415f7a0 0x7ff0641e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06c774640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640cc670 con 0x7ff06415f7a0 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06c774640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640cc4f0 con 0x7ff064144ac0 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06c774640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff064070630 con 0x7ff0640cdd60 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06415f7a0 0x7ff0641e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06415f7a0 0x7ff0641e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57586/0 (socket says 172.21.15.82:57586) 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 -- 172.21.15.82:0/4099808686 learned_addr learned my addr 172.21.15.82:0/4099808686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 -- 172.21.15.82:0/4099808686 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0640cdd60 msgr2=0x7ff064157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0640cdd60 0x7ff064157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 -- 172.21.15.82:0/4099808686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff064144ac0 msgr2=0x7ff06413a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff064144ac0 0x7ff06413a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 -- 172.21.15.82:0/4099808686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0641f4c90 con 0x7ff06415f7a0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06ad11640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06415f7a0 0x7ff0641e5a10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c700 tx=0x7ff05c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff05affd640 1 -- 172.21.15.82:0/4099808686 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c018070 con 0x7ff06415f7a0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff05affd640 1 -- 172.21.15.82:0/4099808686 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c00ce10 con 0x7ff06415f7a0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff05affd640 1 -- 172.21.15.82:0/4099808686 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00a860 con 0x7ff06415f7a0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0641f4f70 con 0x7ff06415f7a0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.030+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0641f5530 con 0x7ff06415f7a0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.031+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06404f050 con 0x7ff06415f7a0 2024-03-14T12:15:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.032+0000 7ff05affd640 1 -- 172.21.15.82:0/4099808686 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff05c00aa00 con 0x7ff06415f7a0 2024-03-14T12:15:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.032+0000 7ff05affd640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff04806bbd0 0x7ff04806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.032+0000 7ff05affd640 1 -- 172.21.15.82:0/4099808686 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff05c089d80 con 0x7ff06415f7a0 2024-03-14T12:15:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.032+0000 7ff06a510640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff04806bbd0 0x7ff04806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.032+0000 7ff06a510640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff04806bbd0 0x7ff04806e090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff064144a90 tx=0x7ff060006ba0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:55.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.036+0000 7ff05affd640 1 -- 172.21.15.82:0/4099808686 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff05c058b70 con 0x7ff06415f7a0 2024-03-14T12:15:55.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.271+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io"]} v 0) v1 -- 0x7ff0640ce3b0 con 0x7ff06415f7a0 2024-03-14T12:15:55.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.610+0000 7ff05affd640 1 -- 172.21.15.82:0/4099808686 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io"]}]=0 v119) v1 ==== 167+0+0 (secure 0 0 0) 0x7ff05c058470 con 0x7ff06415f7a0 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.612+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff04806bbd0 msgr2=0x7ff04806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.612+0000 7ff06c774640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff04806bbd0 0x7ff04806e090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff064144a90 tx=0x7ff060006ba0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.612+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06415f7a0 msgr2=0x7ff0641e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.612+0000 7ff06c774640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06415f7a0 0x7ff0641e5a10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c700 tx=0x7ff05c00cbc0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 shutdown_connections 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06415f7a0 0x7ff0641e5a10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff064144ac0 0x7ff06413a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff04806bbd0 0x7ff04806e090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 --2- 172.21.15.82:0/4099808686 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0640cdd60 0x7ff064157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 >> 172.21.15.82:0/4099808686 conn(0x7ff0640b1c70 msgr2=0x7ff06406d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:55.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 shutdown_connections 2024-03-14T12:15:55.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.613+0000 7ff06c774640 1 -- 172.21.15.82:0/4099808686 wait complete. 2024-03-14T12:15:55.623 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients performing IO. ... ok 2024-03-14T12:15:55.624 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters_on_client_connect (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled metrics are generated during client connect 2024-03-14T12:15:55.624 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters_on_client_connect (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-14T12:15:55.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:15:55.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.726+0000 7efe452c0640 1 Processor -- start 2024-03-14T12:15:55.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.726+0000 7efe452c0640 1 -- start start 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe452c0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 0x7efe400722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe452c0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe4006b800 0x7efe4006e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe452c0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe400d1590 con 0x7efe4006ad10 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe452c0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4006bc90 con 0x7efe4006b800 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe452c0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400717e0 con 0x7efe40145060 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7efe4006ad10 0x7efe4006b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46966/0 (socket says 172.21.15.82:46966) 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3effd640 1 -- 172.21.15.82:0/204253489 learned_addr learned my addr 172.21.15.82:0/204253489 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182506513 0 0) 0x7efe400d1590 con 0x7efe4006ad10 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe2c003520 con 0x7efe4006ad10 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3f7fe640 1 --2- 172.21.15.82:0/204253489 >> v2:172.21.15.164:3300/0 conn(0x7efe4006b800 0x7efe4006e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe37fff640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 0x7efe400722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190196482 0 0) 0x7efe2c003520 con 0x7efe4006ad10 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 msgr2=0x7efe400722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 0x7efe400722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 >> v2:172.21.15.164:3300/0 conn(0x7efe4006b800 msgr2=0x7efe4006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 --2- 172.21.15.82:0/204253489 >> v2:172.21.15.164:3300/0 conn(0x7efe4006b800 0x7efe4006e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe40071ac0 con 0x7efe4006ad10 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7efe30001670 con 0x7efe4006ad10 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 >> v1:172.21.15.82:6789/0 conn(0x7efe4006ad10 legacy=0x7efe4006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe2c0039b0 0x7efe2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c004800 0x7efe2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 0x7efe2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400717e0 con 0x7efe2c0039b0 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00b9f0 con 0x7efe2c004800 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.727+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00bbf0 con 0x7efe40145060 2024-03-14T12:15:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe3effd640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe2c0039b0 0x7efe2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe37fff640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c004800 0x7efe2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe3f7fe640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 0x7efe2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe3effd640 1 -- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 msgr2=0x7efe2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe3effd640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe40145060 0x7efe2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe3effd640 1 -- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c004800 msgr2=0x7efe2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe3effd640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c004800 0x7efe2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.728+0000 7efe3effd640 1 -- 172.21.15.82:0/204253489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c003520 con 0x7efe2c0039b0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe3effd640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe2c0039b0 0x7efe2c003dd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efe30001eb0 tx=0x7efe2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e3ca390c550247 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30007d70 con 0x7efe2c0039b0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe30030070 con 0x7efe2c0039b0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe3e7fc640 1 -- 172.21.15.82:0/204253489 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30005d30 con 0x7efe2c0039b0 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe452c0640 1 -- 172.21.15.82:0/204253489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe2c0039b0 msgr2=0x7efe2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe452c0640 1 --2- 172.21.15.82:0/204253489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe2c0039b0 0x7efe2c003dd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efe30001eb0 tx=0x7efe2c005830 comp rx=0 tx=0).stop 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe3f7fe640 1 -- 172.21.15.82:0/204253489 reap_dead start 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe452c0640 1 -- 172.21.15.82:0/204253489 shutdown_connections 2024-03-14T12:15:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe452c0640 1 -- 172.21.15.82:0/204253489 >> 172.21.15.82:0/204253489 conn(0x7efe400d09d0 msgr2=0x7efe400d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe452c0640 1 -- 172.21.15.82:0/204253489 shutdown_connections 2024-03-14T12:15:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.729+0000 7efe452c0640 1 -- 172.21.15.82:0/204253489 wait complete. 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.730+0000 7efe452c0640 1 Processor -- start 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.730+0000 7efe452c0640 1 -- start start 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe452c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c0039b0 0x7efe4014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe452c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4015b060 0x7efe40138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe452c0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4013d1b0 0x7efe4013d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe452c0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400722c0 con 0x7efe4015b060 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe452c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400d1870 con 0x7efe2c0039b0 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe452c0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe40072440 con 0x7efe4013d1b0 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4015b060 0x7efe40138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4015b060 0x7efe40138c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57604/0 (socket says 172.21.15.82:57604) 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 -- 172.21.15.82:0/3456803045 learned_addr learned my addr 172.21.15.82:0/3456803045 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe3effd640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c0039b0 0x7efe4014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe3f7fe640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4013d1b0 0x7efe4013d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 -- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4013d1b0 msgr2=0x7efe4013d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4013d1b0 0x7efe4013d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 -- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c0039b0 msgr2=0x7efe4014e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c0039b0 0x7efe4014e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.731+0000 7efe37fff640 1 -- 172.21.15.82:0/3456803045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4013f2f0 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.732+0000 7efe37fff640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4015b060 0x7efe40138c50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b7f0 tx=0x7efe2800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.732+0000 7efe377fe640 1 -- 172.21.15.82:0/3456803045 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28018070 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.732+0000 7efe377fe640 1 -- 172.21.15.82:0/3456803045 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28010830 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.732+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4013f590 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.732+0000 7efe377fe640 1 -- 172.21.15.82:0/3456803045 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28010c00 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.732+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4013fb50 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.733+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4004f050 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.733+0000 7efe377fe640 1 -- 172.21.15.82:0/3456803045 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7efe2801b070 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.734+0000 7efe377fe640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe2006bc00 0x7efe2006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.734+0000 7efe377fe640 1 -- 172.21.15.82:0/3456803045 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efe2808b0d0 con 0x7efe4015b060 2024-03-14T12:15:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.734+0000 7efe3effd640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe2006bc00 0x7efe2006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.734+0000 7efe3effd640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe2006bc00 0x7efe2006e0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efe30034000 tx=0x7efe30007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.738+0000 7efe377fe640 1 -- 172.21.15.82:0/3456803045 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7efe28059ec0 con 0x7efe4015b060 2024-03-14T12:15:55.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.925+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe4006b5b0 con 0x7efe4015b060 2024-03-14T12:15:55.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.926+0000 7efe377fe640 1 -- 172.21.15.82:0/3456803045 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13006 (secure 0 0 0) 0x7efe280597c0 con 0x7efe4015b060 2024-03-14T12:15:55.925 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:55.925 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":37,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:15:54.601739+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/682556190":"2024-03-15T12:15:51.594037+0000","172.21.15.82:6836/682556190":"2024-03-15T12:15:51.594037+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:15:55.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe2006bc00 msgr2=0x7efe2006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe2006bc00 0x7efe2006e0c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efe30034000 tx=0x7efe30007000 comp rx=0 tx=0).stop 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4015b060 msgr2=0x7efe40138c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4015b060 0x7efe40138c50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b7f0 tx=0x7efe2800bcb0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 shutdown_connections 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe2006bc00 0x7efe2006e0c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4013d1b0 0x7efe4013d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4015b060 0x7efe40138c50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 --2- 172.21.15.82:0/3456803045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe2c0039b0 0x7efe4014e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 >> 172.21.15.82:0/3456803045 conn(0x7efe400d09d0 msgr2=0x7efe400ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 shutdown_connections 2024-03-14T12:15:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:55.928+0000 7efe452c0640 1 -- 172.21.15.82:0/3456803045 wait complete. 2024-03-14T12:15:55.938 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:15:55.939 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:15:56.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.041+0000 7fc80611b640 1 Processor -- start 2024-03-14T12:15:56.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.042+0000 7fc80611b640 1 -- start start 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.042+0000 7fc80611b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc8000cf3f0 0x7fc8000cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.042+0000 7fc80611b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8000b41d0 0x7fc8000d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.042+0000 7fc80611b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8000b3450 con 0x7fc80015e780 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.042+0000 7fc80611b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8000cf7d0 con 0x7fc8000b41d0 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.042+0000 7fc80611b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80015f4b0 con 0x7fc8000cf3f0 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7f6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8000b41d0 0x7fc8000d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7fffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc80015e780 0x7fc80015eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46972/0 (socket says 172.21.15.82:46972) 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7fffff640 1 -- 172.21.15.82:0/3654649268 learned_addr learned my addr 172.21.15.82:0/3654649268 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7ff7fe640 1 --2- 172.21.15.82:0/3654649268 >> v2:172.21.15.164:3300/0 conn(0x7fc8000cf3f0 0x7fc8000cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7f6ffd640 1 -- 172.21.15.82:0/3654649268 >> v2:172.21.15.164:3300/0 conn(0x7fc8000cf3f0 msgr2=0x7fc8000cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7f6ffd640 1 --2- 172.21.15.82:0/3654649268 >> v2:172.21.15.164:3300/0 conn(0x7fc8000cf3f0 0x7fc8000cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7f6ffd640 1 -- 172.21.15.82:0/3654649268 >> v1:172.21.15.82:6789/0 conn(0x7fc80015e780 legacy=0x7fc80015eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7f6ffd640 1 -- 172.21.15.82:0/3654649268 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000b36f0 con 0x7fc8000b41d0 2024-03-14T12:15:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.043+0000 7fc7feffd640 1 -- 172.21.15.82:0/3654649268 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699464939 0 0) 0x7fc8000b3450 con 0x7fc80015e780 2024-03-14T12:15:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc7f6ffd640 1 --2- 172.21.15.82:0/3654649268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8000b41d0 0x7fc8000d0100 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc80006bcb0 tx=0x7fc7e802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7542ddbb1ccdd1e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc7feffd640 1 -- 172.21.15.82:0/3654649268 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80315a0 con 0x7fc8000b41d0 2024-03-14T12:15:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc7feffd640 1 -- 172.21.15.82:0/3654649268 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e8037070 con 0x7fc8000b41d0 2024-03-14T12:15:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc7feffd640 1 -- 172.21.15.82:0/3654649268 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8031960 con 0x7fc8000b41d0 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 -- 172.21.15.82:0/3654649268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8000b41d0 msgr2=0x7fc8000d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 --2- 172.21.15.82:0/3654649268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8000b41d0 0x7fc8000d0100 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc80006bcb0 tx=0x7fc7e802b840 comp rx=0 tx=0).stop 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 -- 172.21.15.82:0/3654649268 shutdown_connections 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 --2- 172.21.15.82:0/3654649268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8000b41d0 0x7fc8000d0100 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 --2- 172.21.15.82:0/3654649268 >> v2:172.21.15.164:3300/0 conn(0x7fc8000cf3f0 0x7fc8000cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 -- 172.21.15.82:0/3654649268 >> 172.21.15.82:0/3654649268 conn(0x7fc80006aef0 msgr2=0x7fc80006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 -- 172.21.15.82:0/3654649268 shutdown_connections 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.044+0000 7fc80611b640 1 -- 172.21.15.82:0/3654649268 wait complete. 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.045+0000 7fc80611b640 1 Processor -- start 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc80611b640 1 -- start start 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc80611b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8000b41d0 0x7fc8000b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc80611b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8000cf3f0 0x7fc80013c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc80611b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc80015e780 0x7fc80015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc80611b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80006d890 con 0x7fc80015e780 2024-03-14T12:15:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc80611b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000b3cb0 con 0x7fc8000b41d0 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc80611b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80006da10 con 0x7fc8000cf3f0 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc7fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc80015e780 0x7fc80015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc7fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc80015e780 0x7fc80015d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57616/0 (socket says 172.21.15.82:57616) 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.046+0000 7fc7fffff640 1 -- 172.21.15.82:0/496622554 learned_addr learned my addr 172.21.15.82:0/496622554 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7ff7fe640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8000b41d0 0x7fc8000b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7f6ffd640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8000cf3f0 0x7fc80013c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fffff640 1 -- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8000cf3f0 msgr2=0x7fc80013c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fffff640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8000cf3f0 0x7fc80013c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fffff640 1 -- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8000b41d0 msgr2=0x7fc8000b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fffff640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8000b41d0 0x7fc8000b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fffff640 1 -- 172.21.15.82:0/496622554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800149020 con 0x7fc80015e780 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7ff7fe640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8000b41d0 0x7fc8000b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fffff640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc80015e780 0x7fc80015d3d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800ed30 tx=0x7fc7f800c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fcff9640 1 -- 172.21.15.82:0/496622554 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8015070 con 0x7fc80015e780 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fcff9640 1 -- 172.21.15.82:0/496622554 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8011670 con 0x7fc80015e780 2024-03-14T12:15:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc7fcff9640 1 -- 172.21.15.82:0/496622554 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8011aa0 con 0x7fc80015e780 2024-03-14T12:15:56.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc800149300 con 0x7fc80015e780 2024-03-14T12:15:56.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.047+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8001498c0 con 0x7fc80015e780 2024-03-14T12:15:56.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.048+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80004f050 con 0x7fc80015e780 2024-03-14T12:15:56.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.048+0000 7fc7fcff9640 1 -- 172.21.15.82:0/496622554 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc7f8018070 con 0x7fc80015e780 2024-03-14T12:15:56.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.049+0000 7fc7fcff9640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc7e006bbd0 0x7fc7e006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.049+0000 7fc7ff7fe640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc7e006bbd0 0x7fc7e006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.049+0000 7fc7fcff9640 1 -- 172.21.15.82:0/496622554 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc7f808d390 con 0x7fc80015e780 2024-03-14T12:15:56.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.049+0000 7fc7ff7fe640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc7e006bbd0 0x7fc7e006e090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006fd0 tx=0x7fc7f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.053+0000 7fc7fcff9640 1 -- 172.21.15.82:0/496622554 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc7f805c240 con 0x7fc80015e780 2024-03-14T12:15:56.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.224+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc80006d320 con 0x7fc80015e780 2024-03-14T12:15:56.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.224+0000 7fc7fcff9640 1 -- 172.21.15.82:0/496622554 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7f805bb40 con 0x7fc80015e780 2024-03-14T12:15:56.224 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc7e006bbd0 msgr2=0x7fc7e006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc7e006bbd0 0x7fc7e006e090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006fd0 tx=0x7fc7f0008040 comp rx=0 tx=0).stop 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc80015e780 msgr2=0x7fc80015d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc80015e780 0x7fc80015d3d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800ed30 tx=0x7fc7f800c640 comp rx=0 tx=0).stop 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 shutdown_connections 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc80015e780 0x7fc80015d3d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc7e006bbd0 0x7fc7e006e090 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8000cf3f0 0x7fc80013c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 --2- 172.21.15.82:0/496622554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8000b41d0 0x7fc8000b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 >> 172.21.15.82:0/496622554 conn(0x7fc80006aef0 msgr2=0x7fc8000ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:56.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 shutdown_connections 2024-03-14T12:15:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.226+0000 7fc80611b640 1 -- 172.21.15.82:0/496622554 wait complete. 2024-03-14T12:15:56.237 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_connect' 2024-03-14T12:15:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.338+0000 7f72be019640 1 Processor -- start 2024-03-14T12:15:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72be019640 1 -- start start 2024-03-14T12:15:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72be019640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72b80cf370 0x7f72b80d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72be019640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b80cf8c0 con 0x7f72b80d0760 2024-03-14T12:15:56.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72be019640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b80cfba0 con 0x7f72b80b7720 2024-03-14T12:15:56.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72be019640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b80b6c40 con 0x7f72b80cf370 2024-03-14T12:15:56.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72b6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72b80cf370 0x7f72b80d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72b77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f72b80d0760 0x7f72b80d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46982/0 (socket says 172.21.15.82:46982) 2024-03-14T12:15:56.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.339+0000 7f72b77fe640 1 -- 172.21.15.82:0/2333151295 learned_addr learned my addr 172.21.15.82:0/2333151295 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:56.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.340+0000 7f72b67fc640 1 -- 172.21.15.82:0/2333151295 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970138106 0 0) 0x7f72b80cf8c0 con 0x7f72b80d0760 2024-03-14T12:15:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.340+0000 7f72b67fc640 1 -- 172.21.15.82:0/2333151295 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a0003520 con 0x7f72b80d0760 2024-03-14T12:15:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.340+0000 7f72b6ffd640 1 -- 172.21.15.82:0/2333151295 >> v1:172.21.15.164:6789/0 conn(0x7f72b80b7720 legacy=0x7f72b80b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.340+0000 7f72b6ffd640 1 -- 172.21.15.82:0/2333151295 >> v1:172.21.15.82:6789/0 conn(0x7f72b80d0760 legacy=0x7f72b80d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.340+0000 7f72b6ffd640 1 -- 172.21.15.82:0/2333151295 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80b6f20 con 0x7f72b80cf370 2024-03-14T12:15:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.340+0000 7f72b6ffd640 1 --2- 172.21.15.82:0/2333151295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80cf370 0x7f72b80d4660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0097d0 tx=0x7f72ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265c04f42d22f52d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72b67fc640 1 -- 172.21.15.82:0/2333151295 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac034070 con 0x7f72b80cf370 2024-03-14T12:15:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72b67fc640 1 -- 172.21.15.82:0/2333151295 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac030610 con 0x7f72b80cf370 2024-03-14T12:15:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72b67fc640 1 -- 172.21.15.82:0/2333151295 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0309e0 con 0x7f72b80cf370 2024-03-14T12:15:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72be019640 1 -- 172.21.15.82:0/2333151295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80cf370 msgr2=0x7f72b80d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72be019640 1 --2- 172.21.15.82:0/2333151295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80cf370 0x7f72b80d4660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0097d0 tx=0x7f72ac02b960 comp rx=0 tx=0).stop 2024-03-14T12:15:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72be019640 1 -- 172.21.15.82:0/2333151295 shutdown_connections 2024-03-14T12:15:56.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72be019640 1 --2- 172.21.15.82:0/2333151295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80cf370 0x7f72b80d4660 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72be019640 1 -- 172.21.15.82:0/2333151295 >> 172.21.15.82:0/2333151295 conn(0x7f72b809d030 msgr2=0x7f72b80d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:56.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72be019640 1 -- 172.21.15.82:0/2333151295 shutdown_connections 2024-03-14T12:15:56.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.341+0000 7f72be019640 1 -- 172.21.15.82:0/2333151295 wait complete. 2024-03-14T12:15:56.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.342+0000 7f72be019640 1 Processor -- start 2024-03-14T12:15:56.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72be019640 1 -- start start 2024-03-14T12:15:56.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72be019640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b80b7720 0x7f72b81e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72be019640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b80cf370 0x7f72b815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72be019640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80d0760 0x7f72b81f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72be019640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b81617a0 con 0x7f72b80d0760 2024-03-14T12:15:56.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72be019640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80d4ba0 con 0x7f72b80b7720 2024-03-14T12:15:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72be019640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8161920 con 0x7f72b80cf370 2024-03-14T12:15:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80d0760 0x7f72b81f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80d0760 0x7f72b81f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57642/0 (socket says 172.21.15.82:57642) 2024-03-14T12:15:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.343+0000 7f72b7fff640 1 -- 172.21.15.82:0/3098891367 learned_addr learned my addr 172.21.15.82:0/3098891367 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72b7fff640 1 -- 172.21.15.82:0/3098891367 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b80cf370 msgr2=0x7f72b815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72b7fff640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b80cf370 0x7f72b815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72b7fff640 1 -- 172.21.15.82:0/3098891367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b80b7720 msgr2=0x7f72b81e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72b7fff640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b80b7720 0x7f72b81e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72b7fff640 1 -- 172.21.15.82:0/3098891367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80c8870 con 0x7f72b80d0760 2024-03-14T12:15:56.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72b7fff640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80d0760 0x7f72b81f31f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c780 tx=0x7f72b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f729ffff640 1 -- 172.21.15.82:0/3098891367 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0004b60 con 0x7f72b80d0760 2024-03-14T12:15:56.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f729ffff640 1 -- 172.21.15.82:0/3098891367 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0004d00 con 0x7f72b80d0760 2024-03-14T12:15:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b81e7300 con 0x7f72b80d0760 2024-03-14T12:15:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f729ffff640 1 -- 172.21.15.82:0/3098891367 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b000aa90 con 0x7f72b80d0760 2024-03-14T12:15:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.344+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b81e75e0 con 0x7f72b80d0760 2024-03-14T12:15:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.345+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b804f050 con 0x7f72b80d0760 2024-03-14T12:15:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.345+0000 7f729ffff640 1 -- 172.21.15.82:0/3098891367 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f72b000ac30 con 0x7f72b80d0760 2024-03-14T12:15:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.346+0000 7f729ffff640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f72a406bbd0 0x7f72a406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.346+0000 7f729ffff640 1 -- 172.21.15.82:0/3098891367 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f72b008a9b0 con 0x7f72b80d0760 2024-03-14T12:15:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.346+0000 7f72b77fe640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f72a406bbd0 0x7f72a406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.346+0000 7f72b77fe640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f72a406bbd0 0x7f72a406e090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f72a8000f60 tx=0x7f72a80078f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.350+0000 7f729ffff640 1 -- 172.21.15.82:0/3098891367 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f72b0020070 con 0x7f72b80d0760 2024-03-14T12:15:56.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.585+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_connect"]} v 0) v1 -- 0x7f72b80d0540 con 0x7f72b80d0760 2024-03-14T12:15:56.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.611+0000 7f729ffff640 1 -- 172.21.15.82:0/3098891367 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_connect"]}]=0 v120) v1 ==== 185+0+0 (secure 0 0 0) 0x7f72b0059830 con 0x7f72b80d0760 2024-03-14T12:15:56.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.613+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f72a406bbd0 msgr2=0x7f72a406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.613+0000 7f72be019640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f72a406bbd0 0x7f72a406e090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f72a8000f60 tx=0x7f72a80078f0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.613+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80d0760 msgr2=0x7f72b81f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.613+0000 7f72be019640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80d0760 0x7f72b81f31f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c780 tx=0x7f72b000cc40 comp rx=0 tx=0).stop 2024-03-14T12:15:56.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.613+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 shutdown_connections 2024-03-14T12:15:56.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.614+0000 7f72be019640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72b80d0760 0x7f72b81f31f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.614+0000 7f72be019640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f72a406bbd0 0x7f72a406e090 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.614+0000 7f72be019640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b80cf370 0x7f72b815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.614+0000 7f72be019640 1 --2- 172.21.15.82:0/3098891367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b80b7720 0x7f72b81e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.614+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 >> 172.21.15.82:0/3098891367 conn(0x7f72b809d030 msgr2=0x7f72b80d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.614+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 shutdown_connections 2024-03-14T12:15:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.614+0000 7f72be019640 1 -- 172.21.15.82:0/3098891367 wait complete. 2024-03-14T12:15:56.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-14T12:15:56.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.728+0000 7fde28de1640 1 Processor -- start 2024-03-14T12:15:56.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.728+0000 7fde28de1640 1 -- start start 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde28de1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde240d0db0 0x7fde24162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde28de1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde240b2510 0x7fde2415e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde28de1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde2406e8e0 con 0x7fde24147360 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde28de1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2406eb80 con 0x7fde240d0db0 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde28de1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2406f100 con 0x7fde240b2510 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde2259c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde240d0db0 0x7fde24162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde22d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde240b2510 0x7fde2415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde2259c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde240d0db0 0x7fde24162560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:60456/0 (socket says 172.21.15.82:60456) 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.729+0000 7fde2259c640 1 -- 172.21.15.82:0/3938553384 learned_addr learned my addr 172.21.15.82:0/3938553384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.730+0000 7fde2259c640 1 -- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde240b2510 msgr2=0x7fde2415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.730+0000 7fde2259c640 1 --2- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde240b2510 0x7fde2415e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.730+0000 7fde2259c640 1 -- 172.21.15.82:0/3938553384 >> v1:172.21.15.164:6789/0 conn(0x7fde24147360 legacy=0x7fde240b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.730+0000 7fde2259c640 1 -- 172.21.15.82:0/3938553384 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2406ee20 con 0x7fde240d0db0 2024-03-14T12:15:56.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.730+0000 7fde22d9d640 1 --2- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde240b2510 0x7fde2415e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:15:56.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.730+0000 7fde2259c640 1 --2- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240d0db0 0x7fde24162560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde14009870 tx=0x7fde1402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c55346ecace4a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde2159a640 1 -- 172.21.15.82:0/3938553384 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14035020 con 0x7fde240d0db0 2024-03-14T12:15:56.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde2159a640 1 -- 172.21.15.82:0/3938553384 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14031640 con 0x7fde240d0db0 2024-03-14T12:15:56.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde2159a640 1 -- 172.21.15.82:0/3938553384 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1402d040 con 0x7fde240d0db0 2024-03-14T12:15:56.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 -- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240d0db0 msgr2=0x7fde24162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 --2- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240d0db0 0x7fde24162560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde14009870 tx=0x7fde1402b8f0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 -- 172.21.15.82:0/3938553384 shutdown_connections 2024-03-14T12:15:56.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 --2- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde240b2510 0x7fde2415e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 --2- 172.21.15.82:0/3938553384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240d0db0 0x7fde24162560 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 -- 172.21.15.82:0/3938553384 >> 172.21.15.82:0/3938553384 conn(0x7fde240b4fd0 msgr2=0x7fde240b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:56.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 -- 172.21.15.82:0/3938553384 shutdown_connections 2024-03-14T12:15:56.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.731+0000 7fde28de1640 1 -- 172.21.15.82:0/3938553384 wait complete. 2024-03-14T12:15:56.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 Processor -- start 2024-03-14T12:15:56.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 -- start start 2024-03-14T12:15:56.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240b2510 0x7fde24157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde240d0db0 0x7fde2415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde24147360 0x7fde2415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde24162aa0 con 0x7fde240d0db0 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde24162040 con 0x7fde240b2510 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde28de1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2406d330 con 0x7fde24147360 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde21d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde240d0db0 0x7fde2415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde21d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde240d0db0 0x7fde2415c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57652/0 (socket says 172.21.15.82:57652) 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.733+0000 7fde21d9b640 1 -- 172.21.15.82:0/1309907633 learned_addr learned my addr 172.21.15.82:0/1309907633 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde2259c640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240b2510 0x7fde24157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde22d9d640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde24147360 0x7fde2415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde21d9b640 1 -- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde24147360 msgr2=0x7fde2415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde21d9b640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde24147360 0x7fde2415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde21d9b640 1 -- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240b2510 msgr2=0x7fde24157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde21d9b640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240b2510 0x7fde24157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde21d9b640 1 -- 172.21.15.82:0/1309907633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde241f4330 con 0x7fde240d0db0 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde2259c640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240b2510 0x7fde24157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde21d9b640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde240d0db0 0x7fde2415c010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c3c0 tx=0x7fde1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde12ffd640 1 -- 172.21.15.82:0/1309907633 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde180052d0 con 0x7fde240d0db0 2024-03-14T12:15:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde12ffd640 1 -- 172.21.15.82:0/1309907633 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18005490 con 0x7fde240d0db0 2024-03-14T12:15:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde241f4610 con 0x7fde240d0db0 2024-03-14T12:15:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde12ffd640 1 -- 172.21.15.82:0/1309907633 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18007260 con 0x7fde240d0db0 2024-03-14T12:15:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.734+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde241f4bd0 con 0x7fde240d0db0 2024-03-14T12:15:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.735+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde24157830 con 0x7fde240d0db0 2024-03-14T12:15:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.736+0000 7fde12ffd640 1 -- 172.21.15.82:0/1309907633 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fde18007400 con 0x7fde240d0db0 2024-03-14T12:15:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.736+0000 7fde12ffd640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde0406bbd0 0x7fde0406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.736+0000 7fde12ffd640 1 -- 172.21.15.82:0/1309907633 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fde18089c20 con 0x7fde240d0db0 2024-03-14T12:15:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.736+0000 7fde2259c640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde0406bbd0 0x7fde0406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.736+0000 7fde2259c640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde0406bbd0 0x7fde0406e090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fde14036000 tx=0x7fde14008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:56.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.740+0000 7fde12ffd640 1 -- 172.21.15.82:0/1309907633 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fde18058a10 con 0x7fde240d0db0 2024-03-14T12:15:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.911+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fde2415feb0 con 0x7fde240d0db0 2024-03-14T12:15:56.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.915+0000 7fde12ffd640 1 -- 172.21.15.82:0/1309907633 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde18058310 con 0x7fde240d0db0 2024-03-14T12:15:56.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde0406bbd0 msgr2=0x7fde0406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde0406bbd0 0x7fde0406e090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fde14036000 tx=0x7fde14008030 comp rx=0 tx=0).stop 2024-03-14T12:15:56.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde240d0db0 msgr2=0x7fde2415c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde240d0db0 0x7fde2415c010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c3c0 tx=0x7fde1800c880 comp rx=0 tx=0).stop 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 shutdown_connections 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fde0406bbd0 0x7fde0406e090 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde24147360 0x7fde2415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde240d0db0 0x7fde2415c010 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 --2- 172.21.15.82:0/1309907633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde240b2510 0x7fde24157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 >> 172.21.15.82:0/1309907633 conn(0x7fde240b4fd0 msgr2=0x7fde24160350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 shutdown_connections 2024-03-14T12:15:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:56.917+0000 7fde28de1640 1 -- 172.21.15.82:0/1309907633 wait complete. 2024-03-14T12:15:56.927 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:15:56.927 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:15:56.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:15:57.021 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:15:57.021 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:57.021 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:57.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:15:57.037 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:57.037 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:57.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:15:57.054 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:15:57.054 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:57.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:15:57.092 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:15:57.092 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:15:57.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:15:57.111 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-14T12:15:57.111 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:15:57.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.232+0000 7f211a861640 1 Processor -- start 2024-03-14T12:15:57.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.232+0000 7f211a861640 1 -- start start 2024-03-14T12:15:57.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f211a861640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21140cfc40 0x7f21140afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f211a861640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21140cdcf0 0x7f21140b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f211a861640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211406af50 con 0x7f211406bc00 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f211a861640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21140d10b0 con 0x7f21140cfc40 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f211a861640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211406b160 con 0x7f21140cdcf0 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f21137fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21140cfc40 0x7f21140afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f21137fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21140cfc40 0x7f21140afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57664/0 (socket says 172.21.15.82:57664) 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f21137fe640 1 -- 172.21.15.82:0/2577480888 learned_addr learned my addr 172.21.15.82:0/2577480888 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.233+0000 7f2118dfe640 1 --2- 172.21.15.82:0/2577480888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21140cdcf0 0x7f21140b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f21137fe640 1 -- 172.21.15.82:0/2577480888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21140cdcf0 msgr2=0x7f21140b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f21137fe640 1 --2- 172.21.15.82:0/2577480888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21140cdcf0 0x7f21140b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f21137fe640 1 -- 172.21.15.82:0/2577480888 >> v1:172.21.15.164:6789/0 conn(0x7f211406bc00 legacy=0x7f211406bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f21137fe640 1 -- 172.21.15.82:0/2577480888 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211406b3b0 con 0x7f21140cfc40 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f21137fe640 1 --2- 172.21.15.82:0/2577480888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cfc40 0x7f21140afa00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2108009870 tx=0x7f210802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b1febdd30c8b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f2112ffd640 1 -- 172.21.15.82:0/2577480888 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108034070 con 0x7f21140cfc40 2024-03-14T12:15:57.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f2112ffd640 1 -- 172.21.15.82:0/2577480888 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2108030840 con 0x7f21140cfc40 2024-03-14T12:15:57.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.234+0000 7f2112ffd640 1 -- 172.21.15.82:0/2577480888 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108030c10 con 0x7f21140cfc40 2024-03-14T12:15:57.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 -- 172.21.15.82:0/2577480888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cfc40 msgr2=0x7f21140afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 --2- 172.21.15.82:0/2577480888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cfc40 0x7f21140afa00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2108009870 tx=0x7f210802b980 comp rx=0 tx=0).stop 2024-03-14T12:15:57.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 -- 172.21.15.82:0/2577480888 shutdown_connections 2024-03-14T12:15:57.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 --2- 172.21.15.82:0/2577480888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21140cdcf0 0x7f21140b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 --2- 172.21.15.82:0/2577480888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cfc40 0x7f21140afa00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 -- 172.21.15.82:0/2577480888 >> 172.21.15.82:0/2577480888 conn(0x7f2114072310 msgr2=0x7f2114072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 -- 172.21.15.82:0/2577480888 shutdown_connections 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.235+0000 7f211a861640 1 -- 172.21.15.82:0/2577480888 wait complete. 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.236+0000 7f211a861640 1 Processor -- start 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.236+0000 7f211a861640 1 -- start start 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f211a861640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f211406bc00 0x7f21140990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f211a861640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cdcf0 0x7f211414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f211a861640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21140cfc40 0x7f2114156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f211a861640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114066ac0 con 0x7f21140cdcf0 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f211a861640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21140b5be0 con 0x7f21140cfc40 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f211a861640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2114066fe0 con 0x7f211406bc00 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cdcf0 0x7f211414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cdcf0 0x7f211414ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57674/0 (socket says 172.21.15.82:57674) 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 -- 172.21.15.82:0/3306143626 learned_addr learned my addr 172.21.15.82:0/3306143626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f2113fff640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f211406bc00 0x7f21140990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f2118dfe640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21140cfc40 0x7f2114156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 -- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f211406bc00 msgr2=0x7f21140990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f211406bc00 0x7f21140990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 -- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21140cfc40 msgr2=0x7f2114156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21140cfc40 0x7f2114156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.237+0000 7f21137fe640 1 -- 172.21.15.82:0/3306143626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21141532b0 con 0x7f21140cdcf0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.238+0000 7f21137fe640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cdcf0 0x7f211414ff80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2108000c00 tx=0x7f21080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.238+0000 7f2110ff9640 1 -- 172.21.15.82:0/3306143626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108034040 con 0x7f21140cdcf0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.238+0000 7f2110ff9640 1 -- 172.21.15.82:0/3306143626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2108039d90 con 0x7f21140cdcf0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.238+0000 7f2110ff9640 1 -- 172.21.15.82:0/3306143626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108043610 con 0x7f21140cdcf0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.238+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2114150610 con 0x7f21140cdcf0 2024-03-14T12:15:57.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.238+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2114150bd0 con 0x7f21140cdcf0 2024-03-14T12:15:57.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.239+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21141511e0 con 0x7f21140cdcf0 2024-03-14T12:15:57.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.239+0000 7f2110ff9640 1 -- 172.21.15.82:0/3306143626 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f210804a060 con 0x7f21140cdcf0 2024-03-14T12:15:57.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.239+0000 7f2110ff9640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f210006bbd0 0x7f210006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.240+0000 7f2110ff9640 1 -- 172.21.15.82:0/3306143626 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f21080ac660 con 0x7f21140cdcf0 2024-03-14T12:15:57.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.240+0000 7f2113fff640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f210006bbd0 0x7f210006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.240+0000 7f2113fff640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f210006bbd0 0x7f210006e090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2104001780 tx=0x7f2104009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:57.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.244+0000 7f2110ff9640 1 -- 172.21.15.82:0/3306143626 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f210803e070 con 0x7f21140cdcf0 2024-03-14T12:15:57.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.450+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f211404f050 con 0x7f21140cdcf0 2024-03-14T12:15:57.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.451+0000 7f2110ff9640 1 -- 172.21.15.82:0/3306143626 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f210807ae00 con 0x7f21140cdcf0 2024-03-14T12:15:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:57.451 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2478084830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2478084830},{"type":"v1","addr":"172.21.15.82:6835","nonce":2478084830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4834,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1811632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1811632800},{"type":"v1","addr":"172.21.15.164:6834","nonce":1811632800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4176962755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4176962755},{"type":"v1","addr":"172.21.15.164:6835","nonce":4176962755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2624596208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2624596208},{"type":"v1","addr":"172.21.15.82:6837","nonce":2624596208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[]} 2024-03-14T12:15:57.451 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-03-14T12:15:57.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f210006bbd0 msgr2=0x7f210006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f210006bbd0 0x7f210006e090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2104001780 tx=0x7f2104009040 comp rx=0 tx=0).stop 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cdcf0 msgr2=0x7f211414ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cdcf0 0x7f211414ff80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2108000c00 tx=0x7f21080046c0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 shutdown_connections 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21140cfc40 0x7f2114156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140cdcf0 0x7f211414ff80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f210006bbd0 0x7f210006e090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 --2- 172.21.15.82:0/3306143626 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f211406bc00 0x7f21140990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 >> 172.21.15.82:0/3306143626 conn(0x7f2114072310 msgr2=0x7f21140ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 shutdown_connections 2024-03-14T12:15:57.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.453+0000 7f211a861640 1 -- 172.21.15.82:0/3306143626 wait complete. 2024-03-14T12:15:57.464 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T12:15:57.464 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-14T12:15:57.464 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T12:15:57.464 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:15:57.464 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T12:15:57.464 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-14T12:15:57.464 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T12:15:57.465 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:15:57.465 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T12:15:57.465 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-14T12:15:57.465 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T12:15:57.465 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:15:57.465 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T12:15:57.465 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-14T12:15:57.466 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T12:15:57.466 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:15:57.473 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T12:15:57.474 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T12:15:57.474 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T12:15:57.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T12:15:57.476 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T12:15:57.477 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T12:15:57.477 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T12:15:57.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T12:15:57.516 INFO:tasks.ceph.mds.a:Started 2024-03-14T12:15:57.517 DEBUG:teuthology.parallel:result is None 2024-03-14T12:15:57.517 INFO:tasks.ceph.mds.c:Started 2024-03-14T12:15:57.517 DEBUG:teuthology.parallel:result is None 2024-03-14T12:15:57.519 INFO:tasks.ceph.mds.b:Started 2024-03-14T12:15:57.520 DEBUG:teuthology.parallel:result is None 2024-03-14T12:15:57.520 INFO:tasks.ceph.mds.d:Started 2024-03-14T12:15:57.520 DEBUG:teuthology.parallel:result is None 2024-03-14T12:15:57.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-14T12:15:57.592 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-14T12:15:57.592 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:15:57.602 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-03-14T12:15:57.602 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-03-14T12:15:57.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.628+0000 7f920d3f1640 1 Processor -- start 2024-03-14T12:15:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.628+0000 7f920d3f1640 1 -- start start 2024-03-14T12:15:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f920d3f1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92080cf370 0x7f92080d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f920d3f1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92080cf8c0 con 0x7f92080b7720 2024-03-14T12:15:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f920d3f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92080cfba0 con 0x7f92080d0760 2024-03-14T12:15:57.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f920d3f1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92080b6c40 con 0x7f92080cf370 2024-03-14T12:15:57.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f92067fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92080cf370 0x7f92080d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f92067fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92080cf370 0x7f92080d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57714/0 (socket says 172.21.15.82:57714) 2024-03-14T12:15:57.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f92067fc640 1 -- 172.21.15.82:0/579347643 learned_addr learned my addr 172.21.15.82:0/579347643 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:57.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f92067fc640 1 -- 172.21.15.82:0/579347643 >> v1:172.21.15.164:6789/0 conn(0x7f92080d0760 legacy=0x7f92080d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f92067fc640 1 -- 172.21.15.82:0/579347643 >> v1:172.21.15.82:6789/0 conn(0x7f92080b7720 legacy=0x7f92080b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.629+0000 7f92067fc640 1 -- 172.21.15.82:0/579347643 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080b6f20 con 0x7f92080cf370 2024-03-14T12:15:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.630+0000 7f92067fc640 1 --2- 172.21.15.82:0/579347643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080cf370 0x7f92080d4660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0098a0 tx=0x7f91fc02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaadadfdb640c6a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:57.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.633+0000 7f9205ffb640 1 -- 172.21.15.82:0/579347643 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc02f830 con 0x7f92080cf370 2024-03-14T12:15:57.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.633+0000 7f9205ffb640 1 -- 172.21.15.82:0/579347643 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc02f9d0 con 0x7f92080cf370 2024-03-14T12:15:57.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.633+0000 7f9205ffb640 1 -- 172.21.15.82:0/579347643 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc02fda0 con 0x7f92080cf370 2024-03-14T12:15:57.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.633+0000 7f920d3f1640 1 -- 172.21.15.82:0/579347643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080cf370 msgr2=0x7f92080d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.633+0000 7f920d3f1640 1 --2- 172.21.15.82:0/579347643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080cf370 0x7f92080d4660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0098a0 tx=0x7f91fc02ba10 comp rx=0 tx=0).stop 2024-03-14T12:15:57.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.633+0000 7f920d3f1640 1 -- 172.21.15.82:0/579347643 shutdown_connections 2024-03-14T12:15:57.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.634+0000 7f920d3f1640 1 --2- 172.21.15.82:0/579347643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080cf370 0x7f92080d4660 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.634+0000 7f920d3f1640 1 -- 172.21.15.82:0/579347643 >> 172.21.15.82:0/579347643 conn(0x7f920809d030 msgr2=0x7f92080d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.634+0000 7f920d3f1640 1 -- 172.21.15.82:0/579347643 shutdown_connections 2024-03-14T12:15:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.634+0000 7f920d3f1640 1 -- 172.21.15.82:0/579347643 wait complete. 2024-03-14T12:15:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.635+0000 7f920d3f1640 1 Processor -- start 2024-03-14T12:15:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.635+0000 7f920d3f1640 1 -- start start 2024-03-14T12:15:57.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f920d3f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080b7720 0x7f920814a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f920d3f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92080cf370 0x7f920806c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f920d3f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92080d0760 0x7f920806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f920d3f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92081617a0 con 0x7f92080b7720 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f920d3f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080d4ba0 con 0x7f92080d0760 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f9206ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080b7720 0x7f920814a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f920d3f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9208161920 con 0x7f92080cf370 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f9206ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080b7720 0x7f920814a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57730/0 (socket says 172.21.15.82:57730) 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f9206ffd640 1 -- 172.21.15.82:0/4023253874 learned_addr learned my addr 172.21.15.82:0/4023253874 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.636+0000 7f92077fe640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92080d0760 0x7f920806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f92067fc640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92080cf370 0x7f920806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f9206ffd640 1 -- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92080cf370 msgr2=0x7f920806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f9206ffd640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92080cf370 0x7f920806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f9206ffd640 1 -- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92080d0760 msgr2=0x7f920806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f9206ffd640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92080d0760 0x7f920806cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f9206ffd640 1 -- 172.21.15.82:0/4023253874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080ce0e0 con 0x7f92080b7720 2024-03-14T12:15:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f92077fe640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92080d0760 0x7f920806cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f92067fc640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92080cf370 0x7f920806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.637+0000 7f9206ffd640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080b7720 0x7f920814a020 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c3c0 tx=0x7f91f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.640+0000 7f91ef7fe640 1 -- 172.21.15.82:0/4023253874 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80052d0 con 0x7f92080b7720 2024-03-14T12:15:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.640+0000 7f91ef7fe640 1 -- 172.21.15.82:0/4023253874 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f8005490 con 0x7f92080b7720 2024-03-14T12:15:57.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.640+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92080cde00 con 0x7f92080b7720 2024-03-14T12:15:57.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.640+0000 7f91ef7fe640 1 -- 172.21.15.82:0/4023253874 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8007260 con 0x7f92080b7720 2024-03-14T12:15:57.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.640+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92080c8580 con 0x7f92080b7720 2024-03-14T12:15:57.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.641+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920804f050 con 0x7f92080b7720 2024-03-14T12:15:57.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.642+0000 7f91ef7fe640 1 -- 172.21.15.82:0/4023253874 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f91f8007400 con 0x7f92080b7720 2024-03-14T12:15:57.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.642+0000 7f91ef7fe640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f91f406bbd0 0x7f91f406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:57.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.642+0000 7f91ef7fe640 1 -- 172.21.15.82:0/4023253874 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f91f8089e00 con 0x7f92080b7720 2024-03-14T12:15:57.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.642+0000 7f92067fc640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f91f406bbd0 0x7f91f406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:57.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.643+0000 7f92067fc640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f91f406bbd0 0x7f91f406e090 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f91fc036000 tx=0x7f91fc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.646+0000 7f91ef7fe640 1 -- 172.21.15.82:0/4023253874 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f91f8058a40 con 0x7f92080b7720 2024-03-14T12:15:57.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:57.829+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f92080d0540 con 0x7f92080b7720 2024-03-14T12:15:58.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.639+0000 7f91ef7fe640 1 -- 172.21.15.82:0/4023253874 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f91f8058340 con 0x7f92080b7720 2024-03-14T12:15:58.639 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-03-14T12:15:58.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f91f406bbd0 msgr2=0x7f91f406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:58.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f91f406bbd0 0x7f91f406e090 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f91fc036000 tx=0x7f91fc008030 comp rx=0 tx=0).stop 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080b7720 msgr2=0x7f920814a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080b7720 0x7f920814a020 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c3c0 tx=0x7f91f800c880 comp rx=0 tx=0).stop 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 shutdown_connections 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92080d0760 0x7f920806cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f91f406bbd0 0x7f91f406e090 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92080cf370 0x7f920806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 --2- 172.21.15.82:0/4023253874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92080b7720 0x7f920814a020 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c3c0 tx=0x7f91f800c880 comp rx=0 tx=0).stop 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 >> 172.21.15.82:0/4023253874 conn(0x7f920809d030 msgr2=0x7f92080d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 shutdown_connections 2024-03-14T12:15:58.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.642+0000 7f920d3f1640 1 -- 172.21.15.82:0/4023253874 wait complete. 2024-03-14T12:15:58.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-14T12:15:58.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.761+0000 7fd40e4f8640 1 Processor -- start 2024-03-14T12:15:58.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.761+0000 7fd40e4f8640 1 -- start start 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd40e4f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 0x7fd40815b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd40e4f8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4080b3050 con 0x7fd408071740 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd40e4f8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40815ebe0 con 0x7fd40815e670 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd40e4f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40815eec0 con 0x7fd408143f60 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd407fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd408071740 0x7fd408071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47012/0 (socket says 172.21.15.82:47012) 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd407fff640 1 -- 172.21.15.82:0/3488102787 learned_addr learned my addr 172.21.15.82:0/3488102787 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578447300 0 0) 0x7fd4080b3050 con 0x7fd408071740 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd40ca95640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 0x7fd40815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f0005730 con 0x7fd408071740 2024-03-14T12:15:58.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1000969541 0 0) 0x7fd3f0005730 con 0x7fd408071740 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 msgr2=0x7fd40815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd406ffd640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 0x7fd40815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.762+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 >> v1:172.21.15.164:6789/0 conn(0x7fd40815e670 legacy=0x7fd4080b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40815f1a0 con 0x7fd408071740 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fd3f8001670 con 0x7fd408071740 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 >> v1:172.21.15.82:6789/0 conn(0x7fd408071740 legacy=0x7fd408071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f0005bf0 0x7fd3f0006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 0x7fd3f0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3f0006f50 0x7fd3f00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40815eec0 con 0x7fd3f0006f50 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f000dc80 con 0x7fd3f0005bf0 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f000de80 con 0x7fd408143f60 2024-03-14T12:15:58.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd40ca95640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3f0006f50 0x7fd3f00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd407fff640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f0005bf0 0x7fd3f0006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd4077fe640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 0x7fd3f0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd40ca95640 1 -- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 msgr2=0x7fd3f0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd40ca95640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd408143f60 0x7fd3f0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd40ca95640 1 -- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f0005bf0 msgr2=0x7fd3f0006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd40ca95640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f0005bf0 0x7fd3f0006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.763+0000 7fd40ca95640 1 -- 172.21.15.82:0/3488102787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0005730 con 0x7fd3f0006f50 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40ca95640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3f0006f50 0x7fd3f00073d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd400009f90 tx=0x7fd3f0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738cf1688648007a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400039070 con 0x7fd3f0006f50 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40002cbd0 con 0x7fd3f0006f50 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd406ffd640 1 -- 172.21.15.82:0/3488102787 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400009c10 con 0x7fd3f0006f50 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40e4f8640 1 -- 172.21.15.82:0/3488102787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3f0006f50 msgr2=0x7fd3f00073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:58.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40e4f8640 1 --2- 172.21.15.82:0/3488102787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3f0006f50 0x7fd3f00073d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd400009f90 tx=0x7fd3f0007a40 comp rx=0 tx=0).stop 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40ca95640 1 -- 172.21.15.82:0/3488102787 reap_dead start 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40e4f8640 1 -- 172.21.15.82:0/3488102787 shutdown_connections 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40e4f8640 1 -- 172.21.15.82:0/3488102787 >> 172.21.15.82:0/3488102787 conn(0x7fd4080b1c70 msgr2=0x7fd4080cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40e4f8640 1 -- 172.21.15.82:0/3488102787 shutdown_connections 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.764+0000 7fd40e4f8640 1 -- 172.21.15.82:0/3488102787 wait complete. 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.766+0000 7fd40e4f8640 1 Processor -- start 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.766+0000 7fd40e4f8640 1 -- start start 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd40e4f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd40812ff10 0x7fd40815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd40e4f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd40815cd10 0x7fd4080af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd40e4f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4080afaf0 0x7fd4081598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd40e4f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40815b080 con 0x7fd40815cd10 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd40e4f8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd408143ce0 con 0x7fd4080afaf0 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd40e4f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd408144670 con 0x7fd40812ff10 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd40815cd10 0x7fd4080af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd40815cd10 0x7fd4080af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57754/0 (socket says 172.21.15.82:57754) 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 -- 172.21.15.82:0/1426949501 learned_addr learned my addr 172.21.15.82:0/1426949501 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd40ca95640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4080afaf0 0x7fd4081598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd407fff640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd40812ff10 0x7fd40815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 -- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd40812ff10 msgr2=0x7fd40815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd40812ff10 0x7fd40815aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 -- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4080afaf0 msgr2=0x7fd4081598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4080afaf0 0x7fd4081598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.767+0000 7fd4077fe640 1 -- 172.21.15.82:0/1426949501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40813eeb0 con 0x7fd40815cd10 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.768+0000 7fd4077fe640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd40815cd10 0x7fd4080af5b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00f820 tx=0x7fd3fc00fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.768+0000 7fd404ff9640 1 -- 172.21.15.82:0/1426949501 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0050d0 con 0x7fd40815cd10 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.768+0000 7fd404ff9640 1 -- 172.21.15.82:0/1426949501 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc018070 con 0x7fd40815cd10 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.768+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40815d560 con 0x7fd40815cd10 2024-03-14T12:15:58.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.768+0000 7fd404ff9640 1 -- 172.21.15.82:0/1426949501 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00d6d0 con 0x7fd40815cd10 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.768+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40815db20 con 0x7fd40815cd10 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.769+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40804f050 con 0x7fd40815cd10 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.769+0000 7fd404ff9640 1 -- 172.21.15.82:0/1426949501 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd3fc00d870 con 0x7fd40815cd10 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.770+0000 7fd404ff9640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3f406bbd0 0x7fd3f406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.770+0000 7fd407fff640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3f406bbd0 0x7fd3f406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.770+0000 7fd404ff9640 1 -- 172.21.15.82:0/1426949501 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3fc08a450 con 0x7fd40815cd10 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.770+0000 7fd407fff640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3f406bbd0 0x7fd3f406e090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80017e0 tx=0x7fd3f80015d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:58.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:58.774+0000 7fd404ff9640 1 -- 172.21.15.82:0/1426949501 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd3fc059200 con 0x7fd40815cd10 2024-03-14T12:15:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.011+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd4080d07b0 con 0x7fd40815cd10 2024-03-14T12:15:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.012+0000 7fd404ff9640 1 -- 172.21.15.82:0/1426949501 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd3fc058b00 con 0x7fd40815cd10 2024-03-14T12:15:59.011 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:59.011 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:15:59.011 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlAQmRIBAA45IMoIGH+bLaSIro/QUEmQ==", 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-03-14T12:15:59.012 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJljqfOLBAAulbwLxzEv+lHJOPUy3p4BQ==", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlt7bdJxAAH34bGUjVrlElC4nA11eNIQ==", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:15:59.013 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlmeIuNBAAzNKyG5CdCVCURsWZywpTnw==", 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.014 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDJ6fJlKwo/HBAA/Ma3FBjIdO1LF3RySRD0DQ==", 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-03-14T12:15:59.015 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDM6fJlot+1GhAAEQqFqku2rdbqZ3rEBfVO2g==", 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDP6fJlPRIYGRAAUxqEbSYn6AhIE+3lXEqEOg==", 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.016 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDS6fJlEYhJGBAApIl3qciLHwIJzDq5J2NkqA==", 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDY6fJlL2cQHxAAeHY/pdO0TvpUciB22xvOUg==", 2024-03-14T12:15:59.017 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDb6fJlrNxAHRAALwT6cIpCh9fE4NMoqV6PWA==", 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.018 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDe6fJlPLi2GxAAeZ/Nk/340P6Y0ZJYojfNMw==", 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.019 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDh6fJlLhW0GhAAtvxrvm93pbGKTJbzzpIRiQ==", 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg==", 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.020 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlUlkBBhAA9QGPbDVg8uWX5WTR2W936g==", 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlDQeZAhAAJW1K+HBJuwgzu4nVr8Mq8g==", 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJldFHSCxAAci5DVZh9406uu1BZIAZXiw==", 2024-03-14T12:15:59.022 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlhoD/AxAA/LRLwtPkAs14z0fLh+eGjA==", 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:15:59.023 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJl5qEWHRAAI8LqGdvUwjHG/5pNgcAB1A==", 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlxtwQGRAArcSgLFwgKjmSqjq0sst8Gw==", 2024-03-14T12:15:59.024 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:15:59.025 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:15:59.025 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:15:59.025 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:15:59.025 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.025 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.025 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:15:59.025 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3f406bbd0 msgr2=0x7fd3f406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3f406bbd0 0x7fd3f406e090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80017e0 tx=0x7fd3f80015d0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd40815cd10 msgr2=0x7fd4080af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd40815cd10 0x7fd4080af5b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00f820 tx=0x7fd3fc00fc20 comp rx=0 tx=0).stop 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 shutdown_connections 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4080afaf0 0x7fd4081598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd40815cd10 0x7fd4080af5b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd3f406bbd0 0x7fd3f406e090 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 --2- 172.21.15.82:0/1426949501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd40812ff10 0x7fd40815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 >> 172.21.15.82:0/1426949501 conn(0x7fd4080b1c70 msgr2=0x7fd408141370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 shutdown_connections 2024-03-14T12:15:59.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.014+0000 7fd40e4f8640 1 -- 172.21.15.82:0/1426949501 wait complete. 2024-03-14T12:15:59.028 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:15:59.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.130+0000 7fdcb7d2b640 1 Processor -- start 2024-03-14T12:15:59.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.130+0000 7fdcb7d2b640 1 -- start start 2024-03-14T12:15:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb7d2b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdcb006b670 0x7fdcb0072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb7d2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb006eb20 0x7fdcb006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb7d2b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb00727a0 con 0x7fdcb00b0200 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb7d2b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb0071810 con 0x7fdcb006b670 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb7d2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0071dd0 con 0x7fdcb006eb20 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb52c6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdcb00b0200 0x7fdcb00ce730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47020/0 (socket says 172.21.15.82:47020) 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb52c6640 1 -- 172.21.15.82:0/3825768656 learned_addr learned my addr 172.21.15.82:0/3825768656 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106007634 0 0) 0x7fdcb00727a0 con 0x7fdcb00b0200 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb006eb20 0x7fdcb006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc9c003520 con 0x7fdcb00b0200 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.131+0000 7fdcb5ac7640 1 --2- 172.21.15.82:0/3825768656 >> v2:172.21.15.164:3300/0 conn(0x7fdcb006b670 0x7fdcb0072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646038681 0 0) 0x7fdc9c003520 con 0x7fdcb00b0200 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb006eb20 msgr2=0x7fdcb006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb006eb20 0x7fdcb006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 >> v2:172.21.15.164:3300/0 conn(0x7fdcb006b670 msgr2=0x7fdcb0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 --2- 172.21.15.82:0/3825768656 >> v2:172.21.15.164:3300/0 conn(0x7fdcb006b670 0x7fdcb0072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb00d0410 con 0x7fdcb00b0200 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fdca0001670 con 0x7fdcb00b0200 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb5ac7640 1 --2- 172.21.15.82:0/3825768656 >> v2:172.21.15.164:3300/0 conn(0x7fdcb006b670 0x7fdcb0072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 >> v1:172.21.15.82:6789/0 conn(0x7fdcb00b0200 legacy=0x7fdcb00ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc9c0039b0 0x7fdc9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc9c004800 0x7fdc9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdc9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0071810 con 0x7fdc9c0051c0 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0071dd0 con 0x7fdc9c004800 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c00c000 con 0x7fdc9c0039b0 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb006eb20 0x7fdcb006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdc9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb52c6640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc9c004800 0x7fdc9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb5ac7640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc9c0039b0 0x7fdc9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb62c8640 1 -- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc9c0039b0 msgr2=0x7fdc9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc9c0039b0 0x7fdc9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb62c8640 1 -- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc9c004800 msgr2=0x7fdc9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc9c004800 0x7fdc9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.132+0000 7fdcb62c8640 1 -- 172.21.15.82:0/3825768656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c003520 con 0x7fdc9c0051c0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.133+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdc9c00b9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009f00 tx=0x7fdc9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc65055d6720ab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.133+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8032070 con 0x7fdc9c0051c0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.133+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca8030070 con 0x7fdc9c0051c0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.133+0000 7fdcb4ac5640 1 -- 172.21.15.82:0/3825768656 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8009b50 con 0x7fdc9c0051c0 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.133+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/3825768656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 msgr2=0x7fdc9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.133+0000 7fdcb7d2b640 1 --2- 172.21.15.82:0/3825768656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdc9c00b9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009f00 tx=0x7fdc9c005830 comp rx=0 tx=0).stop 2024-03-14T12:15:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.134+0000 7fdcb62c8640 1 -- 172.21.15.82:0/3825768656 reap_dead start 2024-03-14T12:15:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.134+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/3825768656 shutdown_connections 2024-03-14T12:15:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.134+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/3825768656 >> 172.21.15.82:0/3825768656 conn(0x7fdcb00d0b60 msgr2=0x7fdcb00d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.134+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/3825768656 shutdown_connections 2024-03-14T12:15:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.134+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/3825768656 wait complete. 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.135+0000 7fdcb7d2b640 1 Processor -- start 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.135+0000 7fdcb7d2b640 1 -- start start 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.135+0000 7fdcb7d2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdcb0157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.135+0000 7fdcb7d2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb015eac0 0x7fdcb015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.135+0000 7fdcb7d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcb01f4480 0x7fdcb01f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.135+0000 7fdcb7d2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00d0100 con 0x7fdc9c0051c0 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.135+0000 7fdcb7d2b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0071a60 con 0x7fdcb01f4480 2024-03-14T12:15:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb7d2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb006efb0 con 0x7fdcb015eac0 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdcb0157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdcb0157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57760/0 (socket says 172.21.15.82:57760) 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 -- 172.21.15.82:0/1870144919 learned_addr learned my addr 172.21.15.82:0/1870144919 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb52c6640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb015eac0 0x7fdcb015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcb01f4480 0x7fdcb01f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 -- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb015eac0 msgr2=0x7fdcb015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb015eac0 0x7fdcb015ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 -- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcb01f4480 msgr2=0x7fdcb01f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcb01f4480 0x7fdcb01f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 -- 172.21.15.82:0/1870144919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb015c9a0 con 0x7fdc9c0051c0 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb62c8640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcb01f4480 0x7fdcb01f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb52c6640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb015eac0 0x7fdcb015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb5ac7640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdcb0157c20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdcac010860 tx=0x7fdcac010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdca67fc640 1 -- 172.21.15.82:0/1870144919 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac01a020 con 0x7fdc9c0051c0 2024-03-14T12:15:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdca67fc640 1 -- 172.21.15.82:0/1870144919 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac00eb60 con 0x7fdc9c0051c0 2024-03-14T12:15:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdca67fc640 1 -- 172.21.15.82:0/1870144919 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00f040 con 0x7fdc9c0051c0 2024-03-14T12:15:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb015cc80 con 0x7fdc9c0051c0 2024-03-14T12:15:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.136+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb01f3f60 con 0x7fdc9c0051c0 2024-03-14T12:15:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.137+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb0161040 con 0x7fdc9c0051c0 2024-03-14T12:15:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.138+0000 7fdca67fc640 1 -- 172.21.15.82:0/1870144919 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdcac01d020 con 0x7fdc9c0051c0 2024-03-14T12:15:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.138+0000 7fdca67fc640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc9406bbd0 0x7fdc9406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.138+0000 7fdca67fc640 1 -- 172.21.15.82:0/1870144919 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdcac08b100 con 0x7fdc9c0051c0 2024-03-14T12:15:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.138+0000 7fdcb52c6640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc9406bbd0 0x7fdc9406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.138+0000 7fdcb52c6640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc9406bbd0 0x7fdc9406e090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdca0001050 tx=0x7fdca0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.142+0000 7fdca67fc640 1 -- 172.21.15.82:0/1870144919 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdcb0161040 con 0x7fdc9c0051c0 2024-03-14T12:15:59.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.315+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcb004f050 con 0x7fdc9c0051c0 2024-03-14T12:15:59.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.316+0000 7fdca67fc640 1 -- 172.21.15.82:0/1870144919 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12879 (secure 0 0 0) 0x7fdcac024020 con 0x7fdc9c0051c0 2024-03-14T12:15:59.315 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:59.315 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":42,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:15:58.636703+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:15:59.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc9406bbd0 msgr2=0x7fdc9406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc9406bbd0 0x7fdc9406e090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdca0001050 tx=0x7fdca0009040 comp rx=0 tx=0).stop 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 msgr2=0x7fdcb0157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdcb0157c20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdcac010860 tx=0x7fdcac010d20 comp rx=0 tx=0).stop 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 shutdown_connections 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcb01f4480 0x7fdcb01f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdc9406bbd0 0x7fdc9406e090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcb015eac0 0x7fdcb015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 --2- 172.21.15.82:0/1870144919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc9c0051c0 0x7fdcb0157c20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 >> 172.21.15.82:0/1870144919 conn(0x7fdcb00d0b60 msgr2=0x7fdcb00b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 shutdown_connections 2024-03-14T12:15:59.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.318+0000 7fdcb7d2b640 1 -- 172.21.15.82:0/1870144919 wait complete. 2024-03-14T12:15:59.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:15:59.328 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:15:59.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.432+0000 7ff5c884b640 1 Processor -- start 2024-03-14T12:15:59.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.432+0000 7ff5c884b640 1 -- start start 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c884b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5c40d0db0 0x7ff5c4162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c884b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff5c4147360 0x7ff5c40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c884b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c406e8e0 con 0x7ff5c41478b0 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c884b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c406eb80 con 0x7ff5c40d0db0 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c884b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c406f100 con 0x7ff5c4147360 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5c40d0db0 0x7ff5c4162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5c40d0db0 0x7ff5c4162560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57774/0 (socket says 172.21.15.82:57774) 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c259c640 1 -- 172.21.15.82:0/3309845512 learned_addr learned my addr 172.21.15.82:0/3309845512 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c259c640 1 -- 172.21.15.82:0/3309845512 >> v2:172.21.15.164:3300/0 conn(0x7ff5c4147360 msgr2=0x7ff5c40b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c259c640 1 --2- 172.21.15.82:0/3309845512 >> v2:172.21.15.164:3300/0 conn(0x7ff5c4147360 0x7ff5c40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c259c640 1 -- 172.21.15.82:0/3309845512 >> v1:172.21.15.164:6789/0 conn(0x7ff5c41478b0 legacy=0x7ff5c415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.433+0000 7ff5c259c640 1 -- 172.21.15.82:0/3309845512 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c406ee20 con 0x7ff5c40d0db0 2024-03-14T12:15:59.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.434+0000 7ff5c259c640 1 --2- 172.21.15.82:0/3309845512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4162560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4066f00 tx=0x7ff5b402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483bdedd045ad393 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.434+0000 7ff5c159a640 1 -- 172.21.15.82:0/3309845512 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b402c9f0 con 0x7ff5c40d0db0 2024-03-14T12:15:59.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.434+0000 7ff5c159a640 1 -- 172.21.15.82:0/3309845512 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b402cb90 con 0x7ff5c40d0db0 2024-03-14T12:15:59.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.434+0000 7ff5c159a640 1 -- 172.21.15.82:0/3309845512 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40343f0 con 0x7ff5c40d0db0 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 -- 172.21.15.82:0/3309845512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 msgr2=0x7ff5c4162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3309845512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4162560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4066f00 tx=0x7ff5b402be50 comp rx=0 tx=0).stop 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 -- 172.21.15.82:0/3309845512 shutdown_connections 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3309845512 >> v2:172.21.15.164:3300/0 conn(0x7ff5c4147360 0x7ff5c40b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3309845512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4162560 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 -- 172.21.15.82:0/3309845512 >> 172.21.15.82:0/3309845512 conn(0x7ff5c40b4fd0 msgr2=0x7ff5c40b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 -- 172.21.15.82:0/3309845512 shutdown_connections 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.435+0000 7ff5c884b640 1 -- 172.21.15.82:0/3309845512 wait complete. 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.436+0000 7ff5c884b640 1 Processor -- start 2024-03-14T12:15:59.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.436+0000 7ff5c884b640 1 -- start start 2024-03-14T12:15:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c884b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c884b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5c4147360 0x7ff5c4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c884b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5c41478b0 0x7ff5c41595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c884b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c4162aa0 con 0x7ff5c40d0db0 2024-03-14T12:15:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c884b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c4162040 con 0x7ff5c41478b0 2024-03-14T12:15:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c884b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c406d330 con 0x7ff5c4147360 2024-03-14T12:15:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57780/0 (socket says 172.21.15.82:57780) 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c259c640 1 -- 172.21.15.82:0/3051272687 learned_addr learned my addr 172.21.15.82:0/3051272687 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c259c640 1 -- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5c4147360 msgr2=0x7ff5c4159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c1d9b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5c4147360 0x7ff5c4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c2d9d640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5c41478b0 0x7ff5c41595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.437+0000 7ff5c259c640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5c4147360 0x7ff5c4159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5c259c640 1 -- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5c41478b0 msgr2=0x7ff5c41595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5c259c640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5c41478b0 0x7ff5c41595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5c259c640 1 -- 172.21.15.82:0/3051272687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c415b3d0 con 0x7ff5c40d0db0 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5c2d9d640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5c41478b0 0x7ff5c41595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5c259c640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4145c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40093e0 tx=0x7ff5b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5b2ffd640 1 -- 172.21.15.82:0/3051272687 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4043020 con 0x7ff5c40d0db0 2024-03-14T12:15:59.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5b2ffd640 1 -- 172.21.15.82:0/3051272687 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b403e450 con 0x7ff5c40d0db0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c415b6b0 con 0x7ff5c40d0db0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5b2ffd640 1 -- 172.21.15.82:0/3051272687 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b403e840 con 0x7ff5c40d0db0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.438+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c415b080 con 0x7ff5c40d0db0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.439+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c404f050 con 0x7ff5c40d0db0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.439+0000 7ff5b2ffd640 1 -- 172.21.15.82:0/3051272687 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff5b4046020 con 0x7ff5c40d0db0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.440+0000 7ff5b2ffd640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5a406bbd0 0x7ff5a406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.440+0000 7ff5b2ffd640 1 -- 172.21.15.82:0/3051272687 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5b40ad130 con 0x7ff5c40d0db0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.440+0000 7ff5c1d9b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5a406bbd0 0x7ff5a406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.440+0000 7ff5c1d9b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5a406bbd0 0x7ff5a406e090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b5b90 tx=0x7ff5b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.444+0000 7ff5b2ffd640 1 -- 172.21.15.82:0/3051272687 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff5b404d030 con 0x7ff5c40d0db0 2024-03-14T12:15:59.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.615+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5c415f5a0 con 0x7ff5c40d0db0 2024-03-14T12:15:59.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.616+0000 7ff5b2ffd640 1 -- 172.21.15.82:0/3051272687 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5b407bf70 con 0x7ff5c40d0db0 2024-03-14T12:15:59.615 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:15:59.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5a406bbd0 msgr2=0x7ff5a406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5a406bbd0 0x7ff5a406e090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b5b90 tx=0x7ff5b8008040 comp rx=0 tx=0).stop 2024-03-14T12:15:59.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 msgr2=0x7ff5c4145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4145c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40093e0 tx=0x7ff5b4004a40 comp rx=0 tx=0).stop 2024-03-14T12:15:59.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 shutdown_connections 2024-03-14T12:15:59.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5c41478b0 0x7ff5c41595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5a406bbd0 0x7ff5a406e090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5c4147360 0x7ff5c4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.617+0000 7ff5c884b640 1 --2- 172.21.15.82:0/3051272687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c40d0db0 0x7ff5c4145c10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.618+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 >> 172.21.15.82:0/3051272687 conn(0x7ff5c40b4fd0 msgr2=0x7ff5c4160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:59.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.618+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 shutdown_connections 2024-03-14T12:15:59.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.618+0000 7ff5c884b640 1 -- 172.21.15.82:0/3051272687 wait complete. 2024-03-14T12:15:59.628 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T12:15:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.729+0000 7f5dabfd4640 1 Processor -- start 2024-03-14T12:15:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.729+0000 7f5dabfd4640 1 -- start start 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5dabfd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40b0f60 0x7f5da40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5dabfd4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5da40afff0 0x7f5da40d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5dabfd4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5da406ad10 0x7f5da406b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5dabfd4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da40d0ee0 con 0x7f5da40afff0 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5dabfd4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da40b4ee0 con 0x7f5da406ad10 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5dabfd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da406b6e0 con 0x7f5da40b0f60 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5da40afff0 0x7f5da40d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5da40afff0 0x7f5da40d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57782/0 (socket says 172.21.15.82:57782) 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 -- 172.21.15.82:0/340662400 learned_addr learned my addr 172.21.15.82:0/340662400 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 -- 172.21.15.82:0/340662400 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40b0f60 msgr2=0x7f5da40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 --2- 172.21.15.82:0/340662400 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40b0f60 0x7f5da40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 -- 172.21.15.82:0/340662400 >> v2:172.21.15.164:3300/0 conn(0x7f5da406ad10 msgr2=0x7f5da406b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:15:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 --2- 172.21.15.82:0/340662400 >> v2:172.21.15.164:3300/0 conn(0x7f5da406ad10 0x7f5da406b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.730+0000 7f5da956f640 1 -- 172.21.15.82:0/340662400 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da40b1340 con 0x7f5da40afff0 2024-03-14T12:15:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.731+0000 7f5da956f640 1 --2- 172.21.15.82:0/340662400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da40afff0 0x7f5da40d09a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5da406cc80 tx=0x7f5d9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5e108eb5ad16cf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.731+0000 7f5da8d6e640 1 -- 172.21.15.82:0/340662400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94035020 con 0x7f5da40afff0 2024-03-14T12:15:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.731+0000 7f5da8d6e640 1 -- 172.21.15.82:0/340662400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94031870 con 0x7f5da40afff0 2024-03-14T12:15:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.731+0000 7f5da8d6e640 1 -- 172.21.15.82:0/340662400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94031c40 con 0x7f5da40afff0 2024-03-14T12:15:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 -- 172.21.15.82:0/340662400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da40afff0 msgr2=0x7f5da40d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/340662400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da40afff0 0x7f5da40d09a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5da406cc80 tx=0x7f5d9402b910 comp rx=0 tx=0).stop 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 -- 172.21.15.82:0/340662400 shutdown_connections 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/340662400 >> v2:172.21.15.164:3300/0 conn(0x7f5da406ad10 0x7f5da406b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/340662400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da40afff0 0x7f5da40d09a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/340662400 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40b0f60 0x7f5da40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 -- 172.21.15.82:0/340662400 >> 172.21.15.82:0/340662400 conn(0x7f5da4072310 msgr2=0x7f5da4072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 -- 172.21.15.82:0/340662400 shutdown_connections 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.732+0000 7f5dabfd4640 1 -- 172.21.15.82:0/340662400 wait complete. 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.733+0000 7f5dabfd4640 1 Processor -- start 2024-03-14T12:15:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.733+0000 7f5dabfd4640 1 -- start start 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da406ad10 0x7f5da4157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40afff0 0x7f5da40c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da40b0f60 0x7f5da40c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da40cd5f0 con 0x7f5da406ad10 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da40cd310 con 0x7f5da40b0f60 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da40cd8d0 con 0x7f5da40afff0 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da406ad10 0x7f5da4157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da406ad10 0x7f5da4157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57794/0 (socket says 172.21.15.82:57794) 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 -- 172.21.15.82:0/2100456672 learned_addr learned my addr 172.21.15.82:0/2100456672 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 -- 172.21.15.82:0/2100456672 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40afff0 msgr2=0x7f5da40c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40afff0 0x7f5da40c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 -- 172.21.15.82:0/2100456672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da40b0f60 msgr2=0x7f5da40c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da40b0f60 0x7f5da40c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 -- 172.21.15.82:0/2100456672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da40c82c0 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5da9d70640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da406ad10 0x7f5da4157940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ba80 tx=0x7f5da000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5d9a7fc640 1 -- 172.21.15.82:0/2100456672 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0019070 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5d9a7fc640 1 -- 172.21.15.82:0/2100456672 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da000cc20 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da40cdc10 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5d9a7fc640 1 -- 172.21.15.82:0/2100456672 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0014440 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.734+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da40ce1d0 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.735+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da404f050 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.736+0000 7f5d9a7fc640 1 -- 172.21.15.82:0/2100456672 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5da001c070 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.736+0000 7f5d9a7fc640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8806bbd0 0x7f5d8806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.736+0000 7f5d9a7fc640 1 -- 172.21.15.82:0/2100456672 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5da008a2c0 con 0x7f5da406ad10 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.736+0000 7f5da956f640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8806bbd0 0x7f5d8806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:15:59.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.737+0000 7f5da956f640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8806bbd0 0x7f5d8806e090 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5d94036000 tx=0x7f5d94030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:15:59.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.740+0000 7f5d9a7fc640 1 -- 172.21.15.82:0/2100456672 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5da0058b40 con 0x7f5da406ad10 2024-03-14T12:15:59.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.915+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5da4074470 con 0x7f5da406ad10 2024-03-14T12:15:59.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.916+0000 7f5d9a7fc640 1 -- 172.21.15.82:0/2100456672 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5da005cbe0 con 0x7f5da406ad10 2024-03-14T12:15:59.915 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:15:59.915 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-03-14T12:15:59.915 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:15:59.915 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-03-14T12:15:59.915 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-03-14T12:15:59.916 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:15:59.916 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-03-14T12:15:59.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8806bbd0 msgr2=0x7f5d8806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8806bbd0 0x7f5d8806e090 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5d94036000 tx=0x7f5d94030000 comp rx=0 tx=0).stop 2024-03-14T12:15:59.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da406ad10 msgr2=0x7f5da4157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:15:59.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da406ad10 0x7f5da4157940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ba80 tx=0x7f5da000bf40 comp rx=0 tx=0).stop 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 shutdown_connections 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da40b0f60 0x7f5da40c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8806bbd0 0x7f5d8806e090 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5da40afff0 0x7f5da40c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 --2- 172.21.15.82:0/2100456672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5da406ad10 0x7f5da4157940 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 >> 172.21.15.82:0/2100456672 conn(0x7f5da4072310 msgr2=0x7f5da406d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 shutdown_connections 2024-03-14T12:15:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:15:59.918+0000 7f5dabfd4640 1 -- 172.21.15.82:0/2100456672 wait complete. 2024-03-14T12:15:59.928 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T12:15:59.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T12:16:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.033+0000 7f96462dc640 1 Processor -- start 2024-03-14T12:16:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.033+0000 7f96462dc640 1 -- start start 2024-03-14T12:16:00.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.033+0000 7f96462dc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f96400b4a30 0x7f96400b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.033+0000 7f96462dc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400723b0 con 0x7f96400b3370 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.033+0000 7f96462dc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9640072690 con 0x7f9640159c90 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.033+0000 7f96462dc640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400ce050 con 0x7f96400b4a30 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f9644879640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f96400b3370 0x7f96400cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47024/0 (socket says 172.21.15.82:47024) 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f9644879640 1 -- 172.21.15.82:0/3828667679 learned_addr learned my addr 172.21.15.82:0/3828667679 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963f7fe640 1 --2- 172.21.15.82:0/3828667679 >> v2:172.21.15.164:3300/0 conn(0x7f96400b4a30 0x7f96400b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139038910 0 0) 0x7f96400723b0 con 0x7f96400b3370 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9628003520 con 0x7f96400b3370 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3213577895 0 0) 0x7f9628003520 con 0x7f96400b3370 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 >> v2:172.21.15.164:3300/0 conn(0x7f96400b4a30 msgr2=0x7f96400b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 --2- 172.21.15.82:0/3828667679 >> v2:172.21.15.164:3300/0 conn(0x7f96400b4a30 0x7f96400b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 >> v1:172.21.15.164:6789/0 conn(0x7f9640159c90 legacy=0x7f964015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400ce280 con 0x7f96400b3370 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f9638001a10 con 0x7f96400b3370 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 >> v1:172.21.15.82:6789/0 conn(0x7f96400b3370 legacy=0x7f96400cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280039b0 0x7f9628003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9628004800 0x7f9628004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962800c030 con 0x7f96280051c0 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962800c1e0 con 0x7f9628004800 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962800c3e0 con 0x7f96280039b0 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f9644879640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963f7fe640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9628004800 0x7f9628004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.034+0000 7f963ffff640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280039b0 0x7f9628003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f9644879640 1 -- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280039b0 msgr2=0x7f9628003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f9644879640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280039b0 0x7f9628003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f9644879640 1 -- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9628004800 msgr2=0x7f9628004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f9644879640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9628004800 0x7f9628004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f9644879640 1 -- 172.21.15.82:0/3828667679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9628003520 con 0x7f96280051c0 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f963ffff640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280039b0 0x7f9628003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f963f7fe640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9628004800 0x7f9628004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f9644879640 1 -- 172.21.15.82:0/3828667679 reap_dead start 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f9644879640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f964015d660 tx=0x7f9628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929c73aff06ee2ce server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963800a480 con 0x7f96280051c0 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638034630 con 0x7f96280051c0 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.035+0000 7f963effd640 1 -- 172.21.15.82:0/3828667679 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963802ce60 con 0x7f96280051c0 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.036+0000 7f96462dc640 1 -- 172.21.15.82:0/3828667679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96280051c0 msgr2=0x7f962800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.036+0000 7f96462dc640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f964015d660 tx=0x7f9628005830 comp rx=0 tx=0).stop 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.036+0000 7f96462dc640 1 -- 172.21.15.82:0/3828667679 shutdown_connections 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.036+0000 7f96462dc640 1 --2- 172.21.15.82:0/3828667679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.036+0000 7f96462dc640 1 -- 172.21.15.82:0/3828667679 >> 172.21.15.82:0/3828667679 conn(0x7f96400b19b0 msgr2=0x7f96400b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.036+0000 7f96462dc640 1 -- 172.21.15.82:0/3828667679 shutdown_connections 2024-03-14T12:16:00.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.036+0000 7f96462dc640 1 -- 172.21.15.82:0/3828667679 wait complete. 2024-03-14T12:16:00.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.037+0000 7f96462dc640 1 Processor -- start 2024-03-14T12:16:00.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.037+0000 7f96462dc640 1 -- start start 2024-03-14T12:16:00.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f96462dc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280051c0 0x7f9640144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f96462dc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9640144c70 0x7f9640141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f96462dc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9640141f90 0x7f9640148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f96462dc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400b2e30 con 0x7f9640144c70 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f96462dc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9640072970 con 0x7f9640141f90 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f96462dc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400b2fb0 con 0x7f96280051c0 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9640144c70 0x7f9640141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9640144c70 0x7f9640141a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57820/0 (socket says 172.21.15.82:57820) 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 -- 172.21.15.82:0/1840063876 learned_addr learned my addr 172.21.15.82:0/1840063876 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963ffff640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280051c0 0x7f9640144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f9644879640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9640141f90 0x7f9640148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 -- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280051c0 msgr2=0x7f9640144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280051c0 0x7f9640144730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 -- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9640141f90 msgr2=0x7f9640148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9640141f90 0x7f9640148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.038+0000 7f963f7fe640 1 -- 172.21.15.82:0/1840063876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96401435d0 con 0x7f9640144c70 2024-03-14T12:16:00.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.039+0000 7f963f7fe640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9640144c70 0x7f9640141a50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9634011510 tx=0x7f96340119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.039+0000 7f963cff9640 1 -- 172.21.15.82:0/1840063876 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634015870 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.039+0000 7f963cff9640 1 -- 172.21.15.82:0/1840063876 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634015a10 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.039+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96401438b0 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.039+0000 7f963cff9640 1 -- 172.21.15.82:0/1840063876 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634015dc0 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.039+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9640143e70 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.040+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964004f050 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.040+0000 7f963cff9640 1 -- 172.21.15.82:0/1840063876 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9634013400 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.040+0000 7f963cff9640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f962c06bc00 0x7f962c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.041+0000 7f963ffff640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f962c06bc00 0x7f962c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.041+0000 7f963cff9640 1 -- 172.21.15.82:0/1840063876 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f963408af10 con 0x7f9640144c70 2024-03-14T12:16:00.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.041+0000 7f963ffff640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f962c06bc00 0x7f962c06e0c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9630000c00 tx=0x7f9630009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:00.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.045+0000 7f963cff9640 1 -- 172.21.15.82:0/1840063876 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9634019030 con 0x7f9640144c70 2024-03-14T12:16:00.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.214+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f96400b3f30 con 0x7f9640144c70 2024-03-14T12:16:00.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.655+0000 7f963cff9640 1 -- 172.21.15.82:0/1840063876 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9634059d50 con 0x7f9640144c70 2024-03-14T12:16:00.654 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f962c06bc00 msgr2=0x7f962c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f962c06bc00 0x7f962c06e0c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9630000c00 tx=0x7f9630009040 comp rx=0 tx=0).stop 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9640144c70 msgr2=0x7f9640141a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9640144c70 0x7f9640141a50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9634011510 tx=0x7f96340119d0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 shutdown_connections 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9640141f90 0x7f9640148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9640144c70 0x7f9640141a50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f962c06bc00 0x7f962c06e0c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 --2- 172.21.15.82:0/1840063876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96280051c0 0x7f9640144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 >> 172.21.15.82:0/1840063876 conn(0x7f96400b19b0 msgr2=0x7f96400cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:00.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 shutdown_connections 2024-03-14T12:16:00.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.658+0000 7f96462dc640 1 -- 172.21.15.82:0/1840063876 wait complete. 2024-03-14T12:16:00.671 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T12:16:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.779+0000 7f28461c4640 1 Processor -- start 2024-03-14T12:16:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.780+0000 7f28461c4640 1 -- start start 2024-03-14T12:16:00.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.780+0000 7f28461c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28400d2ab0 0x7f28400b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.780+0000 7f28461c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28400cece0 con 0x7f284015de20 2024-03-14T12:16:00.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.780+0000 7f28461c4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28400cefc0 con 0x7f284015e200 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.780+0000 7f28461c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28400cfd80 con 0x7f28400d2ab0 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.780+0000 7f283f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f284015de20 0x7f28400d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47030/0 (socket says 172.21.15.82:47030) 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.780+0000 7f283f7fe640 1 -- 172.21.15.82:0/932785400 learned_addr learned my addr 172.21.15.82:0/932785400 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952168508 0 0) 0x7f28400cece0 con 0x7f284015de20 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283ffff640 1 --2- 172.21.15.82:0/932785400 >> v2:172.21.15.164:3300/0 conn(0x7f28400d2ab0 0x7f28400b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f282c003520 con 0x7f284015de20 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682161000 0 0) 0x7f282c003520 con 0x7f284015de20 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 >> v2:172.21.15.164:3300/0 conn(0x7f28400d2ab0 msgr2=0x7f28400b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 --2- 172.21.15.82:0/932785400 >> v2:172.21.15.164:3300/0 conn(0x7f28400d2ab0 0x7f28400b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 >> v1:172.21.15.164:6789/0 conn(0x7f284015e200 legacy=0x7f28400d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400cf2a0 con 0x7f284015de20 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f2830001670 con 0x7f284015de20 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 >> v1:172.21.15.82:6789/0 conn(0x7f284015de20 legacy=0x7f28400d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0039b0 0x7f282c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f282c004800 0x7f282c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f282c0051c0 0x7f282c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c00c130 con 0x7f282c0051c0 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c00c2e0 con 0x7f282c0039b0 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c00c4e0 con 0x7f282c004800 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283ffff640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f282c0051c0 0x7f282c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283f7fe640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0039b0 0x7f282c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f2836ffd640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f282c004800 0x7f282c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283ffff640 1 -- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f282c004800 msgr2=0x7f282c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283ffff640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f282c004800 0x7f282c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283ffff640 1 -- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0039b0 msgr2=0x7f282c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283ffff640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0039b0 0x7f282c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.781+0000 7f283ffff640 1 -- 172.21.15.82:0/932785400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c003520 con 0x7f282c0051c0 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.782+0000 7f283ffff640 1 -- 172.21.15.82:0/932785400 reap_dead start 2024-03-14T12:16:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.782+0000 7f283ffff640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f282c0051c0 0x7f282c00b9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2838009fc0 tx=0x7f282c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c85c545f2a5448 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.782+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838009c10 con 0x7f282c0051c0 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.782+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283802cca0 con 0x7f282c0051c0 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.782+0000 7f283effd640 1 -- 172.21.15.82:0/932785400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838003bd0 con 0x7f282c0051c0 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.783+0000 7f28461c4640 1 -- 172.21.15.82:0/932785400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f282c0051c0 msgr2=0x7f282c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.783+0000 7f28461c4640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f282c0051c0 0x7f282c00b9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2838009fc0 tx=0x7f282c005830 comp rx=0 tx=0).stop 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.783+0000 7f28461c4640 1 -- 172.21.15.82:0/932785400 shutdown_connections 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.783+0000 7f28461c4640 1 --2- 172.21.15.82:0/932785400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f282c0051c0 0x7f282c00b9c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.783+0000 7f28461c4640 1 -- 172.21.15.82:0/932785400 >> 172.21.15.82:0/932785400 conn(0x7f284009d030 msgr2=0x7f28400d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.783+0000 7f28461c4640 1 -- 172.21.15.82:0/932785400 shutdown_connections 2024-03-14T12:16:00.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.783+0000 7f28461c4640 1 -- 172.21.15.82:0/932785400 wait complete. 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.784+0000 7f28461c4640 1 Processor -- start 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.784+0000 7f28461c4640 1 -- start start 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f28461c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0051c0 0x7f284014f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f28461c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28401f7020 0x7f28400c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f28461c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28401f74a0 0x7f28400ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f28461c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400ceac0 con 0x7f28401f7020 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f28461c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400ce940 con 0x7f282c0051c0 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f28461c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28400d1d60 con 0x7f28401f74a0 2024-03-14T12:16:00.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28401f7020 0x7f28400c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28401f7020 0x7f28400c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57840/0 (socket says 172.21.15.82:57840) 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 -- 172.21.15.82:0/705948444 learned_addr learned my addr 172.21.15.82:0/705948444 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f283ffff640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28401f74a0 0x7f28400ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f283f7fe640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0051c0 0x7f284014f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 -- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28401f74a0 msgr2=0x7f28400ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28401f74a0 0x7f28400ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 -- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0051c0 msgr2=0x7f284014f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0051c0 0x7f284014f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 -- 172.21.15.82:0/705948444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28401f7a70 con 0x7f28401f7020 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.785+0000 7f2836ffd640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28401f7020 0x7f28400c9cb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f282800e480 tx=0x7f282800e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.786+0000 7f283cff9640 1 -- 172.21.15.82:0/705948444 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828019070 con 0x7f28401f7020 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.786+0000 7f283cff9640 1 -- 172.21.15.82:0/705948444 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28280052e0 con 0x7f28401f7020 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.786+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28401f7d50 con 0x7f28401f7020 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.786+0000 7f283cff9640 1 -- 172.21.15.82:0/705948444 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828003830 con 0x7f28401f7020 2024-03-14T12:16:00.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.786+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284014f620 con 0x7f28401f7020 2024-03-14T12:16:00.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.787+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284004f050 con 0x7f28401f7020 2024-03-14T12:16:00.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.787+0000 7f283cff9640 1 -- 172.21.15.82:0/705948444 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f282801c070 con 0x7f28401f7020 2024-03-14T12:16:00.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.787+0000 7f283cff9640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f282006bc00 0x7f282006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:00.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.788+0000 7f283cff9640 1 -- 172.21.15.82:0/705948444 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2828089d30 con 0x7f28401f7020 2024-03-14T12:16:00.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.788+0000 7f283f7fe640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f282006bc00 0x7f282006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:00.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.788+0000 7f283f7fe640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f282006bc00 0x7f282006e0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2830001050 tx=0x7f28300078f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:00.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.792+0000 7f283cff9640 1 -- 172.21.15.82:0/705948444 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f28280583b0 con 0x7f28401f7020 2024-03-14T12:16:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:00.964+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f28401f7f20 con 0x7f28401f7020 2024-03-14T12:16:01.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.661+0000 7f283cff9640 1 -- 172.21.15.82:0/705948444 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f282805c450 con 0x7f28401f7020 2024-03-14T12:16:01.660 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-03-14T12:16:01.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f282006bc00 msgr2=0x7f282006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:01.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f282006bc00 0x7f282006e0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2830001050 tx=0x7f28300078f0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28401f7020 msgr2=0x7f28400c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:01.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28401f7020 0x7f28400c9cb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f282800e480 tx=0x7f282800e880 comp rx=0 tx=0).stop 2024-03-14T12:16:01.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 shutdown_connections 2024-03-14T12:16:01.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f282006bc00 0x7f282006e0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28401f74a0 0x7f28400ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28401f7020 0x7f28400c9cb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 --2- 172.21.15.82:0/705948444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f282c0051c0 0x7f284014f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 >> 172.21.15.82:0/705948444 conn(0x7f284009d030 msgr2=0x7f2840160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:01.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 shutdown_connections 2024-03-14T12:16:01.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.664+0000 7f28461c4640 1 -- 172.21.15.82:0/705948444 wait complete. 2024-03-14T12:16:01.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T12:16:01.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.797+0000 7f532f0f8640 1 Processor -- start 2024-03-14T12:16:01.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.797+0000 7f532f0f8640 1 -- start start 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532f0f8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f532815e0c0 0x7f53280cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532f0f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f53280d08c0 0x7f53280ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532f0f8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53280cf440 con 0x7f53280b0bc0 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532f0f8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280cf1e0 con 0x7f53280d08c0 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532f0f8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280ccb60 con 0x7f532815e0c0 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532d695640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f53280d08c0 0x7f53280ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532d695640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f53280d08c0 0x7f53280ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37010/0 (socket says 172.21.15.82:37010) 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532d695640 1 -- 172.21.15.82:0/1088348545 learned_addr learned my addr 172.21.15.82:0/1088348545 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:01.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532ce94640 1 --2- 172.21.15.82:0/1088348545 >> v2:172.21.15.164:3300/0 conn(0x7f532815e0c0 0x7f53280cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532d695640 1 -- 172.21.15.82:0/1088348545 >> v2:172.21.15.164:3300/0 conn(0x7f532815e0c0 msgr2=0x7f53280cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532d695640 1 --2- 172.21.15.82:0/1088348545 >> v2:172.21.15.164:3300/0 conn(0x7f532815e0c0 0x7f53280cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532d695640 1 -- 172.21.15.82:0/1088348545 >> v1:172.21.15.164:6789/0 conn(0x7f53280b0bc0 legacy=0x7f53280d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.798+0000 7f532d695640 1 -- 172.21.15.82:0/1088348545 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280cce40 con 0x7f53280d08c0 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f532d695640 1 --2- 172.21.15.82:0/1088348545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f53280ceca0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53280d0100 tx=0x7f532002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1084c9ebb9b6332b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f531f7fe640 1 -- 172.21.15.82:0/1088348545 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532002cb20 con 0x7f53280d08c0 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f531f7fe640 1 -- 172.21.15.82:0/1088348545 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532002ccc0 con 0x7f53280d08c0 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f531f7fe640 1 -- 172.21.15.82:0/1088348545 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5320034610 con 0x7f53280d08c0 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f532f0f8640 1 -- 172.21.15.82:0/1088348545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 msgr2=0x7f53280ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f532f0f8640 1 --2- 172.21.15.82:0/1088348545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f53280ceca0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53280d0100 tx=0x7f532002bf80 comp rx=0 tx=0).stop 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f532f0f8640 1 -- 172.21.15.82:0/1088348545 shutdown_connections 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f532f0f8640 1 --2- 172.21.15.82:0/1088348545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f53280ceca0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f532f0f8640 1 --2- 172.21.15.82:0/1088348545 >> v2:172.21.15.164:3300/0 conn(0x7f532815e0c0 0x7f53280cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.799+0000 7f532f0f8640 1 -- 172.21.15.82:0/1088348545 >> 172.21.15.82:0/1088348545 conn(0x7f53280d1bd0 msgr2=0x7f53280d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:01.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.800+0000 7f532f0f8640 1 -- 172.21.15.82:0/1088348545 shutdown_connections 2024-03-14T12:16:01.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.800+0000 7f532f0f8640 1 -- 172.21.15.82:0/1088348545 wait complete. 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.801+0000 7f532f0f8640 1 Processor -- start 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.801+0000 7f532f0f8640 1 -- start start 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b0bc0 0x7f532814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f5328159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815e0c0 0x7f5328159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532815e4a0 con 0x7f53280d08c0 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53280cd400 con 0x7f53280b0bc0 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532806bbc0 con 0x7f532815e0c0 2024-03-14T12:16:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f5328159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f5328159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37014/0 (socket says 172.21.15.82:37014) 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 -- 172.21.15.82:0/3112063289 learned_addr learned my addr 172.21.15.82:0/3112063289 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532ce94640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b0bc0 0x7f532814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532d695640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815e0c0 0x7f5328159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 -- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815e0c0 msgr2=0x7f5328159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815e0c0 0x7f5328159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 -- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b0bc0 msgr2=0x7f532814c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b0bc0 0x7f532814c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 -- 172.21.15.82:0/3112063289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532814c6e0 con 0x7f53280d08c0 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532d695640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815e0c0 0x7f5328159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531ffff640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f5328159320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f531800c400 tx=0x7f531800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531d7fa640 1 -- 172.21.15.82:0/3112063289 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180052d0 con 0x7f53280d08c0 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531d7fa640 1 -- 172.21.15.82:0/3112063289 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318005490 con 0x7f53280d08c0 2024-03-14T12:16:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53280c8e20 con 0x7f53280d08c0 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f531d7fa640 1 -- 172.21.15.82:0/3112063289 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318007260 con 0x7f53280d08c0 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.802+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53280c93c0 con 0x7f53280d08c0 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.803+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532804f050 con 0x7f53280d08c0 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.804+0000 7f531d7fa640 1 -- 172.21.15.82:0/3112063289 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5318007400 con 0x7f53280d08c0 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.804+0000 7f531d7fa640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f531406bbd0 0x7f531406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.804+0000 7f531d7fa640 1 -- 172.21.15.82:0/3112063289 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5318089e40 con 0x7f53280d08c0 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.804+0000 7f532ce94640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f531406bbd0 0x7f531406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.805+0000 7f532ce94640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f531406bbd0 0x7f531406e090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5324009aa0 tx=0x7f5324009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:01.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:01.809+0000 7f531d7fa640 1 -- 172.21.15.82:0/3112063289 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f53180583d0 con 0x7f53280d08c0 2024-03-14T12:16:02.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.015+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f532806c100 con 0x7f53280d08c0 2024-03-14T12:16:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 6 and data pool 7 2024-03-14T12:16:02.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.675+0000 7f531d7fa640 1 -- 172.21.15.82:0/3112063289 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f531805c470 con 0x7f53280d08c0 2024-03-14T12:16:02.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.680+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f531406bbd0 msgr2=0x7f531406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:02.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.680+0000 7f532f0f8640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f531406bbd0 0x7f531406e090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5324009aa0 tx=0x7f5324009190 comp rx=0 tx=0).stop 2024-03-14T12:16:02.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.680+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 msgr2=0x7f5328159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:02.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.680+0000 7f532f0f8640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f5328159320 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f531800c400 tx=0x7f531800c8c0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 shutdown_connections 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f531406bbd0 0x7f531406e090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815e0c0 0x7f5328159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53280d08c0 0x7f5328159320 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 --2- 172.21.15.82:0/3112063289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b0bc0 0x7f532814c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 >> 172.21.15.82:0/3112063289 conn(0x7f53280d1bd0 msgr2=0x7f53280ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 shutdown_connections 2024-03-14T12:16:02.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.685+0000 7f532f0f8640 1 -- 172.21.15.82:0/3112063289 wait complete. 2024-03-14T12:16:02.705 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:02.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.809+0000 7ffa331cd640 1 Processor -- start 2024-03-14T12:16:02.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.809+0000 7ffa331cd640 1 -- start start 2024-03-14T12:16:02.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa331cd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa2c0722c0 0x7ffa2c06b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:02.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa331cd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c0d1590 con 0x7ffa2c06ad10 2024-03-14T12:16:02.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa331cd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c0717e0 con 0x7ffa2c145060 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa331cd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c072710 con 0x7ffa2c0722c0 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa3176a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa2c0722c0 0x7ffa2c06b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa3176a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa2c0722c0 0x7ffa2c06b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37016/0 (socket says 172.21.15.82:37016) 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa30f69640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffa2c06ad10 0x7ffa2c06b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36340/0 (socket says 172.21.15.82:36340) 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa3176a640 1 -- 172.21.15.82:0/4104835287 learned_addr learned my addr 172.21.15.82:0/4104835287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa3176a640 1 -- 172.21.15.82:0/4104835287 >> v1:172.21.15.164:6789/0 conn(0x7ffa2c145060 legacy=0x7ffa2c06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa3176a640 1 -- 172.21.15.82:0/4104835287 >> v1:172.21.15.82:6789/0 conn(0x7ffa2c06ad10 legacy=0x7ffa2c06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa3176a640 1 -- 172.21.15.82:0/4104835287 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c071ac0 con 0x7ffa2c0722c0 2024-03-14T12:16:02.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.810+0000 7ffa2b7fe640 1 -- 172.21.15.82:0/4104835287 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883707424 0 0) 0x7ffa2c0d1590 con 0x7ffa2c06ad10 2024-03-14T12:16:02.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa3176a640 1 --2- 172.21.15.82:0/4104835287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c06b800 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009e50 tx=0x7ffa2402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977bad2e13610039 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:02.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa2b7fe640 1 -- 172.21.15.82:0/4104835287 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2402ca80 con 0x7ffa2c0722c0 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa2b7fe640 1 -- 172.21.15.82:0/4104835287 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2402cc20 con 0x7ffa2c0722c0 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa2b7fe640 1 -- 172.21.15.82:0/4104835287 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa240343f0 con 0x7ffa2c0722c0 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa331cd640 1 -- 172.21.15.82:0/4104835287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 msgr2=0x7ffa2c06b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa331cd640 1 --2- 172.21.15.82:0/4104835287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c06b800 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009e50 tx=0x7ffa2402bee0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa331cd640 1 -- 172.21.15.82:0/4104835287 shutdown_connections 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa331cd640 1 --2- 172.21.15.82:0/4104835287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c06b800 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa331cd640 1 -- 172.21.15.82:0/4104835287 >> 172.21.15.82:0/4104835287 conn(0x7ffa2c0d09d0 msgr2=0x7ffa2c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.811+0000 7ffa331cd640 1 -- 172.21.15.82:0/4104835287 shutdown_connections 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.812+0000 7ffa331cd640 1 -- 172.21.15.82:0/4104835287 wait complete. 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 Processor -- start 2024-03-14T12:16:02.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 -- start start 2024-03-14T12:16:02.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa2c06ad10 0x7ffa2c140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:02.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:02.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa2c145060 0x7ffa2c151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c06eff0 con 0x7ffa2c0722c0 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0d1870 con 0x7ffa2c06ad10 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.813+0000 7ffa331cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c06f170 con 0x7ffa2c145060 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37030/0 (socket says 172.21.15.82:37030) 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 -- 172.21.15.82:0/696559349 learned_addr learned my addr 172.21.15.82:0/696559349 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa30f69640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa2c06ad10 0x7ffa2c140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa3176a640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa2c145060 0x7ffa2c151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 -- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa2c145060 msgr2=0x7ffa2c151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa2c145060 0x7ffa2c151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 -- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa2c06ad10 msgr2=0x7ffa2c140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa2c06ad10 0x7ffa2c140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 -- 172.21.15.82:0/696559349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c15bbb0 con 0x7ffa2c0722c0 2024-03-14T12:16:02.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa2bfff640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c151860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800b3e0 tx=0x7ffa1800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:02.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa297fa640 1 -- 172.21.15.82:0/696559349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18015020 con 0x7ffa2c0722c0 2024-03-14T12:16:02.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa297fa640 1 -- 172.21.15.82:0/696559349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18011850 con 0x7ffa2c0722c0 2024-03-14T12:16:02.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c152420 con 0x7ffa2c0722c0 2024-03-14T12:16:02.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa297fa640 1 -- 172.21.15.82:0/696559349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18011c40 con 0x7ffa2c0722c0 2024-03-14T12:16:02.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.814+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c14f820 con 0x7ffa2c0722c0 2024-03-14T12:16:02.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.815+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c04f050 con 0x7ffa2c0722c0 2024-03-14T12:16:02.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.816+0000 7ffa297fa640 1 -- 172.21.15.82:0/696559349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ffa18018020 con 0x7ffa2c0722c0 2024-03-14T12:16:02.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.816+0000 7ffa297fa640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa1006bbd0 0x7ffa1006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:02.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.816+0000 7ffa297fa640 1 -- 172.21.15.82:0/696559349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa1808a990 con 0x7ffa2c0722c0 2024-03-14T12:16:02.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.816+0000 7ffa30f69640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa1006bbd0 0x7ffa1006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:02.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.816+0000 7ffa30f69640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa1006bbd0 0x7ffa1006e090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffa20007fd0 tx=0x7ffa20009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:02.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.821+0000 7ffa297fa640 1 -- 172.21.15.82:0/696559349 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ffa1801f020 con 0x7ffa2c0722c0 2024-03-14T12:16:02.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.997+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa2c0747b0 con 0x7ffa2c0722c0 2024-03-14T12:16:02.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:02.998+0000 7ffa297fa640 1 -- 172.21.15.82:0/696559349 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16614 (secure 0 0 0) 0x7ffa180172c0 con 0x7ffa2c0722c0 2024-03-14T12:16:02.998 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:02.998 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":45,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:02.016940+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa1006bbd0 msgr2=0x7ffa1006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa1006bbd0 0x7ffa1006e090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffa20007fd0 tx=0x7ffa20009040 comp rx=0 tx=0).stop 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 msgr2=0x7ffa2c151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c151860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800b3e0 tx=0x7ffa1800b8a0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 shutdown_connections 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffa1006bbd0 0x7ffa1006e090 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa2c145060 0x7ffa2c151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa2c0722c0 0x7ffa2c151860 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.000+0000 7ffa331cd640 1 --2- 172.21.15.82:0/696559349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa2c06ad10 0x7ffa2c140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.001+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 >> 172.21.15.82:0/696559349 conn(0x7ffa2c0d09d0 msgr2=0x7ffa2c0d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:03.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.001+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 shutdown_connections 2024-03-14T12:16:03.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.001+0000 7ffa331cd640 1 -- 172.21.15.82:0/696559349 wait complete. 2024-03-14T12:16:03.011 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:03.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.112+0000 7fd664468640 1 Processor -- start 2024-03-14T12:16:03.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.113+0000 7fd664468640 1 -- start start 2024-03-14T12:16:03.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.113+0000 7fd664468640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd65c0cf370 0x7fd65c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.113+0000 7fd664468640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0b7720 0x7fd65c0b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.113+0000 7fd664468640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0cf8c0 con 0x7fd65c0d0760 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.113+0000 7fd664468640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c0cfba0 con 0x7fd65c0cf370 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.113+0000 7fd664468640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0b6c40 con 0x7fd65c0b7720 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd662204640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd65c0d0760 0x7fd65c0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36348/0 (socket says 172.21.15.82:36348) 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd661a03640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd65c0cf370 0x7fd65c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd662204640 1 -- 172.21.15.82:0/799442454 learned_addr learned my addr 172.21.15.82:0/799442454 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd662a05640 1 --2- 172.21.15.82:0/799442454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0b7720 0x7fd65c0b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd661a03640 1 -- 172.21.15.82:0/799442454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0b7720 msgr2=0x7fd65c0b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd661a03640 1 --2- 172.21.15.82:0/799442454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0b7720 0x7fd65c0b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd661a03640 1 -- 172.21.15.82:0/799442454 >> v1:172.21.15.82:6789/0 conn(0x7fd65c0d0760 legacy=0x7fd65c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.114+0000 7fd661a03640 1 -- 172.21.15.82:0/799442454 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0b6f20 con 0x7fd65c0cf370 2024-03-14T12:16:03.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.115+0000 7fd661a03640 1 --2- 172.21.15.82:0/799442454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0cf370 0x7fd65c0d4660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097d0 tx=0x7fd65802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780f9d1231e5a23f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.115+0000 7fd661202640 1 -- 172.21.15.82:0/799442454 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658035020 con 0x7fd65c0cf370 2024-03-14T12:16:03.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.115+0000 7fd661202640 1 -- 172.21.15.82:0/799442454 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658031890 con 0x7fd65c0cf370 2024-03-14T12:16:03.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.115+0000 7fd661202640 1 -- 172.21.15.82:0/799442454 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658031c60 con 0x7fd65c0cf370 2024-03-14T12:16:03.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 -- 172.21.15.82:0/799442454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0cf370 msgr2=0x7fd65c0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 --2- 172.21.15.82:0/799442454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0cf370 0x7fd65c0d4660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097d0 tx=0x7fd65802b930 comp rx=0 tx=0).stop 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 -- 172.21.15.82:0/799442454 shutdown_connections 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 --2- 172.21.15.82:0/799442454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0b7720 0x7fd65c0b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 --2- 172.21.15.82:0/799442454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0cf370 0x7fd65c0d4660 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 -- 172.21.15.82:0/799442454 >> 172.21.15.82:0/799442454 conn(0x7fd65c09d030 msgr2=0x7fd65c0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 -- 172.21.15.82:0/799442454 shutdown_connections 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.116+0000 7fd664468640 1 -- 172.21.15.82:0/799442454 wait complete. 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.117+0000 7fd664468640 1 Processor -- start 2024-03-14T12:16:03.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.117+0000 7fd664468640 1 -- start start 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0b7720 0x7fd65c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd65c0cf370 0x7fd65c0cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0d0760 0x7fd65c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1617a0 con 0x7fd65c0b7720 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0d4ba0 con 0x7fd65c0cf370 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c161920 con 0x7fd65c0d0760 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0b7720 0x7fd65c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0b7720 0x7fd65c1540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37060/0 (socket says 172.21.15.82:37060) 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 -- 172.21.15.82:0/1116363765 learned_addr learned my addr 172.21.15.82:0/1116363765 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662a05640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0d0760 0x7fd65c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd661a03640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd65c0cf370 0x7fd65c0cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 -- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0d0760 msgr2=0x7fd65c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0d0760 0x7fd65c0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 -- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd65c0cf370 msgr2=0x7fd65c0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd65c0cf370 0x7fd65c0cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 -- 172.21.15.82:0/1116363765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0c8330 con 0x7fd65c0b7720 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662a05640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0d0760 0x7fd65c0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd661a03640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd65c0cf370 0x7fd65c0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd662204640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0b7720 0x7fd65c1540c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c3f0 tx=0x7fd64c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd652ffd640 1 -- 172.21.15.82:0/1116363765 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c017070 con 0x7fd65c0b7720 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd652ffd640 1 -- 172.21.15.82:0/1116363765 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c005ce0 con 0x7fd65c0b7720 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c157e50 con 0x7fd65c0b7720 2024-03-14T12:16:03.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd652ffd640 1 -- 172.21.15.82:0/1116363765 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0072f0 con 0x7fd65c0b7720 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.118+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c158410 con 0x7fd65c0b7720 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.119+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c04f050 con 0x7fd65c0b7720 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.120+0000 7fd652ffd640 1 -- 172.21.15.82:0/1116363765 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd64c01a070 con 0x7fd65c0b7720 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.120+0000 7fd652ffd640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd64006bbd0 0x7fd64006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.120+0000 7fd652ffd640 1 -- 172.21.15.82:0/1116363765 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd64c089e10 con 0x7fd65c0b7720 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.120+0000 7fd661a03640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd64006bbd0 0x7fd64006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.121+0000 7fd661a03640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd64006bbd0 0x7fd64006e090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd658036000 tx=0x7fd658008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.125+0000 7fd652ffd640 1 -- 172.21.15.82:0/1116363765 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd64c058900 con 0x7fd65c0b7720 2024-03-14T12:16:03.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.298+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd65c0ccee0 con 0x7fd65c0b7720 2024-03-14T12:16:03.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.299+0000 7fd652ffd640 1 -- 172.21.15.82:0/1116363765 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16614 (secure 0 0 0) 0x7fd64c058200 con 0x7fd65c0b7720 2024-03-14T12:16:03.298 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":45,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:02.016940+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd64006bbd0 msgr2=0x7fd64006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd64006bbd0 0x7fd64006e090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd658036000 tx=0x7fd658008030 comp rx=0 tx=0).stop 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0b7720 msgr2=0x7fd65c1540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0b7720 0x7fd65c1540c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c3f0 tx=0x7fd64c00c8b0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 shutdown_connections 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd64006bbd0 0x7fd64006e090 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd65c0d0760 0x7fd65c0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd65c0cf370 0x7fd65c0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 --2- 172.21.15.82:0/1116363765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd65c0b7720 0x7fd65c1540c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 >> 172.21.15.82:0/1116363765 conn(0x7fd65c09d030 msgr2=0x7fd65c15f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:03.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 shutdown_connections 2024-03-14T12:16:03.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.301+0000 7fd664468640 1 -- 172.21.15.82:0/1116363765 wait complete. 2024-03-14T12:16:03.312 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T12:16:03.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.416+0000 7f5197ae8640 1 Processor -- start 2024-03-14T12:16:03.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.416+0000 7f5197ae8640 1 -- start start 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5197ae8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51900d12a0 0x7f51900d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5197ae8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51900d25a0 con 0x7f51900d0c40 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5197ae8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51900d2880 con 0x7f519015dff0 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5197ae8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51900d2350 con 0x7f51900d12a0 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5195083640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51900d12a0 0x7f51900d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5195083640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51900d12a0 0x7f51900d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37076/0 (socket says 172.21.15.82:37076) 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5195083640 1 -- 172.21.15.82:0/617310855 learned_addr learned my addr 172.21.15.82:0/617310855 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.417+0000 7f5196085640 1 --1- 172.21.15.82:0/617310855 >> v1:172.21.15.82:6789/0 conn(0x7f51900d0c40 0x7f51900cb320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36358/0 (socket says 172.21.15.82:36358) 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.418+0000 7f5195083640 1 -- 172.21.15.82:0/617310855 >> v1:172.21.15.164:6789/0 conn(0x7f519015dff0 legacy=0x7f51900d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:03.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.418+0000 7f5195083640 1 -- 172.21.15.82:0/617310855 >> v1:172.21.15.82:6789/0 conn(0x7f51900d0c40 legacy=0x7f51900cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.418+0000 7f5194882640 1 -- 172.21.15.82:0/617310855 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221755111 0 0) 0x7f51900d25a0 con 0x7f51900d0c40 2024-03-14T12:16:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.418+0000 7f5195083640 1 -- 172.21.15.82:0/617310855 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900d2b60 con 0x7f51900d12a0 2024-03-14T12:16:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.418+0000 7f5195083640 1 --2- 172.21.15.82:0/617310855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51900d12a0 0x7f51900d0700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5180009870 tx=0x7f518002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb36792cfdc878f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5194882640 1 -- 172.21.15.82:0/617310855 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180034070 con 0x7f51900d12a0 2024-03-14T12:16:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5194882640 1 -- 172.21.15.82:0/617310855 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180030860 con 0x7f51900d12a0 2024-03-14T12:16:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5194882640 1 -- 172.21.15.82:0/617310855 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180030c30 con 0x7f51900d12a0 2024-03-14T12:16:03.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5197ae8640 1 -- 172.21.15.82:0/617310855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51900d12a0 msgr2=0x7f51900d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5197ae8640 1 --2- 172.21.15.82:0/617310855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51900d12a0 0x7f51900d0700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5180009870 tx=0x7f518002b9a0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5197ae8640 1 -- 172.21.15.82:0/617310855 shutdown_connections 2024-03-14T12:16:03.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5197ae8640 1 --2- 172.21.15.82:0/617310855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51900d12a0 0x7f51900d0700 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5197ae8640 1 -- 172.21.15.82:0/617310855 >> 172.21.15.82:0/617310855 conn(0x7f519009d0d0 msgr2=0x7f51900d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:03.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5197ae8640 1 -- 172.21.15.82:0/617310855 shutdown_connections 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.419+0000 7f5197ae8640 1 -- 172.21.15.82:0/617310855 wait complete. 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.420+0000 7f5197ae8640 1 Processor -- start 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.420+0000 7f5197ae8640 1 -- start start 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5197ae8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51900d0c40 0x7f51900b8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5197ae8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51900d12a0 0x7f519015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5197ae8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f519015dff0 0x7f51900c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5197ae8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51900cbbb0 con 0x7f519015dff0 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5197ae8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51900cba30 con 0x7f51900d12a0 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5197ae8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51900cbd30 con 0x7f51900d0c40 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f519015dff0 0x7f51900c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f519015dff0 0x7f51900c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37090/0 (socket says 172.21.15.82:37090) 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 -- 172.21.15.82:0/4194324417 learned_addr learned my addr 172.21.15.82:0/4194324417 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5195884640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51900d0c40 0x7f51900b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 -- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51900d0c40 msgr2=0x7f51900b8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5195083640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51900d12a0 0x7f519015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51900d0c40 0x7f51900b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 -- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51900d12a0 msgr2=0x7f519015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51900d12a0 0x7f519015a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.421+0000 7f5196085640 1 -- 172.21.15.82:0/4194324417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900b8e50 con 0x7f519015dff0 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.422+0000 7f5196085640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f519015dff0 0x7f51900c9b40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f518800c450 tx=0x7f518800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.422+0000 7f51867fc640 1 -- 172.21.15.82:0/4194324417 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188018070 con 0x7f519015dff0 2024-03-14T12:16:03.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.422+0000 7f51867fc640 1 -- 172.21.15.82:0/4194324417 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5188007500 con 0x7f519015dff0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.422+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51900b7e70 con 0x7f519015dff0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.422+0000 7f51867fc640 1 -- 172.21.15.82:0/4194324417 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188004a20 con 0x7f519015dff0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.422+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51900b8110 con 0x7f519015dff0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.423+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519004f050 con 0x7f519015dff0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.423+0000 7f51867fc640 1 -- 172.21.15.82:0/4194324417 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f518801b070 con 0x7f519015dff0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.424+0000 7f51867fc640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f517406bbd0 0x7f517406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.424+0000 7f51867fc640 1 -- 172.21.15.82:0/4194324417 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5188089c80 con 0x7f519015dff0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.424+0000 7f5195884640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f517406bbd0 0x7f517406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.424+0000 7f5195884640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f517406bbd0 0x7f517406e090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f519015c6a0 tx=0x7f518c007310 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.428+0000 7f51867fc640 1 -- 172.21.15.82:0/4194324417 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5188058770 con 0x7f519015dff0 2024-03-14T12:16:03.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.636+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f51900cf140 con 0x7f519015dff0 2024-03-14T12:16:03.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.677+0000 7f51867fc640 1 -- 172.21.15.82:0/4194324417 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5188058070 con 0x7f519015dff0 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f517406bbd0 msgr2=0x7f517406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f517406bbd0 0x7f517406e090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f519015c6a0 tx=0x7f518c007310 comp rx=0 tx=0).stop 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f519015dff0 msgr2=0x7f51900c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f519015dff0 0x7f51900c9b40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f518800c450 tx=0x7f518800c910 comp rx=0 tx=0).stop 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 shutdown_connections 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f519015dff0 0x7f51900c9b40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51900d12a0 0x7f519015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f517406bbd0 0x7f517406e090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 --2- 172.21.15.82:0/4194324417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51900d0c40 0x7f51900b8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 >> 172.21.15.82:0/4194324417 conn(0x7f519009d0d0 msgr2=0x7f51900cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:03.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 shutdown_connections 2024-03-14T12:16:03.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.679+0000 7f5197ae8640 1 -- 172.21.15.82:0/4194324417 wait complete. 2024-03-14T12:16:03.689 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T12:16:03.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T12:16:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.792+0000 7f7a9f948640 1 Processor -- start 2024-03-14T12:16:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.792+0000 7f7a9f948640 1 -- start start 2024-03-14T12:16:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9f948640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a9815dff0 0x7f7a980d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9f948640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a980d12a0 0x7f7a980d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9f948640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980d25a0 con 0x7f7a980d0c40 2024-03-14T12:16:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9f948640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a980d10a0 con 0x7f7a9815dff0 2024-03-14T12:16:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9f948640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980d2350 con 0x7f7a980d12a0 2024-03-14T12:16:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9d6e4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a9815dff0 0x7f7a980d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9d6e4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a9815dff0 0x7f7a980d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37106/0 (socket says 172.21.15.82:37106) 2024-03-14T12:16:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.793+0000 7f7a9d6e4640 1 -- 172.21.15.82:0/2941084371 learned_addr learned my addr 172.21.15.82:0/2941084371 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.794+0000 7f7a9d6e4640 1 -- 172.21.15.82:0/2941084371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a980d12a0 msgr2=0x7f7a980d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.794+0000 7f7a9d6e4640 1 --2- 172.21.15.82:0/2941084371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a980d12a0 0x7f7a980d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.794+0000 7f7a9d6e4640 1 -- 172.21.15.82:0/2941084371 >> v1:172.21.15.164:6789/0 conn(0x7f7a980d0c40 legacy=0x7f7a980ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.794+0000 7f7a9d6e4640 1 -- 172.21.15.82:0/2941084371 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980d2880 con 0x7f7a9815dff0 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.794+0000 7f7a9d6e4640 1 --2- 172.21.15.82:0/2941084371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a9815dff0 0x7f7a980d4480 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e50 tx=0x7f7a9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe2e114b0c5f327 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a8ffff640 1 -- 172.21.15.82:0/2941084371 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9402ca70 con 0x7f7a9815dff0 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a8ffff640 1 -- 172.21.15.82:0/2941084371 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9402cc10 con 0x7f7a9815dff0 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a8ffff640 1 -- 172.21.15.82:0/2941084371 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940343f0 con 0x7f7a9815dff0 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 -- 172.21.15.82:0/2941084371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a9815dff0 msgr2=0x7f7a980d4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 --2- 172.21.15.82:0/2941084371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a9815dff0 0x7f7a980d4480 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e50 tx=0x7f7a9402bed0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 -- 172.21.15.82:0/2941084371 shutdown_connections 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 --2- 172.21.15.82:0/2941084371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a980d12a0 0x7f7a980d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 --2- 172.21.15.82:0/2941084371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a9815dff0 0x7f7a980d4480 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 -- 172.21.15.82:0/2941084371 >> 172.21.15.82:0/2941084371 conn(0x7f7a9809d0d0 msgr2=0x7f7a980d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 -- 172.21.15.82:0/2941084371 shutdown_connections 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.795+0000 7f7a9f948640 1 -- 172.21.15.82:0/2941084371 wait complete. 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.796+0000 7f7a9f948640 1 Processor -- start 2024-03-14T12:16:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.796+0000 7f7a9f948640 1 -- start start 2024-03-14T12:16:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9f948640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a980d0c40 0x7f7a981f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9f948640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a980d12a0 0x7f7a981608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9f948640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a9815dff0 0x7f7a98160df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9f948640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980d4b40 con 0x7f7a980d12a0 2024-03-14T12:16:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9f948640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980d49c0 con 0x7f7a980d0c40 2024-03-14T12:16:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9f948640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9815e3d0 con 0x7f7a9815dff0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9cee3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a980d12a0 0x7f7a981608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9cee3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a980d12a0 0x7f7a981608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37120/0 (socket says 172.21.15.82:37120) 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9cee3640 1 -- 172.21.15.82:0/3618440740 learned_addr learned my addr 172.21.15.82:0/3618440740 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9d6e4640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a980d0c40 0x7f7a981f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.797+0000 7f7a9dee5640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a9815dff0 0x7f7a98160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9cee3640 1 -- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a9815dff0 msgr2=0x7f7a98160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9cee3640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a9815dff0 0x7f7a98160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9cee3640 1 -- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a980d0c40 msgr2=0x7f7a981f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9cee3640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a980d0c40 0x7f7a981f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9cee3640 1 -- 172.21.15.82:0/3618440740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980b9870 con 0x7f7a980d12a0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9cee3640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a980d12a0 0x7f7a981608b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800b410 tx=0x7f7a8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a8dffb640 1 -- 172.21.15.82:0/3618440740 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88015020 con 0x7f7a980d12a0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a8dffb640 1 -- 172.21.15.82:0/3618440740 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88011850 con 0x7f7a980d12a0 2024-03-14T12:16:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a8dffb640 1 -- 172.21.15.82:0/3618440740 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011c40 con 0x7f7a980d12a0 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a98143ea0 con 0x7f7a980d12a0 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.798+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a98144460 con 0x7f7a980d12a0 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.799+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a980b9c10 con 0x7f7a980d12a0 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.799+0000 7f7a8dffb640 1 -- 172.21.15.82:0/3618440740 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7a88018020 con 0x7f7a980d12a0 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.800+0000 7f7a8dffb640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a7c06bbd0 0x7f7a7c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.800+0000 7f7a8dffb640 1 -- 172.21.15.82:0/3618440740 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a8808a8f0 con 0x7f7a980d12a0 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.800+0000 7f7a9d6e4640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a7c06bbd0 0x7f7a7c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:03.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.800+0000 7f7a9d6e4640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a7c06bbd0 0x7f7a7c06e090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a94035000 tx=0x7f7a94030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:03.804+0000 7f7a8dffb640 1 -- 172.21.15.82:0/3618440740 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7a8801f020 con 0x7f7a980d12a0 2024-03-14T12:16:04.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.013+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a98144630 con 0x7f7a980d12a0 2024-03-14T12:16:04.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.678+0000 7f7a8dffb640 1 -- 172.21.15.82:0/3618440740 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a88058d00 con 0x7f7a980d12a0 2024-03-14T12:16:04.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a7c06bbd0 msgr2=0x7f7a7c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:04.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a7c06bbd0 0x7f7a7c06e090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a94035000 tx=0x7f7a94030000 comp rx=0 tx=0).stop 2024-03-14T12:16:04.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a980d12a0 msgr2=0x7f7a981608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:04.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a980d12a0 0x7f7a981608b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800b410 tx=0x7f7a8800b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 shutdown_connections 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a7c06bbd0 0x7f7a7c06e090 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a9815dff0 0x7f7a98160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a980d12a0 0x7f7a981608b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 --2- 172.21.15.82:0/3618440740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a980d0c40 0x7f7a981f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 >> 172.21.15.82:0/3618440740 conn(0x7f7a9809d0d0 msgr2=0x7f7a980ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 shutdown_connections 2024-03-14T12:16:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.681+0000 7f7a9f948640 1 -- 172.21.15.82:0/3618440740 wait complete. 2024-03-14T12:16:04.692 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T12:16:04.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:04.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.794+0000 7f00864a7640 1 Processor -- start 2024-03-14T12:16:04.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.794+0000 7f00864a7640 1 -- start start 2024-03-14T12:16:04.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.794+0000 7f00864a7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f008015f7a0 0x7f008015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:04.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f00864a7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00800ce210 0x7f00801451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:04.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f00864a7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00800ce620 con 0x7f00800cdd60 2024-03-14T12:16:04.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f00864a7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800cf150 con 0x7f00800ce210 2024-03-14T12:16:04.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f00864a7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00801600c0 con 0x7f008015f7a0 2024-03-14T12:16:04.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f0084a44640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00800ce210 0x7f00801451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:04.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f007f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f00800cdd60 0x7f0080144ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36362/0 (socket says 172.21.15.82:36362) 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f007f7fe640 1 -- 172.21.15.82:0/3822210630 learned_addr learned my addr 172.21.15.82:0/3822210630 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f0084a44640 1 -- 172.21.15.82:0/3822210630 >> v2:172.21.15.164:3300/0 conn(0x7f008015f7a0 msgr2=0x7f008015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f007ffff640 1 --2- 172.21.15.82:0/3822210630 >> v2:172.21.15.164:3300/0 conn(0x7f008015f7a0 0x7f008015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f0084a44640 1 --2- 172.21.15.82:0/3822210630 >> v2:172.21.15.164:3300/0 conn(0x7f008015f7a0 0x7f008015fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f0084a44640 1 -- 172.21.15.82:0/3822210630 >> v1:172.21.15.82:6789/0 conn(0x7f00800cdd60 legacy=0x7f0080144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f0084a44640 1 -- 172.21.15.82:0/3822210630 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800ce8b0 con 0x7f00800ce210 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.795+0000 7f007effd640 1 -- 172.21.15.82:0/3822210630 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192175818 0 0) 0x7f00800ce620 con 0x7f00800cdd60 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f0084a44640 1 --2- 172.21.15.82:0/3822210630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00801451d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00800b4e00 tx=0x7f007802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3d142634fd48f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f007effd640 1 -- 172.21.15.82:0/3822210630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078039070 con 0x7f00800ce210 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f007effd640 1 -- 172.21.15.82:0/3822210630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007802cbf0 con 0x7f00800ce210 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f007effd640 1 -- 172.21.15.82:0/3822210630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078034440 con 0x7f00800ce210 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 -- 172.21.15.82:0/3822210630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 msgr2=0x7f00801451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:04.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 --2- 172.21.15.82:0/3822210630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00801451d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00800b4e00 tx=0x7f007802be40 comp rx=0 tx=0).stop 2024-03-14T12:16:04.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 -- 172.21.15.82:0/3822210630 shutdown_connections 2024-03-14T12:16:04.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 --2- 172.21.15.82:0/3822210630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00801451d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 --2- 172.21.15.82:0/3822210630 >> v2:172.21.15.164:3300/0 conn(0x7f008015f7a0 0x7f008015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 -- 172.21.15.82:0/3822210630 >> 172.21.15.82:0/3822210630 conn(0x7f00800b1c70 msgr2=0x7f00800cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:04.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 -- 172.21.15.82:0/3822210630 shutdown_connections 2024-03-14T12:16:04.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.796+0000 7f00864a7640 1 -- 172.21.15.82:0/3822210630 wait complete. 2024-03-14T12:16:04.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 Processor -- start 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 -- start start 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cdd60 0x7f0080147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00800b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f008015f7a0 0x7f008006d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008006c4e0 con 0x7f00800ce210 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800cee70 con 0x7f00800cdd60 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f00864a7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800cc4f0 con 0x7f008015f7a0 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.798+0000 7f007f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00800b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00800b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37148/0 (socket says 172.21.15.82:37148) 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 -- 172.21.15.82:0/2768522345 learned_addr learned my addr 172.21.15.82:0/2768522345 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007ffff640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cdd60 0x7f0080147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f0084a44640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f008015f7a0 0x7f008006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:04.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 -- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f008015f7a0 msgr2=0x7f008006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f008015f7a0 0x7f008006d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 -- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cdd60 msgr2=0x7f0080147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cdd60 0x7f0080147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 -- 172.21.15.82:0/2768522345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00801f7bd0 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007f7fe640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00800b3d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f007400c430 tx=0x7f007400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007cff9640 1 -- 172.21.15.82:0/2768522345 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00740074e0 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007cff9640 1 -- 172.21.15.82:0/2768522345 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00740047e0 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f007cff9640 1 -- 172.21.15.82:0/2768522345 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074004bd0 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00800b2c50 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.799+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00800b3210 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.800+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00800b3410 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.801+0000 7f007cff9640 1 -- 172.21.15.82:0/2768522345 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0074011070 con 0x7f00800ce210 2024-03-14T12:16:04.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.801+0000 7f007cff9640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006c06bc00 0x7f006c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:04.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.801+0000 7f007cff9640 1 -- 172.21.15.82:0/2768522345 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f007408a5a0 con 0x7f00800ce210 2024-03-14T12:16:04.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.801+0000 7f007ffff640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006c06bc00 0x7f006c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:04.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.802+0000 7f007ffff640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006c06bc00 0x7f006c06e0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f007000aa20 tx=0x7f0070008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:04.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:04.806+0000 7f007cff9640 1 -- 172.21.15.82:0/2768522345 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f00800b3410 con 0x7f00800ce210 2024-03-14T12:16:05.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.011+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f008004f050 con 0x7f00800ce210 2024-03-14T12:16:05.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.012+0000 7f007cff9640 1 -- 172.21.15.82:0/2768522345 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f0074059120 con 0x7f00800ce210 2024-03-14T12:16:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1414844632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":1414844632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1414844632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/456693586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":456693586},{"type":"v1","addr":"172.21.15.164:6834","nonce":456693586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/58260684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":58260684},{"type":"v1","addr":"172.21.15.82:6836","nonce":58260684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:02.017303+0000","modified":"2024-03-14T12:16:04.014283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2708572159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2708572159},{"type":"v1","addr":"172.21.15.82:6837","nonce":2708572159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T12:16:05.011 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 21 2024-03-14T12:16:05.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.013+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006c06bc00 msgr2=0x7f006c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.013+0000 7f00864a7640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006c06bc00 0x7f006c06e0c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f007000aa20 tx=0x7f0070008040 comp rx=0 tx=0).stop 2024-03-14T12:16:05.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.013+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 msgr2=0x7f00800b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.013+0000 7f00864a7640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00800b3d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f007400c430 tx=0x7f007400c8f0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 shutdown_connections 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006c06bc00 0x7f006c06e0c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f008015f7a0 0x7f008006d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800ce210 0x7f00800b3d90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 --2- 172.21.15.82:0/2768522345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cdd60 0x7f0080147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 >> 172.21.15.82:0/2768522345 conn(0x7f00800b1c70 msgr2=0x7f00801534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 shutdown_connections 2024-03-14T12:16:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.014+0000 7f00864a7640 1 -- 172.21.15.82:0/2768522345 wait complete. 2024-03-14T12:16:05.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:05.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.124+0000 7f9c22c20640 1 Processor -- start 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.124+0000 7f9c22c20640 1 -- start start 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c22c20640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9c1c071740 0x7f9c1c071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c22c20640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c1c15e670 0x7f9c1c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c22c20640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c072060 con 0x7f9c1c143f60 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c22c20640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c15ebe0 con 0x7f9c1c15e670 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c22c20640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c15eec0 con 0x7f9c1c071740 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c1bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c1c15e670 0x7f9c1c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c211bd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9c1c143f60 0x7f9c1c15b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36372/0 (socket says 172.21.15.82:36372) 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c1bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c1c15e670 0x7f9c1c0b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37158/0 (socket says 172.21.15.82:37158) 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c211bd640 1 -- 172.21.15.82:0/1156240984 learned_addr learned my addr 172.21.15.82:0/1156240984 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c209bc640 1 --2- 172.21.15.82:0/1156240984 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c071740 0x7f9c1c071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c1bfff640 1 -- 172.21.15.82:0/1156240984 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c071740 msgr2=0x7f9c1c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/1156240984 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c071740 0x7f9c1c071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c1bfff640 1 -- 172.21.15.82:0/1156240984 >> v1:172.21.15.82:6789/0 conn(0x7f9c1c143f60 legacy=0x7f9c1c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c1bfff640 1 -- 172.21.15.82:0/1156240984 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c15f1a0 con 0x7f9c1c15e670 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.125+0000 7f9c1b7fe640 1 -- 172.21.15.82:0/1156240984 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165379188 0 0) 0x7f9c1c072060 con 0x7f9c1c143f60 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.126+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/1156240984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c15e670 0x7f9c1c0b2940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0d04e0 tx=0x7f9c1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbcb56f1565e56e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.126+0000 7f9c1b7fe640 1 -- 172.21.15.82:0/1156240984 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10035020 con 0x7f9c1c15e670 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.126+0000 7f9c1b7fe640 1 -- 172.21.15.82:0/1156240984 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10031830 con 0x7f9c1c15e670 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.126+0000 7f9c1b7fe640 1 -- 172.21.15.82:0/1156240984 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10031c00 con 0x7f9c1c15e670 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 -- 172.21.15.82:0/1156240984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c15e670 msgr2=0x7f9c1c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 --2- 172.21.15.82:0/1156240984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c15e670 0x7f9c1c0b2940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0d04e0 tx=0x7f9c1002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 -- 172.21.15.82:0/1156240984 shutdown_connections 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 --2- 172.21.15.82:0/1156240984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c15e670 0x7f9c1c0b2940 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 --2- 172.21.15.82:0/1156240984 >> v2:172.21.15.164:3300/0 conn(0x7f9c1c071740 0x7f9c1c071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 -- 172.21.15.82:0/1156240984 >> 172.21.15.82:0/1156240984 conn(0x7f9c1c0b1c70 msgr2=0x7f9c1c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:05.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 -- 172.21.15.82:0/1156240984 shutdown_connections 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.127+0000 7f9c22c20640 1 -- 172.21.15.82:0/1156240984 wait complete. 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.128+0000 7f9c22c20640 1 Processor -- start 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.128+0000 7f9c22c20640 1 -- start start 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c22c20640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c071740 0x7f9c1c13d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c22c20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c143f60 0x7f9c1c157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c22c20640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15e670 0x7f9c1c1f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c22c20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c15b080 con 0x7f9c1c143f60 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c22c20640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c143ce0 con 0x7f9c1c071740 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c22c20640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c144670 con 0x7f9c1c15e670 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c143f60 0x7f9c1c157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c143f60 0x7f9c1c157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37172/0 (socket says 172.21.15.82:37172) 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 -- 172.21.15.82:0/2015979735 learned_addr learned my addr 172.21.15.82:0/2015979735 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c209bc640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c071740 0x7f9c1c13d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c211bd640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15e670 0x7f9c1c1f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 -- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15e670 msgr2=0x7f9c1c1f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15e670 0x7f9c1c1f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 -- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c071740 msgr2=0x7f9c1c13d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c071740 0x7f9c1c13d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 -- 172.21.15.82:0/2015979735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c13a7d0 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c209bc640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c071740 0x7f9c1c13d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c1bfff640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c143f60 0x7f9c1c157f40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009890 tx=0x7f9c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.129+0000 7f9c197fa640 1 -- 172.21.15.82:0/2015979735 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10041070 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.130+0000 7f9c197fa640 1 -- 172.21.15.82:0/2015979735 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1003a950 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.130+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c1f4ab0 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.130+0000 7f9c197fa640 1 -- 172.21.15.82:0/2015979735 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10035040 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.130+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c1f5070 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.131+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1c13a9a0 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.131+0000 7f9c197fa640 1 -- 172.21.15.82:0/2015979735 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9c10038020 con 0x7f9c1c143f60 2024-03-14T12:16:05.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.131+0000 7f9c197fa640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0806bbd0 0x7f9c0806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.131+0000 7f9c209bc640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0806bbd0 0x7f9c0806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.131+0000 7f9c197fa640 1 -- 172.21.15.82:0/2015979735 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c100ac4a0 con 0x7f9c1c143f60 2024-03-14T12:16:05.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.132+0000 7f9c209bc640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0806bbd0 0x7f9c0806e090 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c006fd0 tx=0x7f9c0c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.136+0000 7f9c197fa640 1 -- 172.21.15.82:0/2015979735 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9c1007af90 con 0x7f9c1c143f60 2024-03-14T12:16:05.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.309+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c1c04f050 con 0x7f9c1c143f60 2024-03-14T12:16:05.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.310+0000 7f9c197fa640 1 -- 172.21.15.82:0/2015979735 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f9c1007a890 con 0x7f9c1c143f60 2024-03-14T12:16:05.309 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:05.309 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:03.671135+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:05.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0806bbd0 msgr2=0x7f9c0806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0806bbd0 0x7f9c0806e090 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c006fd0 tx=0x7f9c0c008040 comp rx=0 tx=0).stop 2024-03-14T12:16:05.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c143f60 msgr2=0x7f9c1c157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c143f60 0x7f9c1c157f40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009890 tx=0x7f9c100046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 shutdown_connections 2024-03-14T12:16:05.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c0806bbd0 0x7f9c0806e090 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c1c15e670 0x7f9c1c1f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1c143f60 0x7f9c1c157f40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 --2- 172.21.15.82:0/2015979735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c1c071740 0x7f9c1c13d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 >> 172.21.15.82:0/2015979735 conn(0x7f9c1c0b1c70 msgr2=0x7f9c1c071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:05.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 shutdown_connections 2024-03-14T12:16:05.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.312+0000 7f9c22c20640 1 -- 172.21.15.82:0/2015979735 wait complete. 2024-03-14T12:16:05.322 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T12:16:05.323 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:16:05.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.423+0000 7f8fa71f1640 1 Processor -- start 2024-03-14T12:16:05.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.423+0000 7f8fa71f1640 1 -- start start 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa71f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006b670 0x7f8fa0072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa71f1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8fa006eb20 0x7f8fa006efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa71f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa00727a0 con 0x7f8fa00b0200 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa71f1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa0071810 con 0x7f8fa006eb20 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa71f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa0071dd0 con 0x7f8fa006b670 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa578e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8fa006eb20 0x7f8fa006efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa578e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8fa006eb20 0x7f8fa006efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37176/0 (socket says 172.21.15.82:37176) 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa578e640 1 -- 172.21.15.82:0/1490039927 learned_addr learned my addr 172.21.15.82:0/1490039927 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa4f8d640 1 --2- 172.21.15.82:0/1490039927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006b670 0x7f8fa0072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa578e640 1 -- 172.21.15.82:0/1490039927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006b670 msgr2=0x7f8fa0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa578e640 1 --2- 172.21.15.82:0/1490039927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006b670 0x7f8fa0072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa578e640 1 -- 172.21.15.82:0/1490039927 >> v1:172.21.15.164:6789/0 conn(0x7f8fa00b0200 legacy=0x7f8fa00ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:05.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.424+0000 7f8fa578e640 1 -- 172.21.15.82:0/1490039927 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa00d0410 con 0x7f8fa006eb20 2024-03-14T12:16:05.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.425+0000 7f8fa578e640 1 --2- 172.21.15.82:0/1490039927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa006eb20 0x7f8fa006efd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e50 tx=0x7f8f9802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabc622ea0b5df72 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.425+0000 7f8f9f7fe640 1 -- 172.21.15.82:0/1490039927 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802cb00 con 0x7f8fa006eb20 2024-03-14T12:16:05.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.425+0000 7f8f9f7fe640 1 -- 172.21.15.82:0/1490039927 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9802cca0 con 0x7f8fa006eb20 2024-03-14T12:16:05.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.425+0000 7f8f9f7fe640 1 -- 172.21.15.82:0/1490039927 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98034610 con 0x7f8fa006eb20 2024-03-14T12:16:05.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1490039927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa006eb20 msgr2=0x7f8fa006efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1490039927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa006eb20 0x7f8fa006efd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e50 tx=0x7f8f9802bf60 comp rx=0 tx=0).stop 2024-03-14T12:16:05.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1490039927 shutdown_connections 2024-03-14T12:16:05.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1490039927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa006eb20 0x7f8fa006efd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1490039927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006b670 0x7f8fa0072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1490039927 >> 172.21.15.82:0/1490039927 conn(0x7f8fa00d0b60 msgr2=0x7f8fa00d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:05.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1490039927 shutdown_connections 2024-03-14T12:16:05.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.426+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1490039927 wait complete. 2024-03-14T12:16:05.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.427+0000 7f8fa71f1640 1 Processor -- start 2024-03-14T12:16:05.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.427+0000 7f8fa71f1640 1 -- start start 2024-03-14T12:16:05.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa71f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa006b670 0x7f8fa00b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa71f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006eb20 0x7f8fa00c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa71f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa00b0200 0x7f8fa015dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa71f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa00d0100 con 0x7f8fa00b0200 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa71f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa0071a60 con 0x7f8fa006b670 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa71f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa00cb1b0 con 0x7f8fa006eb20 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa00b0200 0x7f8fa015dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa00b0200 0x7f8fa015dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37188/0 (socket says 172.21.15.82:37188) 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 -- 172.21.15.82:0/1745131652 learned_addr learned my addr 172.21.15.82:0/1745131652 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa4f8d640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa006b670 0x7f8fa00b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8f9ffff640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006eb20 0x7f8fa00c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 -- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006eb20 msgr2=0x7f8fa00c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006eb20 0x7f8fa00c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 -- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa006b670 msgr2=0x7f8fa00b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa006b670 0x7f8fa00b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.428+0000 7f8fa578e640 1 -- 172.21.15.82:0/1745131652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa00c98e0 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.429+0000 7f8fa578e640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa00b0200 0x7f8fa015dda0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e50 tx=0x7f8f98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.429+0000 7f8f9d7fa640 1 -- 172.21.15.82:0/1745131652 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98043020 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.429+0000 7f8f9d7fa640 1 -- 172.21.15.82:0/1745131652 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9802ce50 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.429+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa015e2e0 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.429+0000 7f8f9d7fa640 1 -- 172.21.15.82:0/1745131652 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98009510 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.429+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa00b3fe0 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.430+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa00725c0 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.430+0000 7f8f9d7fa640 1 -- 172.21.15.82:0/1745131652 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8f98046020 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.430+0000 7f8f9d7fa640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8f8c06bbd0 0x7f8f8c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.430+0000 7f8f9d7fa640 1 -- 172.21.15.82:0/1745131652 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f980acf70 con 0x7f8fa00b0200 2024-03-14T12:16:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.431+0000 7f8fa4f8d640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8f8c06bbd0 0x7f8f8c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.431+0000 7f8fa4f8d640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8f8c06bbd0 0x7f8f8c06e090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8f90006fd0 tx=0x7f8f90008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.435+0000 7f8f9d7fa640 1 -- 172.21.15.82:0/1745131652 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8fa00725c0 con 0x7f8fa00b0200 2024-03-14T12:16:05.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.603+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fa004f050 con 0x7f8f8c06bbd0 2024-03-14T12:16:05.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.609+0000 7f8f9d7fa640 1 -- 172.21.15.82:0/1745131652 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251892 (secure 0 0 0) 0x7f8fa004f050 con 0x7f8f8c06bbd0 2024-03-14T12:16:05.611 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:05.612 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:16:05.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8f8c06bbd0 msgr2=0x7f8f8c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8f8c06bbd0 0x7f8f8c06e090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8f90006fd0 tx=0x7f8f90008040 comp rx=0 tx=0).stop 2024-03-14T12:16:05.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa00b0200 msgr2=0x7f8fa015dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa00b0200 0x7f8fa015dda0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e50 tx=0x7f8f98004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:05.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 shutdown_connections 2024-03-14T12:16:05.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa00b0200 0x7f8fa015dda0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8f8c06bbd0 0x7f8f8c06e090 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fa006eb20 0x7f8fa00c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 --2- 172.21.15.82:0/1745131652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa006b670 0x7f8fa00b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 >> 172.21.15.82:0/1745131652 conn(0x7f8fa00d0b60 msgr2=0x7f8fa00b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:05.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 shutdown_connections 2024-03-14T12:16:05.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.612+0000 7f8fa71f1640 1 -- 172.21.15.82:0/1745131652 wait complete. 2024-03-14T12:16:05.622 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-14T12:16:05.525935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254804,"kb_used_data":5332,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719916,"statfs":{"total":773094113280,"available":772833193984,"internally_reserved":0,"allocated":5459968,"data_stored":2881445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.870015"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690295+0000","last_change":"2024-03-14T12:16:02.687478+0000","last_active":"2024-03-14T12:16:02.690295+0000","last_peered":"2024-03-14T12:16:02.690295+0000","last_clean":"2024-03-14T12:16:02.690295+0000","last_became_active":"2024-03-14T12:16:02.687105+0000","last_became_peered":"2024-03-14T12:16:02.687105+0000","last_unstale":"2024-03-14T12:16:02.690295+0000","last_undegraded":"2024-03-14T12:16:02.690295+0000","last_fullsized":"2024-03-14T12:16:02.690295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:48:55.741445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689736+0000","last_change":"2024-03-14T12:16:02.685283+0000","last_active":"2024-03-14T12:16:02.689736+0000","last_peered":"2024-03-14T12:16:02.689736+0000","last_clean":"2024-03-14T12:16:02.689736+0000","last_became_active":"2024-03-14T12:16:02.684970+0000","last_became_peered":"2024-03-14T12:16:02.684970+0000","last_unstale":"2024-03-14T12:16:02.689736+0000","last_undegraded":"2024-03-14T12:16:02.689736+0000","last_fullsized":"2024-03-14T12:16:02.689736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:03.381028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.679795+0000","last_change":"2024-03-14T12:16:02.673620+0000","last_active":"2024-03-14T12:16:02.679795+0000","last_peered":"2024-03-14T12:16:02.679795+0000","last_clean":"2024-03-14T12:16:02.679795+0000","last_became_active":"2024-03-14T12:16:02.673351+0000","last_became_peered":"2024-03-14T12:16:02.673351+0000","last_unstale":"2024-03-14T12:16:02.679795+0000","last_undegraded":"2024-03-14T12:16:02.679795+0000","last_fullsized":"2024-03-14T12:16:02.679795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:16.788654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697386+0000","last_change":"2024-03-14T12:16:02.689867+0000","last_active":"2024-03-14T12:16:02.697386+0000","last_peered":"2024-03-14T12:16:02.697386+0000","last_clean":"2024-03-14T12:16:02.697386+0000","last_became_active":"2024-03-14T12:16:02.689587+0000","last_became_peered":"2024-03-14T12:16:02.689587+0000","last_unstale":"2024-03-14T12:16:02.697386+0000","last_undegraded":"2024-03-14T12:16:02.697386+0000","last_fullsized":"2024-03-14T12:16:02.697386+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:16.788654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690337+0000","last_change":"2024-03-14T12:16:02.685455+0000","last_active":"2024-03-14T12:16:02.690337+0000","last_peered":"2024-03-14T12:16:02.690337+0000","last_clean":"2024-03-14T12:16:02.690337+0000","last_became_active":"2024-03-14T12:16:02.685064+0000","last_became_peered":"2024-03-14T12:16:02.685064+0000","last_unstale":"2024-03-14T12:16:02.690337+0000","last_undegraded":"2024-03-14T12:16:02.690337+0000","last_fullsized":"2024-03-14T12:16:02.690337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:24:50.624619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692372+0000","last_change":"2024-03-14T12:16:02.688403+0000","last_active":"2024-03-14T12:16:02.692372+0000","last_peered":"2024-03-14T12:16:02.692372+0000","last_clean":"2024-03-14T12:16:02.692372+0000","last_became_active":"2024-03-14T12:16:02.688114+0000","last_became_peered":"2024-03-14T12:16:02.688114+0000","last_unstale":"2024-03-14T12:16:02.692372+0000","last_undegraded":"2024-03-14T12:16:02.692372+0000","last_fullsized":"2024-03-14T12:16:02.692372+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:42:04.491643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690302+0000","last_change":"2024-03-14T12:16:02.686997+0000","last_active":"2024-03-14T12:16:02.690302+0000","last_peered":"2024-03-14T12:16:02.690302+0000","last_clean":"2024-03-14T12:16:02.690302+0000","last_became_active":"2024-03-14T12:16:02.686719+0000","last_became_peered":"2024-03-14T12:16:02.686719+0000","last_unstale":"2024-03-14T12:16:02.690302+0000","last_undegraded":"2024-03-14T12:16:02.690302+0000","last_fullsized":"2024-03-14T12:16:02.690302+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:29:00.413372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697767+0000","last_change":"2024-03-14T12:16:02.691682+0000","last_active":"2024-03-14T12:16:02.697767+0000","last_peered":"2024-03-14T12:16:02.697767+0000","last_clean":"2024-03-14T12:16:02.697767+0000","last_became_active":"2024-03-14T12:16:02.690387+0000","last_became_peered":"2024-03-14T12:16:02.690387+0000","last_unstale":"2024-03-14T12:16:02.697767+0000","last_undegraded":"2024-03-14T12:16:02.697767+0000","last_fullsized":"2024-03-14T12:16:02.697767+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:57.883134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697712+0000","last_change":"2024-03-14T12:16:02.690991+0000","last_active":"2024-03-14T12:16:02.697712+0000","last_peered":"2024-03-14T12:16:02.697712+0000","last_clean":"2024-03-14T12:16:02.697712+0000","last_became_active":"2024-03-14T12:16:02.690698+0000","last_became_peered":"2024-03-14T12:16:02.690698+0000","last_unstale":"2024-03-14T12:16:02.697712+0000","last_undegraded":"2024-03-14T12:16:02.697712+0000","last_fullsized":"2024-03-14T12:16:02.697712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:16.518796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697673+0000","last_change":"2024-03-14T12:16:02.690526+0000","last_active":"2024-03-14T12:16:02.697673+0000","last_peered":"2024-03-14T12:16:02.697673+0000","last_clean":"2024-03-14T12:16:02.697673+0000","last_became_active":"2024-03-14T12:16:02.690257+0000","last_became_peered":"2024-03-14T12:16:02.690257+0000","last_unstale":"2024-03-14T12:16:02.697673+0000","last_undegraded":"2024-03-14T12:16:02.697673+0000","last_fullsized":"2024-03-14T12:16:02.697673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:02:40.058933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697816+0000","last_change":"2024-03-14T12:16:02.691726+0000","last_active":"2024-03-14T12:16:02.697816+0000","last_peered":"2024-03-14T12:16:02.697816+0000","last_clean":"2024-03-14T12:16:02.697816+0000","last_became_active":"2024-03-14T12:16:02.691424+0000","last_became_peered":"2024-03-14T12:16:02.691424+0000","last_unstale":"2024-03-14T12:16:02.697816+0000","last_undegraded":"2024-03-14T12:16:02.697816+0000","last_fullsized":"2024-03-14T12:16:02.697816+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:55.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.679030+0000","last_change":"2024-03-14T12:16:02.675450+0000","last_active":"2024-03-14T12:16:02.679030+0000","last_peered":"2024-03-14T12:16:02.679030+0000","last_clean":"2024-03-14T12:16:02.679030+0000","last_became_active":"2024-03-14T12:16:02.674821+0000","last_became_peered":"2024-03-14T12:16:02.674821+0000","last_unstale":"2024-03-14T12:16:02.679030+0000","last_undegraded":"2024-03-14T12:16:02.679030+0000","last_fullsized":"2024-03-14T12:16:02.679030+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:11.055206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691209+0000","last_change":"2024-03-14T12:16:02.686240+0000","last_active":"2024-03-14T12:16:02.691209+0000","last_peered":"2024-03-14T12:16:02.691209+0000","last_clean":"2024-03-14T12:16:02.691209+0000","last_became_active":"2024-03-14T12:16:02.685838+0000","last_became_peered":"2024-03-14T12:16:02.685838+0000","last_unstale":"2024-03-14T12:16:02.691209+0000","last_undegraded":"2024-03-14T12:16:02.691209+0000","last_fullsized":"2024-03-14T12:16:02.691209+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:33:07.491250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697230+0000","last_change":"2024-03-14T12:16:02.689535+0000","last_active":"2024-03-14T12:16:02.697230+0000","last_peered":"2024-03-14T12:16:02.697230+0000","last_clean":"2024-03-14T12:16:02.697230+0000","last_became_active":"2024-03-14T12:16:02.689236+0000","last_became_peered":"2024-03-14T12:16:02.689236+0000","last_unstale":"2024-03-14T12:16:02.697230+0000","last_undegraded":"2024-03-14T12:16:02.697230+0000","last_fullsized":"2024-03-14T12:16:02.697230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:01.220239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.684647+0000","last_change":"2024-03-14T12:16:02.677831+0000","last_active":"2024-03-14T12:16:02.684647+0000","last_peered":"2024-03-14T12:16:02.684647+0000","last_clean":"2024-03-14T12:16:02.684647+0000","last_became_active":"2024-03-14T12:16:02.677537+0000","last_became_peered":"2024-03-14T12:16:02.677537+0000","last_unstale":"2024-03-14T12:16:02.684647+0000","last_undegraded":"2024-03-14T12:16:02.684647+0000","last_fullsized":"2024-03-14T12:16:02.684647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:18:31.470897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692957+0000","last_change":"2024-03-14T12:16:02.692289+0000","last_active":"2024-03-14T12:16:02.692957+0000","last_peered":"2024-03-14T12:16:02.692957+0000","last_clean":"2024-03-14T12:16:02.692957+0000","last_became_active":"2024-03-14T12:16:02.692026+0000","last_became_peered":"2024-03-14T12:16:02.692026+0000","last_unstale":"2024-03-14T12:16:02.692957+0000","last_undegraded":"2024-03-14T12:16:02.692957+0000","last_fullsized":"2024-03-14T12:16:02.692957+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:00:09.135192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.678230+0000","last_change":"2024-03-14T12:16:02.675157+0000","last_active":"2024-03-14T12:16:02.678230+0000","last_peered":"2024-03-14T12:16:02.678230+0000","last_clean":"2024-03-14T12:16:02.678230+0000","last_became_active":"2024-03-14T12:16:02.674727+0000","last_became_peered":"2024-03-14T12:16:02.674727+0000","last_unstale":"2024-03-14T12:16:02.678230+0000","last_undegraded":"2024-03-14T12:16:02.678230+0000","last_fullsized":"2024-03-14T12:16:02.678230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:48.005671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686330+0000","last_change":"2024-03-14T12:16:02.682867+0000","last_active":"2024-03-14T12:16:02.686330+0000","last_peered":"2024-03-14T12:16:02.686330+0000","last_clean":"2024-03-14T12:16:02.686330+0000","last_became_active":"2024-03-14T12:16:02.682594+0000","last_became_peered":"2024-03-14T12:16:02.682594+0000","last_unstale":"2024-03-14T12:16:02.686330+0000","last_undegraded":"2024-03-14T12:16:02.686330+0000","last_fullsized":"2024-03-14T12:16:02.686330+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:57.883134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692950+0000","last_change":"2024-03-14T12:16:02.691583+0000","last_active":"2024-03-14T12:16:02.692950+0000","last_peered":"2024-03-14T12:16:02.692950+0000","last_clean":"2024-03-14T12:16:02.692950+0000","last_became_active":"2024-03-14T12:16:02.691264+0000","last_became_peered":"2024-03-14T12:16:02.691264+0000","last_unstale":"2024-03-14T12:16:02.692950+0000","last_undegraded":"2024-03-14T12:16:02.692950+0000","last_fullsized":"2024-03-14T12:16:02.692950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:38:27.730273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690093+0000","last_change":"2024-03-14T12:16:02.683097+0000","last_active":"2024-03-14T12:16:02.690093+0000","last_peered":"2024-03-14T12:16:02.690093+0000","last_clean":"2024-03-14T12:16:02.690093+0000","last_became_active":"2024-03-14T12:16:02.682740+0000","last_became_peered":"2024-03-14T12:16:02.682740+0000","last_unstale":"2024-03-14T12:16:02.690093+0000","last_undegraded":"2024-03-14T12:16:02.690093+0000","last_fullsized":"2024-03-14T12:16:02.690093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:11.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692429+0000","last_change":"2024-03-14T12:16:02.685667+0000","last_active":"2024-03-14T12:16:02.692429+0000","last_peered":"2024-03-14T12:16:02.692429+0000","last_clean":"2024-03-14T12:16:02.692429+0000","last_became_active":"2024-03-14T12:16:02.685403+0000","last_became_peered":"2024-03-14T12:16:02.685403+0000","last_unstale":"2024-03-14T12:16:02.692429+0000","last_undegraded":"2024-03-14T12:16:02.692429+0000","last_fullsized":"2024-03-14T12:16:02.692429+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:40:57.545081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692193+0000","last_change":"2024-03-14T12:16:02.688847+0000","last_active":"2024-03-14T12:16:02.692193+0000","last_peered":"2024-03-14T12:16:02.692193+0000","last_clean":"2024-03-14T12:16:02.692193+0000","last_became_active":"2024-03-14T12:16:02.688537+0000","last_became_peered":"2024-03-14T12:16:02.688537+0000","last_unstale":"2024-03-14T12:16:02.692193+0000","last_undegraded":"2024-03-14T12:16:02.692193+0000","last_fullsized":"2024-03-14T12:16:02.692193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:20:43.368288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691169+0000","last_change":"2024-03-14T12:16:02.688575+0000","last_active":"2024-03-14T12:16:02.691169+0000","last_peered":"2024-03-14T12:16:02.691169+0000","last_clean":"2024-03-14T12:16:02.691169+0000","last_became_active":"2024-03-14T12:16:02.688228+0000","last_became_peered":"2024-03-14T12:16:02.688228+0000","last_unstale":"2024-03-14T12:16:02.691169+0000","last_undegraded":"2024-03-14T12:16:02.691169+0000","last_fullsized":"2024-03-14T12:16:02.691169+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:58:58.976190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691138+0000","last_change":"2024-03-14T12:16:02.689451+0000","last_active":"2024-03-14T12:16:02.691138+0000","last_peered":"2024-03-14T12:16:02.691138+0000","last_clean":"2024-03-14T12:16:02.691138+0000","last_became_active":"2024-03-14T12:16:02.688861+0000","last_became_peered":"2024-03-14T12:16:02.688861+0000","last_unstale":"2024-03-14T12:16:02.691138+0000","last_undegraded":"2024-03-14T12:16:02.691138+0000","last_fullsized":"2024-03-14T12:16:02.691138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:58:58.380420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689690+0000","last_change":"2024-03-14T12:16:02.685536+0000","last_active":"2024-03-14T12:16:02.689690+0000","last_peered":"2024-03-14T12:16:02.689690+0000","last_clean":"2024-03-14T12:16:02.689690+0000","last_became_active":"2024-03-14T12:16:02.685243+0000","last_became_peered":"2024-03-14T12:16:02.685243+0000","last_unstale":"2024-03-14T12:16:02.689690+0000","last_undegraded":"2024-03-14T12:16:02.689690+0000","last_fullsized":"2024-03-14T12:16:02.689690+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:33:07.491250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691969+0000","last_change":"2024-03-14T12:16:02.686580+0000","last_active":"2024-03-14T12:16:02.691969+0000","last_peered":"2024-03-14T12:16:02.691969+0000","last_clean":"2024-03-14T12:16:02.691969+0000","last_became_active":"2024-03-14T12:16:02.686317+0000","last_became_peered":"2024-03-14T12:16:02.686317+0000","last_unstale":"2024-03-14T12:16:02.691969+0000","last_undegraded":"2024-03-14T12:16:02.691969+0000","last_fullsized":"2024-03-14T12:16:02.691969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:21:54.822695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.683676+0000","last_change":"2024-03-14T12:16:02.679930+0000","last_active":"2024-03-14T12:16:02.683676+0000","last_peered":"2024-03-14T12:16:02.683676+0000","last_clean":"2024-03-14T12:16:02.683676+0000","last_became_active":"2024-03-14T12:16:02.679577+0000","last_became_peered":"2024-03-14T12:16:02.679577+0000","last_unstale":"2024-03-14T12:16:02.683676+0000","last_undegraded":"2024-03-14T12:16:02.683676+0000","last_fullsized":"2024-03-14T12:16:02.683676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:46.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.678988+0000","last_change":"2024-03-14T12:16:02.674561+0000","last_active":"2024-03-14T12:16:02.678988+0000","last_peered":"2024-03-14T12:16:02.678988+0000","last_clean":"2024-03-14T12:16:02.678988+0000","last_became_active":"2024-03-14T12:16:02.674271+0000","last_became_peered":"2024-03-14T12:16:02.674271+0000","last_unstale":"2024-03-14T12:16:02.678988+0000","last_undegraded":"2024-03-14T12:16:02.678988+0000","last_fullsized":"2024-03-14T12:16:02.678988+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:24:57.823630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690382+0000","last_change":"2024-03-14T12:16:02.684628+0000","last_active":"2024-03-14T12:16:02.690382+0000","last_peered":"2024-03-14T12:16:02.690382+0000","last_clean":"2024-03-14T12:16:02.690382+0000","last_became_active":"2024-03-14T12:16:02.684357+0000","last_became_peered":"2024-03-14T12:16:02.684357+0000","last_unstale":"2024-03-14T12:16:02.690382+0000","last_undegraded":"2024-03-14T12:16:02.690382+0000","last_fullsized":"2024-03-14T12:16:02.690382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:11.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689229+0000","last_change":"2024-03-14T12:16:02.683867+0000","last_active":"2024-03-14T12:16:02.689229+0000","last_peered":"2024-03-14T12:16:02.689229+0000","last_clean":"2024-03-14T12:16:02.689229+0000","last_became_active":"2024-03-14T12:16:02.683600+0000","last_became_peered":"2024-03-14T12:16:02.683600+0000","last_unstale":"2024-03-14T12:16:02.689229+0000","last_undegraded":"2024-03-14T12:16:02.689229+0000","last_fullsized":"2024-03-14T12:16:02.689229+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:55.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.684671+0000","last_change":"2024-03-14T12:16:02.680002+0000","last_active":"2024-03-14T12:16:02.684671+0000","last_peered":"2024-03-14T12:16:02.684671+0000","last_clean":"2024-03-14T12:16:02.684671+0000","last_became_active":"2024-03-14T12:16:02.679751+0000","last_became_peered":"2024-03-14T12:16:02.679751+0000","last_unstale":"2024-03-14T12:16:02.684671+0000","last_undegraded":"2024-03-14T12:16:02.684671+0000","last_fullsized":"2024-03-14T12:16:02.684671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:52:14.858524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689833+0000","last_change":"2024-03-14T12:16:02.672616+0000","last_active":"2024-03-14T12:16:02.689833+0000","last_peered":"2024-03-14T12:16:02.689833+0000","last_clean":"2024-03-14T12:16:02.689833+0000","last_became_active":"2024-03-14T12:16:02.672329+0000","last_became_peered":"2024-03-14T12:16:02.672329+0000","last_unstale":"2024-03-14T12:16:02.689833+0000","last_undegraded":"2024-03-14T12:16:02.689833+0000","last_fullsized":"2024-03-14T12:16:02.689833+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:09:39.347025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.687520+0000","last_change":"2024-03-14T12:16:02.685662+0000","last_active":"2024-03-14T12:16:02.687520+0000","last_peered":"2024-03-14T12:16:02.687520+0000","last_clean":"2024-03-14T12:16:02.687520+0000","last_became_active":"2024-03-14T12:16:02.685300+0000","last_became_peered":"2024-03-14T12:16:02.685300+0000","last_unstale":"2024-03-14T12:16:02.687520+0000","last_undegraded":"2024-03-14T12:16:02.687520+0000","last_fullsized":"2024-03-14T12:16:02.687520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:46.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698095+0000","last_change":"2024-03-14T12:16:01.677792+0000","last_active":"2024-03-14T12:16:02.698095+0000","last_peered":"2024-03-14T12:16:02.698095+0000","last_clean":"2024-03-14T12:16:02.698095+0000","last_became_active":"2024-03-14T12:16:01.677479+0000","last_became_peered":"2024-03-14T12:16:01.677479+0000","last_unstale":"2024-03-14T12:16:02.698095+0000","last_undegraded":"2024-03-14T12:16:02.698095+0000","last_fullsized":"2024-03-14T12:16:02.698095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:48:54.732476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.687635+0000","last_change":"2024-03-14T12:16:02.680862+0000","last_active":"2024-03-14T12:16:02.687635+0000","last_peered":"2024-03-14T12:16:02.687635+0000","last_clean":"2024-03-14T12:16:02.687635+0000","last_became_active":"2024-03-14T12:16:02.680586+0000","last_became_peered":"2024-03-14T12:16:02.680586+0000","last_unstale":"2024-03-14T12:16:02.687635+0000","last_undegraded":"2024-03-14T12:16:02.687635+0000","last_fullsized":"2024-03-14T12:16:02.687635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:56:08.364261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691704+0000","last_change":"2024-03-14T12:16:01.678475+0000","last_active":"2024-03-14T12:16:02.691704+0000","last_peered":"2024-03-14T12:16:02.691704+0000","last_clean":"2024-03-14T12:16:02.691704+0000","last_became_active":"2024-03-14T12:16:01.678193+0000","last_became_peered":"2024-03-14T12:16:01.678193+0000","last_unstale":"2024-03-14T12:16:02.691704+0000","last_undegraded":"2024-03-14T12:16:02.691704+0000","last_fullsized":"2024-03-14T12:16:02.691704+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:09:49.869686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690650+0000","last_change":"2024-03-14T12:16:02.688572+0000","last_active":"2024-03-14T12:16:02.690650+0000","last_peered":"2024-03-14T12:16:02.690650+0000","last_clean":"2024-03-14T12:16:02.690650+0000","last_became_active":"2024-03-14T12:16:02.688290+0000","last_became_peered":"2024-03-14T12:16:02.688290+0000","last_unstale":"2024-03-14T12:16:02.690650+0000","last_undegraded":"2024-03-14T12:16:02.690650+0000","last_fullsized":"2024-03-14T12:16:02.690650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:29:00.413372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690350+0000","last_change":"2024-03-14T12:16:01.677698+0000","last_active":"2024-03-14T12:16:02.690350+0000","last_peered":"2024-03-14T12:16:02.690350+0000","last_clean":"2024-03-14T12:16:02.690350+0000","last_became_active":"2024-03-14T12:16:01.677420+0000","last_became_peered":"2024-03-14T12:16:01.677420+0000","last_unstale":"2024-03-14T12:16:02.690350+0000","last_undegraded":"2024-03-14T12:16:02.690350+0000","last_fullsized":"2024-03-14T12:16:02.690350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.211270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.687872+0000","last_change":"2024-03-14T12:16:02.684325+0000","last_active":"2024-03-14T12:16:02.687872+0000","last_peered":"2024-03-14T12:16:02.687872+0000","last_clean":"2024-03-14T12:16:02.687872+0000","last_became_active":"2024-03-14T12:16:02.683852+0000","last_became_peered":"2024-03-14T12:16:02.683852+0000","last_unstale":"2024-03-14T12:16:02.687872+0000","last_undegraded":"2024-03-14T12:16:02.687872+0000","last_fullsized":"2024-03-14T12:16:02.687872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:48.005671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690966+0000","last_change":"2024-03-14T12:16:01.681606+0000","last_active":"2024-03-14T12:16:02.690966+0000","last_peered":"2024-03-14T12:16:02.690966+0000","last_clean":"2024-03-14T12:16:02.690966+0000","last_became_active":"2024-03-14T12:16:01.680984+0000","last_became_peered":"2024-03-14T12:16:01.680984+0000","last_unstale":"2024-03-14T12:16:02.690966+0000","last_undegraded":"2024-03-14T12:16:02.690966+0000","last_fullsized":"2024-03-14T12:16:02.690966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:10.046237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686364+0000","last_change":"2024-03-14T12:16:02.681432+0000","last_active":"2024-03-14T12:16:02.686364+0000","last_peered":"2024-03-14T12:16:02.686364+0000","last_clean":"2024-03-14T12:16:02.686364+0000","last_became_active":"2024-03-14T12:16:02.681144+0000","last_became_peered":"2024-03-14T12:16:02.681144+0000","last_unstale":"2024-03-14T12:16:02.686364+0000","last_undegraded":"2024-03-14T12:16:02.686364+0000","last_fullsized":"2024-03-14T12:16:02.686364+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:02:40.058933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697558+0000","last_change":"2024-03-14T12:16:01.680027+0000","last_active":"2024-03-14T12:16:02.697558+0000","last_peered":"2024-03-14T12:16:02.697558+0000","last_clean":"2024-03-14T12:16:02.697558+0000","last_became_active":"2024-03-14T12:16:01.678891+0000","last_became_peered":"2024-03-14T12:16:01.678891+0000","last_unstale":"2024-03-14T12:16:02.697558+0000","last_undegraded":"2024-03-14T12:16:02.697558+0000","last_fullsized":"2024-03-14T12:16:02.697558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:33.141787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699181+0000","last_change":"2024-03-14T12:16:02.694449+0000","last_active":"2024-03-14T12:16:02.699181+0000","last_peered":"2024-03-14T12:16:02.699181+0000","last_clean":"2024-03-14T12:16:02.699181+0000","last_became_active":"2024-03-14T12:16:02.693746+0000","last_became_peered":"2024-03-14T12:16:02.693746+0000","last_unstale":"2024-03-14T12:16:02.699181+0000","last_undegraded":"2024-03-14T12:16:02.699181+0000","last_fullsized":"2024-03-14T12:16:02.699181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:24:57.823630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690014+0000","last_change":"2024-03-14T12:16:01.676852+0000","last_active":"2024-03-14T12:16:02.690014+0000","last_peered":"2024-03-14T12:16:02.690014+0000","last_clean":"2024-03-14T12:16:02.690014+0000","last_became_active":"2024-03-14T12:16:01.676563+0000","last_became_peered":"2024-03-14T12:16:01.676563+0000","last_unstale":"2024-03-14T12:16:02.690014+0000","last_undegraded":"2024-03-14T12:16:02.690014+0000","last_fullsized":"2024-03-14T12:16:02.690014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:20:42.359319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697743+0000","last_change":"2024-03-14T12:16:02.690356+0000","last_active":"2024-03-14T12:16:02.697743+0000","last_peered":"2024-03-14T12:16:02.697743+0000","last_clean":"2024-03-14T12:16:02.697743+0000","last_became_active":"2024-03-14T12:16:02.684412+0000","last_became_peered":"2024-03-14T12:16:02.684412+0000","last_unstale":"2024-03-14T12:16:02.697743+0000","last_undegraded":"2024-03-14T12:16:02.697743+0000","last_fullsized":"2024-03-14T12:16:02.697743+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:46.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.666180+0000","last_change":"2024-03-14T12:16:01.675440+0000","last_active":"2024-03-14T12:16:02.666180+0000","last_peered":"2024-03-14T12:16:02.666180+0000","last_clean":"2024-03-14T12:16:02.666180+0000","last_became_active":"2024-03-14T12:16:01.675159+0000","last_became_peered":"2024-03-14T12:16:01.675159+0000","last_unstale":"2024-03-14T12:16:02.666180+0000","last_undegraded":"2024-03-14T12:16:02.666180+0000","last_fullsized":"2024-03-14T12:16:02.666180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:15.509827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692045+0000","last_change":"2024-03-14T12:16:02.690719+0000","last_active":"2024-03-14T12:16:02.692045+0000","last_peered":"2024-03-14T12:16:02.692045+0000","last_clean":"2024-03-14T12:16:02.692045+0000","last_became_active":"2024-03-14T12:16:02.690026+0000","last_became_peered":"2024-03-14T12:16:02.690026+0000","last_unstale":"2024-03-14T12:16:02.692045+0000","last_undegraded":"2024-03-14T12:16:02.692045+0000","last_fullsized":"2024-03-14T12:16:02.692045+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:29.572333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.675998+0000","last_change":"2024-03-14T12:16:01.679198+0000","last_active":"2024-03-14T12:16:02.675998+0000","last_peered":"2024-03-14T12:16:02.675998+0000","last_clean":"2024-03-14T12:16:02.675998+0000","last_became_active":"2024-03-14T12:16:01.678929+0000","last_became_peered":"2024-03-14T12:16:01.678929+0000","last_unstale":"2024-03-14T12:16:02.675998+0000","last_undegraded":"2024-03-14T12:16:02.675998+0000","last_fullsized":"2024-03-14T12:16:02.675998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:38:26.721304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686308+0000","last_change":"2024-03-14T12:16:02.682289+0000","last_active":"2024-03-14T12:16:02.686308+0000","last_peered":"2024-03-14T12:16:02.686308+0000","last_clean":"2024-03-14T12:16:02.686308+0000","last_became_active":"2024-03-14T12:16:02.681982+0000","last_became_peered":"2024-03-14T12:16:02.681982+0000","last_unstale":"2024-03-14T12:16:02.686308+0000","last_undegraded":"2024-03-14T12:16:02.686308+0000","last_fullsized":"2024-03-14T12:16:02.686308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:16.518796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.667553+0000","last_change":"2024-03-14T12:16:01.676109+0000","last_active":"2024-03-14T12:16:02.667553+0000","last_peered":"2024-03-14T12:16:02.667553+0000","last_clean":"2024-03-14T12:16:02.667553+0000","last_became_active":"2024-03-14T12:16:01.675821+0000","last_became_peered":"2024-03-14T12:16:01.675821+0000","last_unstale":"2024-03-14T12:16:02.667553+0000","last_undegraded":"2024-03-14T12:16:02.667553+0000","last_fullsized":"2024-03-14T12:16:02.667553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.211270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690327+0000","last_change":"2024-03-14T12:16:02.687697+0000","last_active":"2024-03-14T12:16:02.690327+0000","last_peered":"2024-03-14T12:16:02.690327+0000","last_clean":"2024-03-14T12:16:02.690327+0000","last_became_active":"2024-03-14T12:16:02.687393+0000","last_became_peered":"2024-03-14T12:16:02.687393+0000","last_unstale":"2024-03-14T12:16:02.690327+0000","last_undegraded":"2024-03-14T12:16:02.690327+0000","last_fullsized":"2024-03-14T12:16:02.690327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:28.315595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676744+0000","last_change":"2024-03-14T12:16:01.679923+0000","last_active":"2024-03-14T12:16:02.676744+0000","last_peered":"2024-03-14T12:16:02.676744+0000","last_clean":"2024-03-14T12:16:02.676744+0000","last_became_active":"2024-03-14T12:16:01.679288+0000","last_became_peered":"2024-03-14T12:16:01.679288+0000","last_unstale":"2024-03-14T12:16:02.676744+0000","last_undegraded":"2024-03-14T12:16:02.676744+0000","last_fullsized":"2024-03-14T12:16:02.676744+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:40:04.426917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698146+0000","last_change":"2024-03-14T12:16:02.696554+0000","last_active":"2024-03-14T12:16:02.698146+0000","last_peered":"2024-03-14T12:16:02.698146+0000","last_clean":"2024-03-14T12:16:02.698146+0000","last_became_active":"2024-03-14T12:16:02.695815+0000","last_became_peered":"2024-03-14T12:16:02.695815+0000","last_unstale":"2024-03-14T12:16:02.698146+0000","last_undegraded":"2024-03-14T12:16:02.698146+0000","last_fullsized":"2024-03-14T12:16:02.698146+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:16:50.910057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691434+0000","last_change":"2024-03-14T12:16:01.682600+0000","last_active":"2024-03-14T12:16:02.691434+0000","last_peered":"2024-03-14T12:16:02.691434+0000","last_clean":"2024-03-14T12:16:02.691434+0000","last_became_active":"2024-03-14T12:16:01.682325+0000","last_became_peered":"2024-03-14T12:16:01.682325+0000","last_unstale":"2024-03-14T12:16:02.691434+0000","last_undegraded":"2024-03-14T12:16:02.691434+0000","last_fullsized":"2024-03-14T12:16:02.691434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:56.874165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690348+0000","last_change":"2024-03-14T12:16:02.686201+0000","last_active":"2024-03-14T12:16:02.690348+0000","last_peered":"2024-03-14T12:16:02.690348+0000","last_clean":"2024-03-14T12:16:02.690348+0000","last_became_active":"2024-03-14T12:16:02.685907+0000","last_became_peered":"2024-03-14T12:16:02.685907+0000","last_unstale":"2024-03-14T12:16:02.690348+0000","last_undegraded":"2024-03-14T12:16:02.690348+0000","last_fullsized":"2024-03-14T12:16:02.690348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:24:50.624619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699820+0000","last_change":"2024-03-14T12:16:01.671671+0000","last_active":"2024-03-14T12:16:02.699820+0000","last_peered":"2024-03-14T12:16:02.699820+0000","last_clean":"2024-03-14T12:16:02.699820+0000","last_became_active":"2024-03-14T12:16:01.671378+0000","last_became_peered":"2024-03-14T12:16:01.671378+0000","last_unstale":"2024-03-14T12:16:02.699820+0000","last_undegraded":"2024-03-14T12:16:02.699820+0000","last_fullsized":"2024-03-14T12:16:02.699820+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:53.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.695190+0000","last_change":"2024-03-14T12:16:02.694497+0000","last_active":"2024-03-14T12:16:02.695190+0000","last_peered":"2024-03-14T12:16:02.695190+0000","last_clean":"2024-03-14T12:16:02.695190+0000","last_became_active":"2024-03-14T12:16:02.694113+0000","last_became_peered":"2024-03-14T12:16:02.694113+0000","last_unstale":"2024-03-14T12:16:02.695190+0000","last_undegraded":"2024-03-14T12:16:02.695190+0000","last_fullsized":"2024-03-14T12:16:02.695190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:09:50.878654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.700254+0000","last_change":"2024-03-14T12:16:01.684242+0000","last_active":"2024-03-14T12:16:02.700254+0000","last_peered":"2024-03-14T12:16:02.700254+0000","last_clean":"2024-03-14T12:16:02.700254+0000","last_became_active":"2024-03-14T12:16:01.683942+0000","last_became_peered":"2024-03-14T12:16:01.683942+0000","last_unstale":"2024-03-14T12:16:02.700254+0000","last_undegraded":"2024-03-14T12:16:02.700254+0000","last_fullsized":"2024-03-14T12:16:02.700254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:14:59.279221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699150+0000","last_change":"2024-03-14T12:16:02.692982+0000","last_active":"2024-03-14T12:16:02.699150+0000","last_peered":"2024-03-14T12:16:02.699150+0000","last_clean":"2024-03-14T12:16:02.699150+0000","last_became_active":"2024-03-14T12:16:02.692527+0000","last_became_peered":"2024-03-14T12:16:02.692527+0000","last_unstale":"2024-03-14T12:16:02.699150+0000","last_undegraded":"2024-03-14T12:16:02.699150+0000","last_fullsized":"2024-03-14T12:16:02.699150+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:49.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.679034+0000","last_change":"2024-03-14T12:16:01.682074+0000","last_active":"2024-03-14T12:16:02.679034+0000","last_peered":"2024-03-14T12:16:02.679034+0000","last_clean":"2024-03-14T12:16:02.679034+0000","last_became_active":"2024-03-14T12:16:01.681549+0000","last_became_peered":"2024-03-14T12:16:01.681549+0000","last_unstale":"2024-03-14T12:16:02.679034+0000","last_undegraded":"2024-03-14T12:16:02.679034+0000","last_fullsized":"2024-03-14T12:16:02.679034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:02.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698097+0000","last_change":"2024-03-14T12:16:02.696718+0000","last_active":"2024-03-14T12:16:02.698097+0000","last_peered":"2024-03-14T12:16:02.698097+0000","last_clean":"2024-03-14T12:16:02.698097+0000","last_became_active":"2024-03-14T12:16:02.695787+0000","last_became_peered":"2024-03-14T12:16:02.695787+0000","last_unstale":"2024-03-14T12:16:02.698097+0000","last_undegraded":"2024-03-14T12:16:02.698097+0000","last_fullsized":"2024-03-14T12:16:02.698097+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:50:45.793287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.673973+0000","last_change":"2024-03-14T12:16:01.683671+0000","last_active":"2024-03-14T12:16:02.673973+0000","last_peered":"2024-03-14T12:16:02.673973+0000","last_clean":"2024-03-14T12:16:02.673973+0000","last_became_active":"2024-03-14T12:16:01.683368+0000","last_became_peered":"2024-03-14T12:16:01.683368+0000","last_unstale":"2024-03-14T12:16:02.673973+0000","last_undegraded":"2024-03-14T12:16:02.673973+0000","last_fullsized":"2024-03-14T12:16:02.673973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:09:06.112616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690883+0000","last_change":"2024-03-14T12:16:02.689528+0000","last_active":"2024-03-14T12:16:02.690883+0000","last_peered":"2024-03-14T12:16:02.690883+0000","last_clean":"2024-03-14T12:16:02.690883+0000","last_became_active":"2024-03-14T12:16:02.689179+0000","last_became_peered":"2024-03-14T12:16:02.689179+0000","last_unstale":"2024-03-14T12:16:02.690883+0000","last_undegraded":"2024-03-14T12:16:02.690883+0000","last_fullsized":"2024-03-14T12:16:02.690883+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:09:07.121584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.694824+0000","last_change":"2024-03-14T12:16:01.678880+0000","last_active":"2024-03-14T12:16:02.694824+0000","last_peered":"2024-03-14T12:16:02.694824+0000","last_clean":"2024-03-14T12:16:02.694824+0000","last_became_active":"2024-03-14T12:16:01.678559+0000","last_became_peered":"2024-03-14T12:16:01.678559+0000","last_unstale":"2024-03-14T12:16:02.694824+0000","last_undegraded":"2024-03-14T12:16:02.694824+0000","last_fullsized":"2024-03-14T12:16:02.694824+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:00:08.126224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691944+0000","last_change":"2024-03-14T12:16:02.676009+0000","last_active":"2024-03-14T12:16:02.691944+0000","last_peered":"2024-03-14T12:16:02.691944+0000","last_clean":"2024-03-14T12:16:02.691944+0000","last_became_active":"2024-03-14T12:16:02.675728+0000","last_became_peered":"2024-03-14T12:16:02.675728+0000","last_unstale":"2024-03-14T12:16:02.691944+0000","last_undegraded":"2024-03-14T12:16:02.691944+0000","last_fullsized":"2024-03-14T12:16:02.691944+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:56:08.364261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.696164+0000","last_change":"2024-03-14T12:16:01.680513+0000","last_active":"2024-03-14T12:16:02.696164+0000","last_peered":"2024-03-14T12:16:02.696164+0000","last_clean":"2024-03-14T12:16:02.696164+0000","last_became_active":"2024-03-14T12:16:01.680252+0000","last_became_peered":"2024-03-14T12:16:01.680252+0000","last_unstale":"2024-03-14T12:16:02.696164+0000","last_undegraded":"2024-03-14T12:16:02.696164+0000","last_fullsized":"2024-03-14T12:16:02.696164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:50.005267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699208+0000","last_change":"2024-03-14T12:16:02.696683+0000","last_active":"2024-03-14T12:16:02.699208+0000","last_peered":"2024-03-14T12:16:02.699208+0000","last_clean":"2024-03-14T12:16:02.699208+0000","last_became_active":"2024-03-14T12:16:02.696420+0000","last_became_peered":"2024-03-14T12:16:02.696420+0000","last_unstale":"2024-03-14T12:16:02.699208+0000","last_undegraded":"2024-03-14T12:16:02.699208+0000","last_fullsized":"2024-03-14T12:16:02.699208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:11.055206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689172+0000","last_change":"2024-03-14T12:16:01.679648+0000","last_active":"2024-03-14T12:16:02.689172+0000","last_peered":"2024-03-14T12:16:02.689172+0000","last_clean":"2024-03-14T12:16:02.689172+0000","last_became_active":"2024-03-14T12:16:01.678650+0000","last_became_peered":"2024-03-14T12:16:01.678650+0000","last_unstale":"2024-03-14T12:16:02.689172+0000","last_undegraded":"2024-03-14T12:16:02.689172+0000","last_fullsized":"2024-03-14T12:16:02.689172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:40:04.426917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689732+0000","last_change":"2024-03-14T12:16:02.686565+0000","last_active":"2024-03-14T12:16:02.689732+0000","last_peered":"2024-03-14T12:16:02.689732+0000","last_clean":"2024-03-14T12:16:02.689732+0000","last_became_active":"2024-03-14T12:16:02.686310+0000","last_became_peered":"2024-03-14T12:16:02.686310+0000","last_unstale":"2024-03-14T12:16:02.689732+0000","last_undegraded":"2024-03-14T12:16:02.689732+0000","last_fullsized":"2024-03-14T12:16:02.689732+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:49:43.398827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.688992+0000","last_change":"2024-03-14T12:16:01.676759+0000","last_active":"2024-03-14T12:16:02.688992+0000","last_peered":"2024-03-14T12:16:02.688992+0000","last_clean":"2024-03-14T12:16:02.688992+0000","last_became_active":"2024-03-14T12:16:01.676132+0000","last_became_peered":"2024-03-14T12:16:01.676132+0000","last_unstale":"2024-03-14T12:16:02.688992+0000","last_undegraded":"2024-03-14T12:16:02.688992+0000","last_fullsized":"2024-03-14T12:16:02.688992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:56.874165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690000+0000","last_change":"2024-03-14T12:16:02.686243+0000","last_active":"2024-03-14T12:16:02.690000+0000","last_peered":"2024-03-14T12:16:02.690000+0000","last_clean":"2024-03-14T12:16:02.690000+0000","last_became_active":"2024-03-14T12:16:02.685867+0000","last_became_peered":"2024-03-14T12:16:02.685867+0000","last_unstale":"2024-03-14T12:16:02.690000+0000","last_undegraded":"2024-03-14T12:16:02.690000+0000","last_fullsized":"2024-03-14T12:16:02.690000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:21:54.822695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.673847+0000","last_change":"2024-03-14T12:16:01.682781+0000","last_active":"2024-03-14T12:16:02.673847+0000","last_peered":"2024-03-14T12:16:02.673847+0000","last_clean":"2024-03-14T12:16:02.673847+0000","last_became_active":"2024-03-14T12:16:01.682445+0000","last_became_peered":"2024-03-14T12:16:01.682445+0000","last_unstale":"2024-03-14T12:16:02.673847+0000","last_undegraded":"2024-03-14T12:16:02.673847+0000","last_fullsized":"2024-03-14T12:16:02.673847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:27.306626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697997+0000","last_change":"2024-03-14T12:16:02.696086+0000","last_active":"2024-03-14T12:16:02.697997+0000","last_peered":"2024-03-14T12:16:02.697997+0000","last_clean":"2024-03-14T12:16:02.697997+0000","last_became_active":"2024-03-14T12:16:02.695755+0000","last_became_peered":"2024-03-14T12:16:02.695755+0000","last_unstale":"2024-03-14T12:16:02.697997+0000","last_undegraded":"2024-03-14T12:16:02.697997+0000","last_fullsized":"2024-03-14T12:16:02.697997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:46:04.081089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697644+0000","last_change":"2024-03-14T12:16:01.678470+0000","last_active":"2024-03-14T12:16:02.697644+0000","last_peered":"2024-03-14T12:16:02.697644+0000","last_clean":"2024-03-14T12:16:02.697644+0000","last_became_active":"2024-03-14T12:16:01.678183+0000","last_became_peered":"2024-03-14T12:16:01.678183+0000","last_unstale":"2024-03-14T12:16:02.697644+0000","last_undegraded":"2024-03-14T12:16:02.697644+0000","last_fullsized":"2024-03-14T12:16:02.697644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:34:45.933291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.683656+0000","last_change":"2024-03-14T12:16:02.680131+0000","last_active":"2024-03-14T12:16:02.683656+0000","last_peered":"2024-03-14T12:16:02.683656+0000","last_clean":"2024-03-14T12:16:02.683656+0000","last_became_active":"2024-03-14T12:16:02.679858+0000","last_became_peered":"2024-03-14T12:16:02.679858+0000","last_unstale":"2024-03-14T12:16:02.683656+0000","last_undegraded":"2024-03-14T12:16:02.683656+0000","last_fullsized":"2024-03-14T12:16:02.683656+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:16.788654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.666384+0000","last_change":"2024-03-14T12:16:01.680638+0000","last_active":"2024-03-14T12:16:02.666384+0000","last_peered":"2024-03-14T12:16:02.666384+0000","last_clean":"2024-03-14T12:16:02.666384+0000","last_became_active":"2024-03-14T12:16:01.680340+0000","last_became_peered":"2024-03-14T12:16:01.680340+0000","last_unstale":"2024-03-14T12:16:02.666384+0000","last_undegraded":"2024-03-14T12:16:02.666384+0000","last_fullsized":"2024-03-14T12:16:02.666384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:02:09.736541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691713+0000","last_change":"2024-03-14T12:16:02.689233+0000","last_active":"2024-03-14T12:16:02.691713+0000","last_peered":"2024-03-14T12:16:02.691713+0000","last_clean":"2024-03-14T12:16:02.691713+0000","last_became_active":"2024-03-14T12:16:02.688927+0000","last_became_peered":"2024-03-14T12:16:02.688927+0000","last_unstale":"2024-03-14T12:16:02.691713+0000","last_undegraded":"2024-03-14T12:16:02.691713+0000","last_fullsized":"2024-03-14T12:16:02.691713+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:49:43.398827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.675521+0000","last_change":"2024-03-14T12:16:01.679687+0000","last_active":"2024-03-14T12:16:02.675521+0000","last_peered":"2024-03-14T12:16:02.675521+0000","last_clean":"2024-03-14T12:16:02.675521+0000","last_became_active":"2024-03-14T12:16:01.679231+0000","last_became_peered":"2024-03-14T12:16:01.679231+0000","last_unstale":"2024-03-14T12:16:02.675521+0000","last_undegraded":"2024-03-14T12:16:02.675521+0000","last_fullsized":"2024-03-14T12:16:02.675521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:28.563364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697617+0000","last_change":"2024-03-14T12:16:02.690300+0000","last_active":"2024-03-14T12:16:02.697617+0000","last_peered":"2024-03-14T12:16:02.697617+0000","last_clean":"2024-03-14T12:16:02.697617+0000","last_became_active":"2024-03-14T12:16:02.690002+0000","last_became_peered":"2024-03-14T12:16:02.690002+0000","last_unstale":"2024-03-14T12:16:02.697617+0000","last_undegraded":"2024-03-14T12:16:02.697617+0000","last_fullsized":"2024-03-14T12:16:02.697617+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:48.005671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676142+0000","last_change":"2024-03-14T12:16:01.679089+0000","last_active":"2024-03-14T12:16:02.676142+0000","last_peered":"2024-03-14T12:16:02.676142+0000","last_clean":"2024-03-14T12:16:02.676142+0000","last_became_active":"2024-03-14T12:16:01.678794+0000","last_became_peered":"2024-03-14T12:16:01.678794+0000","last_unstale":"2024-03-14T12:16:02.676142+0000","last_undegraded":"2024-03-14T12:16:02.676142+0000","last_fullsized":"2024-03-14T12:16:02.676142+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:48.054785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.695206+0000","last_change":"2024-03-14T12:16:02.694501+0000","last_active":"2024-03-14T12:16:02.695206+0000","last_peered":"2024-03-14T12:16:02.695206+0000","last_clean":"2024-03-14T12:16:02.695206+0000","last_became_active":"2024-03-14T12:16:02.694193+0000","last_became_peered":"2024-03-14T12:16:02.694193+0000","last_unstale":"2024-03-14T12:16:02.695206+0000","last_undegraded":"2024-03-14T12:16:02.695206+0000","last_fullsized":"2024-03-14T12:16:02.695206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:40:05.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.674170+0000","last_change":"2024-03-14T12:16:01.681204+0000","last_active":"2024-03-14T12:16:02.674170+0000","last_peered":"2024-03-14T12:16:02.674170+0000","last_clean":"2024-03-14T12:16:02.674170+0000","last_became_active":"2024-03-14T12:16:01.680728+0000","last_became_peered":"2024-03-14T12:16:01.680728+0000","last_unstale":"2024-03-14T12:16:02.674170+0000","last_undegraded":"2024-03-14T12:16:02.674170+0000","last_fullsized":"2024-03-14T12:16:02.674170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:01:04.190553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692027+0000","last_change":"2024-03-14T12:16:02.681391+0000","last_active":"2024-03-14T12:16:02.692027+0000","last_peered":"2024-03-14T12:16:02.692027+0000","last_clean":"2024-03-14T12:16:02.692027+0000","last_became_active":"2024-03-14T12:16:02.681128+0000","last_became_peered":"2024-03-14T12:16:02.681128+0000","last_unstale":"2024-03-14T12:16:02.692027+0000","last_undegraded":"2024-03-14T12:16:02.692027+0000","last_fullsized":"2024-03-14T12:16:02.692027+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:48.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.675890+0000","last_change":"2024-03-14T12:16:01.683639+0000","last_active":"2024-03-14T12:16:02.675890+0000","last_peered":"2024-03-14T12:16:02.675890+0000","last_clean":"2024-03-14T12:16:02.675890+0000","last_became_active":"2024-03-14T12:16:01.683285+0000","last_became_peered":"2024-03-14T12:16:01.683285+0000","last_unstale":"2024-03-14T12:16:02.675890+0000","last_undegraded":"2024-03-14T12:16:02.675890+0000","last_fullsized":"2024-03-14T12:16:02.675890+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:54.293322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692977+0000","last_change":"2024-03-14T12:16:02.692090+0000","last_active":"2024-03-14T12:16:02.692977+0000","last_peered":"2024-03-14T12:16:02.692977+0000","last_clean":"2024-03-14T12:16:02.692977+0000","last_became_active":"2024-03-14T12:16:02.691667+0000","last_became_peered":"2024-03-14T12:16:02.691667+0000","last_unstale":"2024-03-14T12:16:02.692977+0000","last_undegraded":"2024-03-14T12:16:02.692977+0000","last_fullsized":"2024-03-14T12:16:02.692977+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:01:05.199522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.670038+0000","last_change":"2024-03-14T12:16:01.678519+0000","last_active":"2024-03-14T12:16:02.670038+0000","last_peered":"2024-03-14T12:16:02.670038+0000","last_clean":"2024-03-14T12:16:02.670038+0000","last_became_active":"2024-03-14T12:16:01.678242+0000","last_became_peered":"2024-03-14T12:16:01.678242+0000","last_unstale":"2024-03-14T12:16:02.670038+0000","last_undegraded":"2024-03-14T12:16:02.670038+0000","last_fullsized":"2024-03-14T12:16:02.670038+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:47.065534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.684610+0000","last_change":"2024-03-14T12:16:02.677835+0000","last_active":"2024-03-14T12:16:02.684610+0000","last_peered":"2024-03-14T12:16:02.684610+0000","last_clean":"2024-03-14T12:16:02.684610+0000","last_became_active":"2024-03-14T12:16:02.677519+0000","last_became_peered":"2024-03-14T12:16:02.677519+0000","last_unstale":"2024-03-14T12:16:02.684610+0000","last_undegraded":"2024-03-14T12:16:02.684610+0000","last_fullsized":"2024-03-14T12:16:02.684610+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:57:57.012520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676235+0000","last_change":"2024-03-14T12:16:01.681629+0000","last_active":"2024-03-14T12:16:02.676235+0000","last_peered":"2024-03-14T12:16:02.676235+0000","last_clean":"2024-03-14T12:16:02.676235+0000","last_became_active":"2024-03-14T12:16:01.681318+0000","last_became_peered":"2024-03-14T12:16:01.681318+0000","last_unstale":"2024-03-14T12:16:02.676235+0000","last_undegraded":"2024-03-14T12:16:02.676235+0000","last_fullsized":"2024-03-14T12:16:02.676235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:24:56.814661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698118+0000","last_change":"2024-03-14T12:16:02.696292+0000","last_active":"2024-03-14T12:16:02.698118+0000","last_peered":"2024-03-14T12:16:02.698118+0000","last_clean":"2024-03-14T12:16:02.698118+0000","last_became_active":"2024-03-14T12:16:02.696004+0000","last_became_peered":"2024-03-14T12:16:02.696004+0000","last_unstale":"2024-03-14T12:16:02.698118+0000","last_undegraded":"2024-03-14T12:16:02.698118+0000","last_fullsized":"2024-03-14T12:16:02.698118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:01.324877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.667236+0000","last_change":"2024-03-14T12:16:01.682514+0000","last_active":"2024-03-14T12:16:02.667236+0000","last_peered":"2024-03-14T12:16:02.667236+0000","last_clean":"2024-03-14T12:16:02.667236+0000","last_became_active":"2024-03-14T12:16:01.682217+0000","last_became_peered":"2024-03-14T12:16:01.682217+0000","last_unstale":"2024-03-14T12:16:02.667236+0000","last_undegraded":"2024-03-14T12:16:02.667236+0000","last_fullsized":"2024-03-14T12:16:02.667236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:47.065534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.678927+0000","last_change":"2024-03-14T12:16:02.671927+0000","last_active":"2024-03-14T12:16:02.678927+0000","last_peered":"2024-03-14T12:16:02.678927+0000","last_clean":"2024-03-14T12:16:02.678927+0000","last_became_active":"2024-03-14T12:16:02.671597+0000","last_became_peered":"2024-03-14T12:16:02.671597+0000","last_unstale":"2024-03-14T12:16:02.678927+0000","last_undegraded":"2024-03-14T12:16:02.678927+0000","last_fullsized":"2024-03-14T12:16:02.678927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:49.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.670053+0000","last_change":"2024-03-14T12:16:01.677577+0000","last_active":"2024-03-14T12:16:02.670053+0000","last_peered":"2024-03-14T12:16:02.670053+0000","last_clean":"2024-03-14T12:16:02.670053+0000","last_became_active":"2024-03-14T12:16:01.677267+0000","last_became_peered":"2024-03-14T12:16:01.677267+0000","last_unstale":"2024-03-14T12:16:02.670053+0000","last_undegraded":"2024-03-14T12:16:02.670053+0000","last_fullsized":"2024-03-14T12:16:02.670053+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:20:42.359319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686419+0000","last_change":"2024-03-14T12:16:02.682532+0000","last_active":"2024-03-14T12:16:02.686419+0000","last_peered":"2024-03-14T12:16:02.686419+0000","last_clean":"2024-03-14T12:16:02.686419+0000","last_became_active":"2024-03-14T12:16:02.681828+0000","last_became_peered":"2024-03-14T12:16:02.681828+0000","last_unstale":"2024-03-14T12:16:02.686419+0000","last_undegraded":"2024-03-14T12:16:02.686419+0000","last_fullsized":"2024-03-14T12:16:02.686419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:02:10.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676487+0000","last_change":"2024-03-14T12:16:01.682613+0000","last_active":"2024-03-14T12:16:02.676487+0000","last_peered":"2024-03-14T12:16:02.676487+0000","last_clean":"2024-03-14T12:16:02.676487+0000","last_became_active":"2024-03-14T12:16:01.682262+0000","last_became_peered":"2024-03-14T12:16:01.682262+0000","last_unstale":"2024-03-14T12:16:02.676487+0000","last_undegraded":"2024-03-14T12:16:02.676487+0000","last_fullsized":"2024-03-14T12:16:02.676487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:09:49.869686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697791+0000","last_change":"2024-03-14T12:16:02.691387+0000","last_active":"2024-03-14T12:16:02.697791+0000","last_peered":"2024-03-14T12:16:02.697791+0000","last_clean":"2024-03-14T12:16:02.697791+0000","last_became_active":"2024-03-14T12:16:02.691127+0000","last_became_peered":"2024-03-14T12:16:02.691127+0000","last_unstale":"2024-03-14T12:16:02.697791+0000","last_undegraded":"2024-03-14T12:16:02.697791+0000","last_fullsized":"2024-03-14T12:16:02.697791+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:02:10.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.696299+0000","last_change":"2024-03-14T12:16:01.682568+0000","last_active":"2024-03-14T12:16:02.696299+0000","last_peered":"2024-03-14T12:16:02.696299+0000","last_clean":"2024-03-14T12:16:02.696299+0000","last_became_active":"2024-03-14T12:16:01.682303+0000","last_became_peered":"2024-03-14T12:16:01.682303+0000","last_unstale":"2024-03-14T12:16:02.696299+0000","last_undegraded":"2024-03-14T12:16:02.696299+0000","last_fullsized":"2024-03-14T12:16:02.696299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:54.293322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676474+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:16:02.676474+0000","last_peered":"2024-03-14T12:16:02.676474+0000","last_clean":"2024-03-14T12:16:02.676474+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:16:02.676474+0000","last_undegraded":"2024-03-14T12:16:02.676474+0000","last_fullsized":"2024-03-14T12:16:02.676474+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":803375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":206142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":212987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":540672,"data_stored":214529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35876,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335964,"statfs":{"total":96636764160,"available":96600027136,"internally_reserved":0,"allocated":540672,"data_stored":213555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":214551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":532480,"data_stored":212987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":803319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:16:05.628 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:16:05.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.765+0000 7fa162280640 1 Processor -- start 2024-03-14T12:16:05.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.765+0000 7fa162280640 1 -- start start 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa162280640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa15c071740 0x7fa15c071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa162280640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 0x7fa15c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa162280640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15c0b3050 con 0x7fa15c15e670 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa162280640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c072060 con 0x7fa15c071740 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa162280640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15ebe0 con 0x7fa15c143f60 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa15c15e670 0x7fa15c0b2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36388/0 (socket says 172.21.15.82:36388) 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15b7fe640 1 -- 172.21.15.82:0/1411005554 learned_addr learned my addr 172.21.15.82:0/1411005554 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15bfff640 1 --2- 172.21.15.82:0/1411005554 >> v2:172.21.15.164:3300/0 conn(0x7fa15c071740 0x7fa15c071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa16081d640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 0x7fa15c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618050520 0 0) 0x7fa15c0b3050 con 0x7fa15c15e670 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa144003520 con 0x7fa15c15e670 2024-03-14T12:16:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1044212883 0 0) 0x7fa144003520 con 0x7fa15c15e670 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 msgr2=0x7fa15c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 0x7fa15c15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 >> v2:172.21.15.164:3300/0 conn(0x7fa15c071740 msgr2=0x7fa15c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 --2- 172.21.15.82:0/1411005554 >> v2:172.21.15.164:3300/0 conn(0x7fa15c071740 0x7fa15c071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c15eec0 con 0x7fa15c15e670 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fa150001670 con 0x7fa15c15e670 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 >> v1:172.21.15.82:6789/0 conn(0x7fa15c15e670 legacy=0x7fa15c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa144003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa144004800 0x7fa144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 0x7fa1440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c072060 con 0x7fa1440039b0 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15ebe0 con 0x7fa144004800 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.766+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14400b9c0 con 0x7fa15c143f60 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15bfff640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa144003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15b7fe640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa144004800 0x7fa144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa16081d640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 0x7fa1440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15bfff640 1 -- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 msgr2=0x7fa1440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15bfff640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c143f60 0x7fa1440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15bfff640 1 -- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa144004800 msgr2=0x7fa144004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15bfff640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa144004800 0x7fa144004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15bfff640 1 -- 172.21.15.82:0/1411005554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa144003520 con 0x7fa1440039b0 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15b7fe640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa144004800 0x7fa144004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.767+0000 7fa15bfff640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa144003dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009ac0 tx=0x7fa144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d14d90353154e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c039070 con 0x7fa1440039b0 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c02c710 con 0x7fa1440039b0 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa15affd640 1 -- 172.21.15.82:0/1411005554 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c009690 con 0x7fa1440039b0 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa162280640 1 -- 172.21.15.82:0/1411005554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 msgr2=0x7fa144003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa162280640 1 --2- 172.21.15.82:0/1411005554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa144003dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009ac0 tx=0x7fa144005830 comp rx=0 tx=0).stop 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa16081d640 1 -- 172.21.15.82:0/1411005554 reap_dead start 2024-03-14T12:16:05.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa162280640 1 -- 172.21.15.82:0/1411005554 shutdown_connections 2024-03-14T12:16:05.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa162280640 1 -- 172.21.15.82:0/1411005554 >> 172.21.15.82:0/1411005554 conn(0x7fa15c0b1c70 msgr2=0x7fa15c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:05.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa162280640 1 -- 172.21.15.82:0/1411005554 shutdown_connections 2024-03-14T12:16:05.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.768+0000 7fa162280640 1 -- 172.21.15.82:0/1411005554 wait complete. 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.769+0000 7fa162280640 1 Processor -- start 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa162280640 1 -- start start 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa162280640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa15c1f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa162280640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c0cb0e0 0x7fa15c0aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa162280640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa15c0aef50 0x7fa15c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa162280640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c143ce0 con 0x7fa1440039b0 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa162280640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15f480 con 0x7fa15c0aef50 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa162280640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15b080 con 0x7fa15c0cb0e0 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa15bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa15c1f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa15bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa15c1f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37202/0 (socket says 172.21.15.82:37202) 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa15bfff640 1 -- 172.21.15.82:0/2013250487 learned_addr learned my addr 172.21.15.82:0/2013250487 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:05.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa15b7fe640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c0cb0e0 0x7fa15c0aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.770+0000 7fa16081d640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa15c0aef50 0x7fa15c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa15bfff640 1 -- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c0cb0e0 msgr2=0x7fa15c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa15bfff640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c0cb0e0 0x7fa15c0aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa15bfff640 1 -- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa15c0aef50 msgr2=0x7fa15c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa15bfff640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa15c0aef50 0x7fa15c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa15bfff640 1 -- 172.21.15.82:0/2013250487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c1e4870 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa15bfff640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa15c1f4300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa14c037040 tx=0x7fa14c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa158ff9640 1 -- 172.21.15.82:0/2013250487 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c042020 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa158ff9640 1 -- 172.21.15.82:0/2013250487 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c03a040 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa158ff9640 1 -- 172.21.15.82:0/2013250487 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c039070 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c1e4b50 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.771+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c1e5110 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.772+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c04f050 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.772+0000 7fa158ff9640 1 -- 172.21.15.82:0/2013250487 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa14c045020 con 0x7fa1440039b0 2024-03-14T12:16:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.773+0000 7fa158ff9640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa14806bbd0 0x7fa14806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.773+0000 7fa15b7fe640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa14806bbd0 0x7fa14806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.773+0000 7fa158ff9640 1 -- 172.21.15.82:0/2013250487 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa14c0abe60 con 0x7fa1440039b0 2024-03-14T12:16:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.773+0000 7fa15b7fe640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa14806bbd0 0x7fa14806e090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa15c15a880 tx=0x7fa150009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.777+0000 7fa158ff9640 1 -- 172.21.15.82:0/2013250487 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa14c04c070 con 0x7fa1440039b0 2024-03-14T12:16:05.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.943+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa15c1f4840 con 0x7fa14806bbd0 2024-03-14T12:16:05.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.949+0000 7fa158ff9640 1 -- 172.21.15.82:0/2013250487 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251892 (secure 0 0 0) 0x7fa150010860 con 0x7fa14806bbd0 2024-03-14T12:16:05.951 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:05.952 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa14806bbd0 msgr2=0x7fa14806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa14806bbd0 0x7fa14806e090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa15c15a880 tx=0x7fa150009040 comp rx=0 tx=0).stop 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 msgr2=0x7fa15c1f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa15c1f4300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa14c037040 tx=0x7fa14c004730 comp rx=0 tx=0).stop 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 shutdown_connections 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa15c0aef50 0x7fa15c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa14806bbd0 0x7fa14806e090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa15c0cb0e0 0x7fa15c0aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 --2- 172.21.15.82:0/2013250487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1440039b0 0x7fa15c1f4300 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:05.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 >> 172.21.15.82:0/2013250487 conn(0x7fa15c0b1c70 msgr2=0x7fa15c0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:05.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 shutdown_connections 2024-03-14T12:16:05.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:05.952+0000 7fa162280640 1 -- 172.21.15.82:0/2013250487 wait complete. 2024-03-14T12:16:05.962 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-14T12:16:05.525935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254804,"kb_used_data":5332,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719916,"statfs":{"total":773094113280,"available":772833193984,"internally_reserved":0,"allocated":5459968,"data_stored":2881445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":255315507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.870015"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690295+0000","last_change":"2024-03-14T12:16:02.687478+0000","last_active":"2024-03-14T12:16:02.690295+0000","last_peered":"2024-03-14T12:16:02.690295+0000","last_clean":"2024-03-14T12:16:02.690295+0000","last_became_active":"2024-03-14T12:16:02.687105+0000","last_became_peered":"2024-03-14T12:16:02.687105+0000","last_unstale":"2024-03-14T12:16:02.690295+0000","last_undegraded":"2024-03-14T12:16:02.690295+0000","last_fullsized":"2024-03-14T12:16:02.690295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:48:55.741445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689736+0000","last_change":"2024-03-14T12:16:02.685283+0000","last_active":"2024-03-14T12:16:02.689736+0000","last_peered":"2024-03-14T12:16:02.689736+0000","last_clean":"2024-03-14T12:16:02.689736+0000","last_became_active":"2024-03-14T12:16:02.684970+0000","last_became_peered":"2024-03-14T12:16:02.684970+0000","last_unstale":"2024-03-14T12:16:02.689736+0000","last_undegraded":"2024-03-14T12:16:02.689736+0000","last_fullsized":"2024-03-14T12:16:02.689736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:03.381028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.679795+0000","last_change":"2024-03-14T12:16:02.673620+0000","last_active":"2024-03-14T12:16:02.679795+0000","last_peered":"2024-03-14T12:16:02.679795+0000","last_clean":"2024-03-14T12:16:02.679795+0000","last_became_active":"2024-03-14T12:16:02.673351+0000","last_became_peered":"2024-03-14T12:16:02.673351+0000","last_unstale":"2024-03-14T12:16:02.679795+0000","last_undegraded":"2024-03-14T12:16:02.679795+0000","last_fullsized":"2024-03-14T12:16:02.679795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:16.788654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697386+0000","last_change":"2024-03-14T12:16:02.689867+0000","last_active":"2024-03-14T12:16:02.697386+0000","last_peered":"2024-03-14T12:16:02.697386+0000","last_clean":"2024-03-14T12:16:02.697386+0000","last_became_active":"2024-03-14T12:16:02.689587+0000","last_became_peered":"2024-03-14T12:16:02.689587+0000","last_unstale":"2024-03-14T12:16:02.697386+0000","last_undegraded":"2024-03-14T12:16:02.697386+0000","last_fullsized":"2024-03-14T12:16:02.697386+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:16.788654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690337+0000","last_change":"2024-03-14T12:16:02.685455+0000","last_active":"2024-03-14T12:16:02.690337+0000","last_peered":"2024-03-14T12:16:02.690337+0000","last_clean":"2024-03-14T12:16:02.690337+0000","last_became_active":"2024-03-14T12:16:02.685064+0000","last_became_peered":"2024-03-14T12:16:02.685064+0000","last_unstale":"2024-03-14T12:16:02.690337+0000","last_undegraded":"2024-03-14T12:16:02.690337+0000","last_fullsized":"2024-03-14T12:16:02.690337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:24:50.624619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692372+0000","last_change":"2024-03-14T12:16:02.688403+0000","last_active":"2024-03-14T12:16:02.692372+0000","last_peered":"2024-03-14T12:16:02.692372+0000","last_clean":"2024-03-14T12:16:02.692372+0000","last_became_active":"2024-03-14T12:16:02.688114+0000","last_became_peered":"2024-03-14T12:16:02.688114+0000","last_unstale":"2024-03-14T12:16:02.692372+0000","last_undegraded":"2024-03-14T12:16:02.692372+0000","last_fullsized":"2024-03-14T12:16:02.692372+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:42:04.491643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690302+0000","last_change":"2024-03-14T12:16:02.686997+0000","last_active":"2024-03-14T12:16:02.690302+0000","last_peered":"2024-03-14T12:16:02.690302+0000","last_clean":"2024-03-14T12:16:02.690302+0000","last_became_active":"2024-03-14T12:16:02.686719+0000","last_became_peered":"2024-03-14T12:16:02.686719+0000","last_unstale":"2024-03-14T12:16:02.690302+0000","last_undegraded":"2024-03-14T12:16:02.690302+0000","last_fullsized":"2024-03-14T12:16:02.690302+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:29:00.413372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697767+0000","last_change":"2024-03-14T12:16:02.691682+0000","last_active":"2024-03-14T12:16:02.697767+0000","last_peered":"2024-03-14T12:16:02.697767+0000","last_clean":"2024-03-14T12:16:02.697767+0000","last_became_active":"2024-03-14T12:16:02.690387+0000","last_became_peered":"2024-03-14T12:16:02.690387+0000","last_unstale":"2024-03-14T12:16:02.697767+0000","last_undegraded":"2024-03-14T12:16:02.697767+0000","last_fullsized":"2024-03-14T12:16:02.697767+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:57.883134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697712+0000","last_change":"2024-03-14T12:16:02.690991+0000","last_active":"2024-03-14T12:16:02.697712+0000","last_peered":"2024-03-14T12:16:02.697712+0000","last_clean":"2024-03-14T12:16:02.697712+0000","last_became_active":"2024-03-14T12:16:02.690698+0000","last_became_peered":"2024-03-14T12:16:02.690698+0000","last_unstale":"2024-03-14T12:16:02.697712+0000","last_undegraded":"2024-03-14T12:16:02.697712+0000","last_fullsized":"2024-03-14T12:16:02.697712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:16.518796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697673+0000","last_change":"2024-03-14T12:16:02.690526+0000","last_active":"2024-03-14T12:16:02.697673+0000","last_peered":"2024-03-14T12:16:02.697673+0000","last_clean":"2024-03-14T12:16:02.697673+0000","last_became_active":"2024-03-14T12:16:02.690257+0000","last_became_peered":"2024-03-14T12:16:02.690257+0000","last_unstale":"2024-03-14T12:16:02.697673+0000","last_undegraded":"2024-03-14T12:16:02.697673+0000","last_fullsized":"2024-03-14T12:16:02.697673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:02:40.058933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697816+0000","last_change":"2024-03-14T12:16:02.691726+0000","last_active":"2024-03-14T12:16:02.697816+0000","last_peered":"2024-03-14T12:16:02.697816+0000","last_clean":"2024-03-14T12:16:02.697816+0000","last_became_active":"2024-03-14T12:16:02.691424+0000","last_became_peered":"2024-03-14T12:16:02.691424+0000","last_unstale":"2024-03-14T12:16:02.697816+0000","last_undegraded":"2024-03-14T12:16:02.697816+0000","last_fullsized":"2024-03-14T12:16:02.697816+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:55.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.679030+0000","last_change":"2024-03-14T12:16:02.675450+0000","last_active":"2024-03-14T12:16:02.679030+0000","last_peered":"2024-03-14T12:16:02.679030+0000","last_clean":"2024-03-14T12:16:02.679030+0000","last_became_active":"2024-03-14T12:16:02.674821+0000","last_became_peered":"2024-03-14T12:16:02.674821+0000","last_unstale":"2024-03-14T12:16:02.679030+0000","last_undegraded":"2024-03-14T12:16:02.679030+0000","last_fullsized":"2024-03-14T12:16:02.679030+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:11.055206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691209+0000","last_change":"2024-03-14T12:16:02.686240+0000","last_active":"2024-03-14T12:16:02.691209+0000","last_peered":"2024-03-14T12:16:02.691209+0000","last_clean":"2024-03-14T12:16:02.691209+0000","last_became_active":"2024-03-14T12:16:02.685838+0000","last_became_peered":"2024-03-14T12:16:02.685838+0000","last_unstale":"2024-03-14T12:16:02.691209+0000","last_undegraded":"2024-03-14T12:16:02.691209+0000","last_fullsized":"2024-03-14T12:16:02.691209+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:33:07.491250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697230+0000","last_change":"2024-03-14T12:16:02.689535+0000","last_active":"2024-03-14T12:16:02.697230+0000","last_peered":"2024-03-14T12:16:02.697230+0000","last_clean":"2024-03-14T12:16:02.697230+0000","last_became_active":"2024-03-14T12:16:02.689236+0000","last_became_peered":"2024-03-14T12:16:02.689236+0000","last_unstale":"2024-03-14T12:16:02.697230+0000","last_undegraded":"2024-03-14T12:16:02.697230+0000","last_fullsized":"2024-03-14T12:16:02.697230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:01.220239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.684647+0000","last_change":"2024-03-14T12:16:02.677831+0000","last_active":"2024-03-14T12:16:02.684647+0000","last_peered":"2024-03-14T12:16:02.684647+0000","last_clean":"2024-03-14T12:16:02.684647+0000","last_became_active":"2024-03-14T12:16:02.677537+0000","last_became_peered":"2024-03-14T12:16:02.677537+0000","last_unstale":"2024-03-14T12:16:02.684647+0000","last_undegraded":"2024-03-14T12:16:02.684647+0000","last_fullsized":"2024-03-14T12:16:02.684647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:18:31.470897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692957+0000","last_change":"2024-03-14T12:16:02.692289+0000","last_active":"2024-03-14T12:16:02.692957+0000","last_peered":"2024-03-14T12:16:02.692957+0000","last_clean":"2024-03-14T12:16:02.692957+0000","last_became_active":"2024-03-14T12:16:02.692026+0000","last_became_peered":"2024-03-14T12:16:02.692026+0000","last_unstale":"2024-03-14T12:16:02.692957+0000","last_undegraded":"2024-03-14T12:16:02.692957+0000","last_fullsized":"2024-03-14T12:16:02.692957+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:00:09.135192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.678230+0000","last_change":"2024-03-14T12:16:02.675157+0000","last_active":"2024-03-14T12:16:02.678230+0000","last_peered":"2024-03-14T12:16:02.678230+0000","last_clean":"2024-03-14T12:16:02.678230+0000","last_became_active":"2024-03-14T12:16:02.674727+0000","last_became_peered":"2024-03-14T12:16:02.674727+0000","last_unstale":"2024-03-14T12:16:02.678230+0000","last_undegraded":"2024-03-14T12:16:02.678230+0000","last_fullsized":"2024-03-14T12:16:02.678230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:48.005671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686330+0000","last_change":"2024-03-14T12:16:02.682867+0000","last_active":"2024-03-14T12:16:02.686330+0000","last_peered":"2024-03-14T12:16:02.686330+0000","last_clean":"2024-03-14T12:16:02.686330+0000","last_became_active":"2024-03-14T12:16:02.682594+0000","last_became_peered":"2024-03-14T12:16:02.682594+0000","last_unstale":"2024-03-14T12:16:02.686330+0000","last_undegraded":"2024-03-14T12:16:02.686330+0000","last_fullsized":"2024-03-14T12:16:02.686330+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:57.883134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692950+0000","last_change":"2024-03-14T12:16:02.691583+0000","last_active":"2024-03-14T12:16:02.692950+0000","last_peered":"2024-03-14T12:16:02.692950+0000","last_clean":"2024-03-14T12:16:02.692950+0000","last_became_active":"2024-03-14T12:16:02.691264+0000","last_became_peered":"2024-03-14T12:16:02.691264+0000","last_unstale":"2024-03-14T12:16:02.692950+0000","last_undegraded":"2024-03-14T12:16:02.692950+0000","last_fullsized":"2024-03-14T12:16:02.692950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:38:27.730273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690093+0000","last_change":"2024-03-14T12:16:02.683097+0000","last_active":"2024-03-14T12:16:02.690093+0000","last_peered":"2024-03-14T12:16:02.690093+0000","last_clean":"2024-03-14T12:16:02.690093+0000","last_became_active":"2024-03-14T12:16:02.682740+0000","last_became_peered":"2024-03-14T12:16:02.682740+0000","last_unstale":"2024-03-14T12:16:02.690093+0000","last_undegraded":"2024-03-14T12:16:02.690093+0000","last_fullsized":"2024-03-14T12:16:02.690093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:11.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692429+0000","last_change":"2024-03-14T12:16:02.685667+0000","last_active":"2024-03-14T12:16:02.692429+0000","last_peered":"2024-03-14T12:16:02.692429+0000","last_clean":"2024-03-14T12:16:02.692429+0000","last_became_active":"2024-03-14T12:16:02.685403+0000","last_became_peered":"2024-03-14T12:16:02.685403+0000","last_unstale":"2024-03-14T12:16:02.692429+0000","last_undegraded":"2024-03-14T12:16:02.692429+0000","last_fullsized":"2024-03-14T12:16:02.692429+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:40:57.545081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692193+0000","last_change":"2024-03-14T12:16:02.688847+0000","last_active":"2024-03-14T12:16:02.692193+0000","last_peered":"2024-03-14T12:16:02.692193+0000","last_clean":"2024-03-14T12:16:02.692193+0000","last_became_active":"2024-03-14T12:16:02.688537+0000","last_became_peered":"2024-03-14T12:16:02.688537+0000","last_unstale":"2024-03-14T12:16:02.692193+0000","last_undegraded":"2024-03-14T12:16:02.692193+0000","last_fullsized":"2024-03-14T12:16:02.692193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:20:43.368288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691169+0000","last_change":"2024-03-14T12:16:02.688575+0000","last_active":"2024-03-14T12:16:02.691169+0000","last_peered":"2024-03-14T12:16:02.691169+0000","last_clean":"2024-03-14T12:16:02.691169+0000","last_became_active":"2024-03-14T12:16:02.688228+0000","last_became_peered":"2024-03-14T12:16:02.688228+0000","last_unstale":"2024-03-14T12:16:02.691169+0000","last_undegraded":"2024-03-14T12:16:02.691169+0000","last_fullsized":"2024-03-14T12:16:02.691169+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:58:58.976190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691138+0000","last_change":"2024-03-14T12:16:02.689451+0000","last_active":"2024-03-14T12:16:02.691138+0000","last_peered":"2024-03-14T12:16:02.691138+0000","last_clean":"2024-03-14T12:16:02.691138+0000","last_became_active":"2024-03-14T12:16:02.688861+0000","last_became_peered":"2024-03-14T12:16:02.688861+0000","last_unstale":"2024-03-14T12:16:02.691138+0000","last_undegraded":"2024-03-14T12:16:02.691138+0000","last_fullsized":"2024-03-14T12:16:02.691138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:58:58.380420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689690+0000","last_change":"2024-03-14T12:16:02.685536+0000","last_active":"2024-03-14T12:16:02.689690+0000","last_peered":"2024-03-14T12:16:02.689690+0000","last_clean":"2024-03-14T12:16:02.689690+0000","last_became_active":"2024-03-14T12:16:02.685243+0000","last_became_peered":"2024-03-14T12:16:02.685243+0000","last_unstale":"2024-03-14T12:16:02.689690+0000","last_undegraded":"2024-03-14T12:16:02.689690+0000","last_fullsized":"2024-03-14T12:16:02.689690+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:33:07.491250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691969+0000","last_change":"2024-03-14T12:16:02.686580+0000","last_active":"2024-03-14T12:16:02.691969+0000","last_peered":"2024-03-14T12:16:02.691969+0000","last_clean":"2024-03-14T12:16:02.691969+0000","last_became_active":"2024-03-14T12:16:02.686317+0000","last_became_peered":"2024-03-14T12:16:02.686317+0000","last_unstale":"2024-03-14T12:16:02.691969+0000","last_undegraded":"2024-03-14T12:16:02.691969+0000","last_fullsized":"2024-03-14T12:16:02.691969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:21:54.822695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.683676+0000","last_change":"2024-03-14T12:16:02.679930+0000","last_active":"2024-03-14T12:16:02.683676+0000","last_peered":"2024-03-14T12:16:02.683676+0000","last_clean":"2024-03-14T12:16:02.683676+0000","last_became_active":"2024-03-14T12:16:02.679577+0000","last_became_peered":"2024-03-14T12:16:02.679577+0000","last_unstale":"2024-03-14T12:16:02.683676+0000","last_undegraded":"2024-03-14T12:16:02.683676+0000","last_fullsized":"2024-03-14T12:16:02.683676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:46.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.678988+0000","last_change":"2024-03-14T12:16:02.674561+0000","last_active":"2024-03-14T12:16:02.678988+0000","last_peered":"2024-03-14T12:16:02.678988+0000","last_clean":"2024-03-14T12:16:02.678988+0000","last_became_active":"2024-03-14T12:16:02.674271+0000","last_became_peered":"2024-03-14T12:16:02.674271+0000","last_unstale":"2024-03-14T12:16:02.678988+0000","last_undegraded":"2024-03-14T12:16:02.678988+0000","last_fullsized":"2024-03-14T12:16:02.678988+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:24:57.823630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690382+0000","last_change":"2024-03-14T12:16:02.684628+0000","last_active":"2024-03-14T12:16:02.690382+0000","last_peered":"2024-03-14T12:16:02.690382+0000","last_clean":"2024-03-14T12:16:02.690382+0000","last_became_active":"2024-03-14T12:16:02.684357+0000","last_became_peered":"2024-03-14T12:16:02.684357+0000","last_unstale":"2024-03-14T12:16:02.690382+0000","last_undegraded":"2024-03-14T12:16:02.690382+0000","last_fullsized":"2024-03-14T12:16:02.690382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:11.114126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689229+0000","last_change":"2024-03-14T12:16:02.683867+0000","last_active":"2024-03-14T12:16:02.689229+0000","last_peered":"2024-03-14T12:16:02.689229+0000","last_clean":"2024-03-14T12:16:02.689229+0000","last_became_active":"2024-03-14T12:16:02.683600+0000","last_became_peered":"2024-03-14T12:16:02.683600+0000","last_unstale":"2024-03-14T12:16:02.689229+0000","last_undegraded":"2024-03-14T12:16:02.689229+0000","last_fullsized":"2024-03-14T12:16:02.689229+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:55.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.684671+0000","last_change":"2024-03-14T12:16:02.680002+0000","last_active":"2024-03-14T12:16:02.684671+0000","last_peered":"2024-03-14T12:16:02.684671+0000","last_clean":"2024-03-14T12:16:02.684671+0000","last_became_active":"2024-03-14T12:16:02.679751+0000","last_became_peered":"2024-03-14T12:16:02.679751+0000","last_unstale":"2024-03-14T12:16:02.684671+0000","last_undegraded":"2024-03-14T12:16:02.684671+0000","last_fullsized":"2024-03-14T12:16:02.684671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:52:14.858524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689833+0000","last_change":"2024-03-14T12:16:02.672616+0000","last_active":"2024-03-14T12:16:02.689833+0000","last_peered":"2024-03-14T12:16:02.689833+0000","last_clean":"2024-03-14T12:16:02.689833+0000","last_became_active":"2024-03-14T12:16:02.672329+0000","last_became_peered":"2024-03-14T12:16:02.672329+0000","last_unstale":"2024-03-14T12:16:02.689833+0000","last_undegraded":"2024-03-14T12:16:02.689833+0000","last_fullsized":"2024-03-14T12:16:02.689833+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:09:39.347025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.687520+0000","last_change":"2024-03-14T12:16:02.685662+0000","last_active":"2024-03-14T12:16:02.687520+0000","last_peered":"2024-03-14T12:16:02.687520+0000","last_clean":"2024-03-14T12:16:02.687520+0000","last_became_active":"2024-03-14T12:16:02.685300+0000","last_became_peered":"2024-03-14T12:16:02.685300+0000","last_unstale":"2024-03-14T12:16:02.687520+0000","last_undegraded":"2024-03-14T12:16:02.687520+0000","last_fullsized":"2024-03-14T12:16:02.687520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:46.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698095+0000","last_change":"2024-03-14T12:16:01.677792+0000","last_active":"2024-03-14T12:16:02.698095+0000","last_peered":"2024-03-14T12:16:02.698095+0000","last_clean":"2024-03-14T12:16:02.698095+0000","last_became_active":"2024-03-14T12:16:01.677479+0000","last_became_peered":"2024-03-14T12:16:01.677479+0000","last_unstale":"2024-03-14T12:16:02.698095+0000","last_undegraded":"2024-03-14T12:16:02.698095+0000","last_fullsized":"2024-03-14T12:16:02.698095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:48:54.732476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.687635+0000","last_change":"2024-03-14T12:16:02.680862+0000","last_active":"2024-03-14T12:16:02.687635+0000","last_peered":"2024-03-14T12:16:02.687635+0000","last_clean":"2024-03-14T12:16:02.687635+0000","last_became_active":"2024-03-14T12:16:02.680586+0000","last_became_peered":"2024-03-14T12:16:02.680586+0000","last_unstale":"2024-03-14T12:16:02.687635+0000","last_undegraded":"2024-03-14T12:16:02.687635+0000","last_fullsized":"2024-03-14T12:16:02.687635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:56:08.364261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691704+0000","last_change":"2024-03-14T12:16:01.678475+0000","last_active":"2024-03-14T12:16:02.691704+0000","last_peered":"2024-03-14T12:16:02.691704+0000","last_clean":"2024-03-14T12:16:02.691704+0000","last_became_active":"2024-03-14T12:16:01.678193+0000","last_became_peered":"2024-03-14T12:16:01.678193+0000","last_unstale":"2024-03-14T12:16:02.691704+0000","last_undegraded":"2024-03-14T12:16:02.691704+0000","last_fullsized":"2024-03-14T12:16:02.691704+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:09:49.869686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690650+0000","last_change":"2024-03-14T12:16:02.688572+0000","last_active":"2024-03-14T12:16:02.690650+0000","last_peered":"2024-03-14T12:16:02.690650+0000","last_clean":"2024-03-14T12:16:02.690650+0000","last_became_active":"2024-03-14T12:16:02.688290+0000","last_became_peered":"2024-03-14T12:16:02.688290+0000","last_unstale":"2024-03-14T12:16:02.690650+0000","last_undegraded":"2024-03-14T12:16:02.690650+0000","last_fullsized":"2024-03-14T12:16:02.690650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:29:00.413372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690350+0000","last_change":"2024-03-14T12:16:01.677698+0000","last_active":"2024-03-14T12:16:02.690350+0000","last_peered":"2024-03-14T12:16:02.690350+0000","last_clean":"2024-03-14T12:16:02.690350+0000","last_became_active":"2024-03-14T12:16:01.677420+0000","last_became_peered":"2024-03-14T12:16:01.677420+0000","last_unstale":"2024-03-14T12:16:02.690350+0000","last_undegraded":"2024-03-14T12:16:02.690350+0000","last_fullsized":"2024-03-14T12:16:02.690350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.211270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.687872+0000","last_change":"2024-03-14T12:16:02.684325+0000","last_active":"2024-03-14T12:16:02.687872+0000","last_peered":"2024-03-14T12:16:02.687872+0000","last_clean":"2024-03-14T12:16:02.687872+0000","last_became_active":"2024-03-14T12:16:02.683852+0000","last_became_peered":"2024-03-14T12:16:02.683852+0000","last_unstale":"2024-03-14T12:16:02.687872+0000","last_undegraded":"2024-03-14T12:16:02.687872+0000","last_fullsized":"2024-03-14T12:16:02.687872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:48.005671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690966+0000","last_change":"2024-03-14T12:16:01.681606+0000","last_active":"2024-03-14T12:16:02.690966+0000","last_peered":"2024-03-14T12:16:02.690966+0000","last_clean":"2024-03-14T12:16:02.690966+0000","last_became_active":"2024-03-14T12:16:01.680984+0000","last_became_peered":"2024-03-14T12:16:01.680984+0000","last_unstale":"2024-03-14T12:16:02.690966+0000","last_undegraded":"2024-03-14T12:16:02.690966+0000","last_fullsized":"2024-03-14T12:16:02.690966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:10.046237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686364+0000","last_change":"2024-03-14T12:16:02.681432+0000","last_active":"2024-03-14T12:16:02.686364+0000","last_peered":"2024-03-14T12:16:02.686364+0000","last_clean":"2024-03-14T12:16:02.686364+0000","last_became_active":"2024-03-14T12:16:02.681144+0000","last_became_peered":"2024-03-14T12:16:02.681144+0000","last_unstale":"2024-03-14T12:16:02.686364+0000","last_undegraded":"2024-03-14T12:16:02.686364+0000","last_fullsized":"2024-03-14T12:16:02.686364+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:02:40.058933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697558+0000","last_change":"2024-03-14T12:16:01.680027+0000","last_active":"2024-03-14T12:16:02.697558+0000","last_peered":"2024-03-14T12:16:02.697558+0000","last_clean":"2024-03-14T12:16:02.697558+0000","last_became_active":"2024-03-14T12:16:01.678891+0000","last_became_peered":"2024-03-14T12:16:01.678891+0000","last_unstale":"2024-03-14T12:16:02.697558+0000","last_undegraded":"2024-03-14T12:16:02.697558+0000","last_fullsized":"2024-03-14T12:16:02.697558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:33.141787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699181+0000","last_change":"2024-03-14T12:16:02.694449+0000","last_active":"2024-03-14T12:16:02.699181+0000","last_peered":"2024-03-14T12:16:02.699181+0000","last_clean":"2024-03-14T12:16:02.699181+0000","last_became_active":"2024-03-14T12:16:02.693746+0000","last_became_peered":"2024-03-14T12:16:02.693746+0000","last_unstale":"2024-03-14T12:16:02.699181+0000","last_undegraded":"2024-03-14T12:16:02.699181+0000","last_fullsized":"2024-03-14T12:16:02.699181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:24:57.823630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690014+0000","last_change":"2024-03-14T12:16:01.676852+0000","last_active":"2024-03-14T12:16:02.690014+0000","last_peered":"2024-03-14T12:16:02.690014+0000","last_clean":"2024-03-14T12:16:02.690014+0000","last_became_active":"2024-03-14T12:16:01.676563+0000","last_became_peered":"2024-03-14T12:16:01.676563+0000","last_unstale":"2024-03-14T12:16:02.690014+0000","last_undegraded":"2024-03-14T12:16:02.690014+0000","last_fullsized":"2024-03-14T12:16:02.690014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:20:42.359319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697743+0000","last_change":"2024-03-14T12:16:02.690356+0000","last_active":"2024-03-14T12:16:02.697743+0000","last_peered":"2024-03-14T12:16:02.697743+0000","last_clean":"2024-03-14T12:16:02.697743+0000","last_became_active":"2024-03-14T12:16:02.684412+0000","last_became_peered":"2024-03-14T12:16:02.684412+0000","last_unstale":"2024-03-14T12:16:02.697743+0000","last_undegraded":"2024-03-14T12:16:02.697743+0000","last_fullsized":"2024-03-14T12:16:02.697743+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:46.823059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.666180+0000","last_change":"2024-03-14T12:16:01.675440+0000","last_active":"2024-03-14T12:16:02.666180+0000","last_peered":"2024-03-14T12:16:02.666180+0000","last_clean":"2024-03-14T12:16:02.666180+0000","last_became_active":"2024-03-14T12:16:01.675159+0000","last_became_peered":"2024-03-14T12:16:01.675159+0000","last_unstale":"2024-03-14T12:16:02.666180+0000","last_undegraded":"2024-03-14T12:16:02.666180+0000","last_fullsized":"2024-03-14T12:16:02.666180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:15.509827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692045+0000","last_change":"2024-03-14T12:16:02.690719+0000","last_active":"2024-03-14T12:16:02.692045+0000","last_peered":"2024-03-14T12:16:02.692045+0000","last_clean":"2024-03-14T12:16:02.692045+0000","last_became_active":"2024-03-14T12:16:02.690026+0000","last_became_peered":"2024-03-14T12:16:02.690026+0000","last_unstale":"2024-03-14T12:16:02.692045+0000","last_undegraded":"2024-03-14T12:16:02.692045+0000","last_fullsized":"2024-03-14T12:16:02.692045+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:29.572333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.675998+0000","last_change":"2024-03-14T12:16:01.679198+0000","last_active":"2024-03-14T12:16:02.675998+0000","last_peered":"2024-03-14T12:16:02.675998+0000","last_clean":"2024-03-14T12:16:02.675998+0000","last_became_active":"2024-03-14T12:16:01.678929+0000","last_became_peered":"2024-03-14T12:16:01.678929+0000","last_unstale":"2024-03-14T12:16:02.675998+0000","last_undegraded":"2024-03-14T12:16:02.675998+0000","last_fullsized":"2024-03-14T12:16:02.675998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:38:26.721304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686308+0000","last_change":"2024-03-14T12:16:02.682289+0000","last_active":"2024-03-14T12:16:02.686308+0000","last_peered":"2024-03-14T12:16:02.686308+0000","last_clean":"2024-03-14T12:16:02.686308+0000","last_became_active":"2024-03-14T12:16:02.681982+0000","last_became_peered":"2024-03-14T12:16:02.681982+0000","last_unstale":"2024-03-14T12:16:02.686308+0000","last_undegraded":"2024-03-14T12:16:02.686308+0000","last_fullsized":"2024-03-14T12:16:02.686308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:16.518796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.667553+0000","last_change":"2024-03-14T12:16:01.676109+0000","last_active":"2024-03-14T12:16:02.667553+0000","last_peered":"2024-03-14T12:16:02.667553+0000","last_clean":"2024-03-14T12:16:02.667553+0000","last_became_active":"2024-03-14T12:16:01.675821+0000","last_became_peered":"2024-03-14T12:16:01.675821+0000","last_unstale":"2024-03-14T12:16:02.667553+0000","last_undegraded":"2024-03-14T12:16:02.667553+0000","last_fullsized":"2024-03-14T12:16:02.667553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.211270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690327+0000","last_change":"2024-03-14T12:16:02.687697+0000","last_active":"2024-03-14T12:16:02.690327+0000","last_peered":"2024-03-14T12:16:02.690327+0000","last_clean":"2024-03-14T12:16:02.690327+0000","last_became_active":"2024-03-14T12:16:02.687393+0000","last_became_peered":"2024-03-14T12:16:02.687393+0000","last_unstale":"2024-03-14T12:16:02.690327+0000","last_undegraded":"2024-03-14T12:16:02.690327+0000","last_fullsized":"2024-03-14T12:16:02.690327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:28.315595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676744+0000","last_change":"2024-03-14T12:16:01.679923+0000","last_active":"2024-03-14T12:16:02.676744+0000","last_peered":"2024-03-14T12:16:02.676744+0000","last_clean":"2024-03-14T12:16:02.676744+0000","last_became_active":"2024-03-14T12:16:01.679288+0000","last_became_peered":"2024-03-14T12:16:01.679288+0000","last_unstale":"2024-03-14T12:16:02.676744+0000","last_undegraded":"2024-03-14T12:16:02.676744+0000","last_fullsized":"2024-03-14T12:16:02.676744+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:40:04.426917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698146+0000","last_change":"2024-03-14T12:16:02.696554+0000","last_active":"2024-03-14T12:16:02.698146+0000","last_peered":"2024-03-14T12:16:02.698146+0000","last_clean":"2024-03-14T12:16:02.698146+0000","last_became_active":"2024-03-14T12:16:02.695815+0000","last_became_peered":"2024-03-14T12:16:02.695815+0000","last_unstale":"2024-03-14T12:16:02.698146+0000","last_undegraded":"2024-03-14T12:16:02.698146+0000","last_fullsized":"2024-03-14T12:16:02.698146+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:16:50.910057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691434+0000","last_change":"2024-03-14T12:16:01.682600+0000","last_active":"2024-03-14T12:16:02.691434+0000","last_peered":"2024-03-14T12:16:02.691434+0000","last_clean":"2024-03-14T12:16:02.691434+0000","last_became_active":"2024-03-14T12:16:01.682325+0000","last_became_peered":"2024-03-14T12:16:01.682325+0000","last_unstale":"2024-03-14T12:16:02.691434+0000","last_undegraded":"2024-03-14T12:16:02.691434+0000","last_fullsized":"2024-03-14T12:16:02.691434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:56.874165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690348+0000","last_change":"2024-03-14T12:16:02.686201+0000","last_active":"2024-03-14T12:16:02.690348+0000","last_peered":"2024-03-14T12:16:02.690348+0000","last_clean":"2024-03-14T12:16:02.690348+0000","last_became_active":"2024-03-14T12:16:02.685907+0000","last_became_peered":"2024-03-14T12:16:02.685907+0000","last_unstale":"2024-03-14T12:16:02.690348+0000","last_undegraded":"2024-03-14T12:16:02.690348+0000","last_fullsized":"2024-03-14T12:16:02.690348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:24:50.624619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699820+0000","last_change":"2024-03-14T12:16:01.671671+0000","last_active":"2024-03-14T12:16:02.699820+0000","last_peered":"2024-03-14T12:16:02.699820+0000","last_clean":"2024-03-14T12:16:02.699820+0000","last_became_active":"2024-03-14T12:16:01.671378+0000","last_became_peered":"2024-03-14T12:16:01.671378+0000","last_unstale":"2024-03-14T12:16:02.699820+0000","last_undegraded":"2024-03-14T12:16:02.699820+0000","last_fullsized":"2024-03-14T12:16:02.699820+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:53.007538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.695190+0000","last_change":"2024-03-14T12:16:02.694497+0000","last_active":"2024-03-14T12:16:02.695190+0000","last_peered":"2024-03-14T12:16:02.695190+0000","last_clean":"2024-03-14T12:16:02.695190+0000","last_became_active":"2024-03-14T12:16:02.694113+0000","last_became_peered":"2024-03-14T12:16:02.694113+0000","last_unstale":"2024-03-14T12:16:02.695190+0000","last_undegraded":"2024-03-14T12:16:02.695190+0000","last_fullsized":"2024-03-14T12:16:02.695190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:09:50.878654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.700254+0000","last_change":"2024-03-14T12:16:01.684242+0000","last_active":"2024-03-14T12:16:02.700254+0000","last_peered":"2024-03-14T12:16:02.700254+0000","last_clean":"2024-03-14T12:16:02.700254+0000","last_became_active":"2024-03-14T12:16:01.683942+0000","last_became_peered":"2024-03-14T12:16:01.683942+0000","last_unstale":"2024-03-14T12:16:02.700254+0000","last_undegraded":"2024-03-14T12:16:02.700254+0000","last_fullsized":"2024-03-14T12:16:02.700254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:14:59.279221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699150+0000","last_change":"2024-03-14T12:16:02.692982+0000","last_active":"2024-03-14T12:16:02.699150+0000","last_peered":"2024-03-14T12:16:02.699150+0000","last_clean":"2024-03-14T12:16:02.699150+0000","last_became_active":"2024-03-14T12:16:02.692527+0000","last_became_peered":"2024-03-14T12:16:02.692527+0000","last_unstale":"2024-03-14T12:16:02.699150+0000","last_undegraded":"2024-03-14T12:16:02.699150+0000","last_fullsized":"2024-03-14T12:16:02.699150+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:49.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.679034+0000","last_change":"2024-03-14T12:16:01.682074+0000","last_active":"2024-03-14T12:16:02.679034+0000","last_peered":"2024-03-14T12:16:02.679034+0000","last_clean":"2024-03-14T12:16:02.679034+0000","last_became_active":"2024-03-14T12:16:01.681549+0000","last_became_peered":"2024-03-14T12:16:01.681549+0000","last_unstale":"2024-03-14T12:16:02.679034+0000","last_undegraded":"2024-03-14T12:16:02.679034+0000","last_fullsized":"2024-03-14T12:16:02.679034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:02.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698097+0000","last_change":"2024-03-14T12:16:02.696718+0000","last_active":"2024-03-14T12:16:02.698097+0000","last_peered":"2024-03-14T12:16:02.698097+0000","last_clean":"2024-03-14T12:16:02.698097+0000","last_became_active":"2024-03-14T12:16:02.695787+0000","last_became_peered":"2024-03-14T12:16:02.695787+0000","last_unstale":"2024-03-14T12:16:02.698097+0000","last_undegraded":"2024-03-14T12:16:02.698097+0000","last_fullsized":"2024-03-14T12:16:02.698097+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:50:45.793287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.673973+0000","last_change":"2024-03-14T12:16:01.683671+0000","last_active":"2024-03-14T12:16:02.673973+0000","last_peered":"2024-03-14T12:16:02.673973+0000","last_clean":"2024-03-14T12:16:02.673973+0000","last_became_active":"2024-03-14T12:16:01.683368+0000","last_became_peered":"2024-03-14T12:16:01.683368+0000","last_unstale":"2024-03-14T12:16:02.673973+0000","last_undegraded":"2024-03-14T12:16:02.673973+0000","last_fullsized":"2024-03-14T12:16:02.673973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:09:06.112616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690883+0000","last_change":"2024-03-14T12:16:02.689528+0000","last_active":"2024-03-14T12:16:02.690883+0000","last_peered":"2024-03-14T12:16:02.690883+0000","last_clean":"2024-03-14T12:16:02.690883+0000","last_became_active":"2024-03-14T12:16:02.689179+0000","last_became_peered":"2024-03-14T12:16:02.689179+0000","last_unstale":"2024-03-14T12:16:02.690883+0000","last_undegraded":"2024-03-14T12:16:02.690883+0000","last_fullsized":"2024-03-14T12:16:02.690883+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:09:07.121584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.694824+0000","last_change":"2024-03-14T12:16:01.678880+0000","last_active":"2024-03-14T12:16:02.694824+0000","last_peered":"2024-03-14T12:16:02.694824+0000","last_clean":"2024-03-14T12:16:02.694824+0000","last_became_active":"2024-03-14T12:16:01.678559+0000","last_became_peered":"2024-03-14T12:16:01.678559+0000","last_unstale":"2024-03-14T12:16:02.694824+0000","last_undegraded":"2024-03-14T12:16:02.694824+0000","last_fullsized":"2024-03-14T12:16:02.694824+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:00:08.126224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691944+0000","last_change":"2024-03-14T12:16:02.676009+0000","last_active":"2024-03-14T12:16:02.691944+0000","last_peered":"2024-03-14T12:16:02.691944+0000","last_clean":"2024-03-14T12:16:02.691944+0000","last_became_active":"2024-03-14T12:16:02.675728+0000","last_became_peered":"2024-03-14T12:16:02.675728+0000","last_unstale":"2024-03-14T12:16:02.691944+0000","last_undegraded":"2024-03-14T12:16:02.691944+0000","last_fullsized":"2024-03-14T12:16:02.691944+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:56:08.364261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.696164+0000","last_change":"2024-03-14T12:16:01.680513+0000","last_active":"2024-03-14T12:16:02.696164+0000","last_peered":"2024-03-14T12:16:02.696164+0000","last_clean":"2024-03-14T12:16:02.696164+0000","last_became_active":"2024-03-14T12:16:01.680252+0000","last_became_peered":"2024-03-14T12:16:01.680252+0000","last_unstale":"2024-03-14T12:16:02.696164+0000","last_undegraded":"2024-03-14T12:16:02.696164+0000","last_fullsized":"2024-03-14T12:16:02.696164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:50.005267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.699208+0000","last_change":"2024-03-14T12:16:02.696683+0000","last_active":"2024-03-14T12:16:02.699208+0000","last_peered":"2024-03-14T12:16:02.699208+0000","last_clean":"2024-03-14T12:16:02.699208+0000","last_became_active":"2024-03-14T12:16:02.696420+0000","last_became_peered":"2024-03-14T12:16:02.696420+0000","last_unstale":"2024-03-14T12:16:02.699208+0000","last_undegraded":"2024-03-14T12:16:02.699208+0000","last_fullsized":"2024-03-14T12:16:02.699208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:11.055206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689172+0000","last_change":"2024-03-14T12:16:01.679648+0000","last_active":"2024-03-14T12:16:02.689172+0000","last_peered":"2024-03-14T12:16:02.689172+0000","last_clean":"2024-03-14T12:16:02.689172+0000","last_became_active":"2024-03-14T12:16:01.678650+0000","last_became_peered":"2024-03-14T12:16:01.678650+0000","last_unstale":"2024-03-14T12:16:02.689172+0000","last_undegraded":"2024-03-14T12:16:02.689172+0000","last_fullsized":"2024-03-14T12:16:02.689172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:40:04.426917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.689732+0000","last_change":"2024-03-14T12:16:02.686565+0000","last_active":"2024-03-14T12:16:02.689732+0000","last_peered":"2024-03-14T12:16:02.689732+0000","last_clean":"2024-03-14T12:16:02.689732+0000","last_became_active":"2024-03-14T12:16:02.686310+0000","last_became_peered":"2024-03-14T12:16:02.686310+0000","last_unstale":"2024-03-14T12:16:02.689732+0000","last_undegraded":"2024-03-14T12:16:02.689732+0000","last_fullsized":"2024-03-14T12:16:02.689732+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:49:43.398827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.688992+0000","last_change":"2024-03-14T12:16:01.676759+0000","last_active":"2024-03-14T12:16:02.688992+0000","last_peered":"2024-03-14T12:16:02.688992+0000","last_clean":"2024-03-14T12:16:02.688992+0000","last_became_active":"2024-03-14T12:16:01.676132+0000","last_became_peered":"2024-03-14T12:16:01.676132+0000","last_unstale":"2024-03-14T12:16:02.688992+0000","last_undegraded":"2024-03-14T12:16:02.688992+0000","last_fullsized":"2024-03-14T12:16:02.688992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:28:56.874165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.690000+0000","last_change":"2024-03-14T12:16:02.686243+0000","last_active":"2024-03-14T12:16:02.690000+0000","last_peered":"2024-03-14T12:16:02.690000+0000","last_clean":"2024-03-14T12:16:02.690000+0000","last_became_active":"2024-03-14T12:16:02.685867+0000","last_became_peered":"2024-03-14T12:16:02.685867+0000","last_unstale":"2024-03-14T12:16:02.690000+0000","last_undegraded":"2024-03-14T12:16:02.690000+0000","last_fullsized":"2024-03-14T12:16:02.690000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:21:54.822695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.673847+0000","last_change":"2024-03-14T12:16:01.682781+0000","last_active":"2024-03-14T12:16:02.673847+0000","last_peered":"2024-03-14T12:16:02.673847+0000","last_clean":"2024-03-14T12:16:02.673847+0000","last_became_active":"2024-03-14T12:16:01.682445+0000","last_became_peered":"2024-03-14T12:16:01.682445+0000","last_unstale":"2024-03-14T12:16:02.673847+0000","last_undegraded":"2024-03-14T12:16:02.673847+0000","last_fullsized":"2024-03-14T12:16:02.673847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:18:27.306626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697997+0000","last_change":"2024-03-14T12:16:02.696086+0000","last_active":"2024-03-14T12:16:02.697997+0000","last_peered":"2024-03-14T12:16:02.697997+0000","last_clean":"2024-03-14T12:16:02.697997+0000","last_became_active":"2024-03-14T12:16:02.695755+0000","last_became_peered":"2024-03-14T12:16:02.695755+0000","last_unstale":"2024-03-14T12:16:02.697997+0000","last_undegraded":"2024-03-14T12:16:02.697997+0000","last_fullsized":"2024-03-14T12:16:02.697997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:46:04.081089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697644+0000","last_change":"2024-03-14T12:16:01.678470+0000","last_active":"2024-03-14T12:16:02.697644+0000","last_peered":"2024-03-14T12:16:02.697644+0000","last_clean":"2024-03-14T12:16:02.697644+0000","last_became_active":"2024-03-14T12:16:01.678183+0000","last_became_peered":"2024-03-14T12:16:01.678183+0000","last_unstale":"2024-03-14T12:16:02.697644+0000","last_undegraded":"2024-03-14T12:16:02.697644+0000","last_fullsized":"2024-03-14T12:16:02.697644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:34:45.933291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.683656+0000","last_change":"2024-03-14T12:16:02.680131+0000","last_active":"2024-03-14T12:16:02.683656+0000","last_peered":"2024-03-14T12:16:02.683656+0000","last_clean":"2024-03-14T12:16:02.683656+0000","last_became_active":"2024-03-14T12:16:02.679858+0000","last_became_peered":"2024-03-14T12:16:02.679858+0000","last_unstale":"2024-03-14T12:16:02.683656+0000","last_undegraded":"2024-03-14T12:16:02.683656+0000","last_fullsized":"2024-03-14T12:16:02.683656+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:38:16.788654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.666384+0000","last_change":"2024-03-14T12:16:01.680638+0000","last_active":"2024-03-14T12:16:02.666384+0000","last_peered":"2024-03-14T12:16:02.666384+0000","last_clean":"2024-03-14T12:16:02.666384+0000","last_became_active":"2024-03-14T12:16:01.680340+0000","last_became_peered":"2024-03-14T12:16:01.680340+0000","last_unstale":"2024-03-14T12:16:02.666384+0000","last_undegraded":"2024-03-14T12:16:02.666384+0000","last_fullsized":"2024-03-14T12:16:02.666384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:02:09.736541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.691713+0000","last_change":"2024-03-14T12:16:02.689233+0000","last_active":"2024-03-14T12:16:02.691713+0000","last_peered":"2024-03-14T12:16:02.691713+0000","last_clean":"2024-03-14T12:16:02.691713+0000","last_became_active":"2024-03-14T12:16:02.688927+0000","last_became_peered":"2024-03-14T12:16:02.688927+0000","last_unstale":"2024-03-14T12:16:02.691713+0000","last_undegraded":"2024-03-14T12:16:02.691713+0000","last_fullsized":"2024-03-14T12:16:02.691713+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:49:43.398827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.675521+0000","last_change":"2024-03-14T12:16:01.679687+0000","last_active":"2024-03-14T12:16:02.675521+0000","last_peered":"2024-03-14T12:16:02.675521+0000","last_clean":"2024-03-14T12:16:02.675521+0000","last_became_active":"2024-03-14T12:16:01.679231+0000","last_became_peered":"2024-03-14T12:16:01.679231+0000","last_unstale":"2024-03-14T12:16:02.675521+0000","last_undegraded":"2024-03-14T12:16:02.675521+0000","last_fullsized":"2024-03-14T12:16:02.675521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:28.563364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697617+0000","last_change":"2024-03-14T12:16:02.690300+0000","last_active":"2024-03-14T12:16:02.697617+0000","last_peered":"2024-03-14T12:16:02.697617+0000","last_clean":"2024-03-14T12:16:02.697617+0000","last_became_active":"2024-03-14T12:16:02.690002+0000","last_became_peered":"2024-03-14T12:16:02.690002+0000","last_unstale":"2024-03-14T12:16:02.697617+0000","last_undegraded":"2024-03-14T12:16:02.697617+0000","last_fullsized":"2024-03-14T12:16:02.697617+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:48.005671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676142+0000","last_change":"2024-03-14T12:16:01.679089+0000","last_active":"2024-03-14T12:16:02.676142+0000","last_peered":"2024-03-14T12:16:02.676142+0000","last_clean":"2024-03-14T12:16:02.676142+0000","last_became_active":"2024-03-14T12:16:01.678794+0000","last_became_peered":"2024-03-14T12:16:01.678794+0000","last_unstale":"2024-03-14T12:16:02.676142+0000","last_undegraded":"2024-03-14T12:16:02.676142+0000","last_fullsized":"2024-03-14T12:16:02.676142+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:48.054785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.695206+0000","last_change":"2024-03-14T12:16:02.694501+0000","last_active":"2024-03-14T12:16:02.695206+0000","last_peered":"2024-03-14T12:16:02.695206+0000","last_clean":"2024-03-14T12:16:02.695206+0000","last_became_active":"2024-03-14T12:16:02.694193+0000","last_became_peered":"2024-03-14T12:16:02.694193+0000","last_unstale":"2024-03-14T12:16:02.695206+0000","last_undegraded":"2024-03-14T12:16:02.695206+0000","last_fullsized":"2024-03-14T12:16:02.695206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:40:05.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.674170+0000","last_change":"2024-03-14T12:16:01.681204+0000","last_active":"2024-03-14T12:16:02.674170+0000","last_peered":"2024-03-14T12:16:02.674170+0000","last_clean":"2024-03-14T12:16:02.674170+0000","last_became_active":"2024-03-14T12:16:01.680728+0000","last_became_peered":"2024-03-14T12:16:01.680728+0000","last_unstale":"2024-03-14T12:16:02.674170+0000","last_undegraded":"2024-03-14T12:16:02.674170+0000","last_fullsized":"2024-03-14T12:16:02.674170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:01:04.190553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692027+0000","last_change":"2024-03-14T12:16:02.681391+0000","last_active":"2024-03-14T12:16:02.692027+0000","last_peered":"2024-03-14T12:16:02.692027+0000","last_clean":"2024-03-14T12:16:02.692027+0000","last_became_active":"2024-03-14T12:16:02.681128+0000","last_became_peered":"2024-03-14T12:16:02.681128+0000","last_unstale":"2024-03-14T12:16:02.692027+0000","last_undegraded":"2024-03-14T12:16:02.692027+0000","last_fullsized":"2024-03-14T12:16:02.692027+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:48.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.675890+0000","last_change":"2024-03-14T12:16:01.683639+0000","last_active":"2024-03-14T12:16:02.675890+0000","last_peered":"2024-03-14T12:16:02.675890+0000","last_clean":"2024-03-14T12:16:02.675890+0000","last_became_active":"2024-03-14T12:16:01.683285+0000","last_became_peered":"2024-03-14T12:16:01.683285+0000","last_unstale":"2024-03-14T12:16:02.675890+0000","last_undegraded":"2024-03-14T12:16:02.675890+0000","last_fullsized":"2024-03-14T12:16:02.675890+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:54.293322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.692977+0000","last_change":"2024-03-14T12:16:02.692090+0000","last_active":"2024-03-14T12:16:02.692977+0000","last_peered":"2024-03-14T12:16:02.692977+0000","last_clean":"2024-03-14T12:16:02.692977+0000","last_became_active":"2024-03-14T12:16:02.691667+0000","last_became_peered":"2024-03-14T12:16:02.691667+0000","last_unstale":"2024-03-14T12:16:02.692977+0000","last_undegraded":"2024-03-14T12:16:02.692977+0000","last_fullsized":"2024-03-14T12:16:02.692977+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:01:05.199522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.670038+0000","last_change":"2024-03-14T12:16:01.678519+0000","last_active":"2024-03-14T12:16:02.670038+0000","last_peered":"2024-03-14T12:16:02.670038+0000","last_clean":"2024-03-14T12:16:02.670038+0000","last_became_active":"2024-03-14T12:16:01.678242+0000","last_became_peered":"2024-03-14T12:16:01.678242+0000","last_unstale":"2024-03-14T12:16:02.670038+0000","last_undegraded":"2024-03-14T12:16:02.670038+0000","last_fullsized":"2024-03-14T12:16:02.670038+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:47.065534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.684610+0000","last_change":"2024-03-14T12:16:02.677835+0000","last_active":"2024-03-14T12:16:02.684610+0000","last_peered":"2024-03-14T12:16:02.684610+0000","last_clean":"2024-03-14T12:16:02.684610+0000","last_became_active":"2024-03-14T12:16:02.677519+0000","last_became_peered":"2024-03-14T12:16:02.677519+0000","last_unstale":"2024-03-14T12:16:02.684610+0000","last_undegraded":"2024-03-14T12:16:02.684610+0000","last_fullsized":"2024-03-14T12:16:02.684610+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:57:57.012520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676235+0000","last_change":"2024-03-14T12:16:01.681629+0000","last_active":"2024-03-14T12:16:02.676235+0000","last_peered":"2024-03-14T12:16:02.676235+0000","last_clean":"2024-03-14T12:16:02.676235+0000","last_became_active":"2024-03-14T12:16:01.681318+0000","last_became_peered":"2024-03-14T12:16:01.681318+0000","last_unstale":"2024-03-14T12:16:02.676235+0000","last_undegraded":"2024-03-14T12:16:02.676235+0000","last_fullsized":"2024-03-14T12:16:02.676235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:24:56.814661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.698118+0000","last_change":"2024-03-14T12:16:02.696292+0000","last_active":"2024-03-14T12:16:02.698118+0000","last_peered":"2024-03-14T12:16:02.698118+0000","last_clean":"2024-03-14T12:16:02.698118+0000","last_became_active":"2024-03-14T12:16:02.696004+0000","last_became_peered":"2024-03-14T12:16:02.696004+0000","last_unstale":"2024-03-14T12:16:02.698118+0000","last_undegraded":"2024-03-14T12:16:02.698118+0000","last_fullsized":"2024-03-14T12:16:02.698118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:01.324877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.667236+0000","last_change":"2024-03-14T12:16:01.682514+0000","last_active":"2024-03-14T12:16:02.667236+0000","last_peered":"2024-03-14T12:16:02.667236+0000","last_clean":"2024-03-14T12:16:02.667236+0000","last_became_active":"2024-03-14T12:16:01.682217+0000","last_became_peered":"2024-03-14T12:16:01.682217+0000","last_unstale":"2024-03-14T12:16:02.667236+0000","last_undegraded":"2024-03-14T12:16:02.667236+0000","last_fullsized":"2024-03-14T12:16:02.667236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:47.065534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.678927+0000","last_change":"2024-03-14T12:16:02.671927+0000","last_active":"2024-03-14T12:16:02.678927+0000","last_peered":"2024-03-14T12:16:02.678927+0000","last_clean":"2024-03-14T12:16:02.678927+0000","last_became_active":"2024-03-14T12:16:02.671597+0000","last_became_peered":"2024-03-14T12:16:02.671597+0000","last_unstale":"2024-03-14T12:16:02.678927+0000","last_undegraded":"2024-03-14T12:16:02.678927+0000","last_fullsized":"2024-03-14T12:16:02.678927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:49.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.670053+0000","last_change":"2024-03-14T12:16:01.677577+0000","last_active":"2024-03-14T12:16:02.670053+0000","last_peered":"2024-03-14T12:16:02.670053+0000","last_clean":"2024-03-14T12:16:02.670053+0000","last_became_active":"2024-03-14T12:16:01.677267+0000","last_became_peered":"2024-03-14T12:16:01.677267+0000","last_unstale":"2024-03-14T12:16:02.670053+0000","last_undegraded":"2024-03-14T12:16:02.670053+0000","last_fullsized":"2024-03-14T12:16:02.670053+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:20:42.359319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.686419+0000","last_change":"2024-03-14T12:16:02.682532+0000","last_active":"2024-03-14T12:16:02.686419+0000","last_peered":"2024-03-14T12:16:02.686419+0000","last_clean":"2024-03-14T12:16:02.686419+0000","last_became_active":"2024-03-14T12:16:02.681828+0000","last_became_peered":"2024-03-14T12:16:02.681828+0000","last_unstale":"2024-03-14T12:16:02.686419+0000","last_undegraded":"2024-03-14T12:16:02.686419+0000","last_fullsized":"2024-03-14T12:16:02.686419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:02:10.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676487+0000","last_change":"2024-03-14T12:16:01.682613+0000","last_active":"2024-03-14T12:16:02.676487+0000","last_peered":"2024-03-14T12:16:02.676487+0000","last_clean":"2024-03-14T12:16:02.676487+0000","last_became_active":"2024-03-14T12:16:01.682262+0000","last_became_peered":"2024-03-14T12:16:01.682262+0000","last_unstale":"2024-03-14T12:16:02.676487+0000","last_undegraded":"2024-03-14T12:16:02.676487+0000","last_fullsized":"2024-03-14T12:16:02.676487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:09:49.869686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.697791+0000","last_change":"2024-03-14T12:16:02.691387+0000","last_active":"2024-03-14T12:16:02.697791+0000","last_peered":"2024-03-14T12:16:02.697791+0000","last_clean":"2024-03-14T12:16:02.697791+0000","last_became_active":"2024-03-14T12:16:02.691127+0000","last_became_peered":"2024-03-14T12:16:02.691127+0000","last_unstale":"2024-03-14T12:16:02.697791+0000","last_undegraded":"2024-03-14T12:16:02.697791+0000","last_fullsized":"2024-03-14T12:16:02.697791+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:01.656069+0000","last_clean_scrub_stamp":"2024-03-14T12:16:01.656069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:02:10.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.696299+0000","last_change":"2024-03-14T12:16:01.682568+0000","last_active":"2024-03-14T12:16:02.696299+0000","last_peered":"2024-03-14T12:16:02.696299+0000","last_clean":"2024-03-14T12:16:02.696299+0000","last_became_active":"2024-03-14T12:16:01.682303+0000","last_became_peered":"2024-03-14T12:16:01.682303+0000","last_unstale":"2024-03-14T12:16:02.696299+0000","last_undegraded":"2024-03-14T12:16:02.696299+0000","last_fullsized":"2024-03-14T12:16:02.696299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:00.647100+0000","last_clean_scrub_stamp":"2024-03-14T12:16:00.647100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:54.293322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-03-14T12:16:02.676474+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:16:02.676474+0000","last_peered":"2024-03-14T12:16:02.676474+0000","last_clean":"2024-03-14T12:16:02.676474+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:16:02.676474+0000","last_undegraded":"2024-03-14T12:16:02.676474+0000","last_fullsized":"2024-03-14T12:16:02.676474+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":803375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":206142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":532480,"data_stored":212987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":540672,"data_stored":214529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35876,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335964,"statfs":{"total":96636764160,"available":96600027136,"internally_reserved":0,"allocated":540672,"data_stored":213555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":214551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":532480,"data_stored":212987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":803319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:16:05.966 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T12:16:05.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-14T12:16:06.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.110+0000 7f43ce189640 1 Processor -- start 2024-03-14T12:16:06.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.110+0000 7f43ce189640 1 -- start start 2024-03-14T12:16:06.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43ce189640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f43c80cfe30 0x7f43c8143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43ce189640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43c80d0210 0x7f43c8144520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43ce189640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c815e770 0x7f43c815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43ce189640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c80b34e0 con 0x7f43c80d0210 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43ce189640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c815f4a0 con 0x7f43c80cfe30 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43ce189640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80b3780 con 0x7f43c815e770 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43c80d0210 0x7f43c8144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43c80d0210 0x7f43c8144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37210/0 (socket says 172.21.15.82:37210) 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 -- 172.21.15.82:0/4196391659 learned_addr learned my addr 172.21.15.82:0/4196391659 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c7fff640 1 --2- 172.21.15.82:0/4196391659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c815e770 0x7f43c815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 -- 172.21.15.82:0/4196391659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c815e770 msgr2=0x7f43c815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 --2- 172.21.15.82:0/4196391659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c815e770 0x7f43c815ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 -- 172.21.15.82:0/4196391659 >> v2:172.21.15.164:3300/0 conn(0x7f43c80cfe30 msgr2=0x7f43c8143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c77fe640 1 --2- 172.21.15.82:0/4196391659 >> v2:172.21.15.164:3300/0 conn(0x7f43c80cfe30 0x7f43c8143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 --2- 172.21.15.82:0/4196391659 >> v2:172.21.15.164:3300/0 conn(0x7f43c80cfe30 0x7f43c8143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.111+0000 7f43c6ffd640 1 -- 172.21.15.82:0/4196391659 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80b3a60 con 0x7f43c80d0210 2024-03-14T12:16:06.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.112+0000 7f43c6ffd640 1 --2- 172.21.15.82:0/4196391659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80d0210 0x7f43c8144520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f43c80d0a60 tx=0x7f43bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33395c0b28aef405 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:06.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.112+0000 7f43c67fc640 1 -- 172.21.15.82:0/4196391659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc035020 con 0x7f43c80d0210 2024-03-14T12:16:06.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.112+0000 7f43c67fc640 1 -- 172.21.15.82:0/4196391659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc031870 con 0x7f43c80d0210 2024-03-14T12:16:06.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.112+0000 7f43c67fc640 1 -- 172.21.15.82:0/4196391659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc031c40 con 0x7f43c80d0210 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 -- 172.21.15.82:0/4196391659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80d0210 msgr2=0x7f43c8144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 --2- 172.21.15.82:0/4196391659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80d0210 0x7f43c8144520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f43c80d0a60 tx=0x7f43bc02b910 comp rx=0 tx=0).stop 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 -- 172.21.15.82:0/4196391659 shutdown_connections 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 --2- 172.21.15.82:0/4196391659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c815e770 0x7f43c815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 --2- 172.21.15.82:0/4196391659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80d0210 0x7f43c8144520 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 --2- 172.21.15.82:0/4196391659 >> v2:172.21.15.164:3300/0 conn(0x7f43c80cfe30 0x7f43c8143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 -- 172.21.15.82:0/4196391659 >> 172.21.15.82:0/4196391659 conn(0x7f43c806aef0 msgr2=0x7f43c806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 -- 172.21.15.82:0/4196391659 shutdown_connections 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.113+0000 7f43ce189640 1 -- 172.21.15.82:0/4196391659 wait complete. 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.114+0000 7f43ce189640 1 Processor -- start 2024-03-14T12:16:06.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.114+0000 7f43ce189640 1 -- start start 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43ce189640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80cfe30 0x7f43c80b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43ce189640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c80d0210 0x7f43c81433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43ce189640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43c815e770 0x7f43c815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43ce189640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80b4260 con 0x7f43c80cfe30 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43ce189640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c815f110 con 0x7f43c815e770 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43ce189640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80b43e0 con 0x7f43c80d0210 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80cfe30 0x7f43c80b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80cfe30 0x7f43c80b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37226/0 (socket says 172.21.15.82:37226) 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 -- 172.21.15.82:0/2769999170 learned_addr learned my addr 172.21.15.82:0/2769999170 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c6ffd640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c80d0210 0x7f43c81433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c7fff640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43c815e770 0x7f43c815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 -- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c80d0210 msgr2=0x7f43c81433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c80d0210 0x7f43c81433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 -- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43c815e770 msgr2=0x7f43c815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43c815e770 0x7f43c815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c77fe640 1 -- 172.21.15.82:0/2769999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80b1890 con 0x7f43c80cfe30 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c6ffd640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c80d0210 0x7f43c81433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.115+0000 7f43c7fff640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43c815e770 0x7f43c815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.116+0000 7f43c77fe640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80cfe30 0x7f43c80b2610 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c570 tx=0x7f43b800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.116+0000 7f43affff640 1 -- 172.21.15.82:0/2769999170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8015070 con 0x7f43c80cfe30 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.116+0000 7f43affff640 1 -- 172.21.15.82:0/2769999170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b8011800 con 0x7f43c80cfe30 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.116+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c80b1b30 con 0x7f43c80cfe30 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.116+0000 7f43affff640 1 -- 172.21.15.82:0/2769999170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8011bf0 con 0x7f43c80cfe30 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.116+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c80b20f0 con 0x7f43c80cfe30 2024-03-14T12:16:06.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.117+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c804f050 con 0x7f43c80cfe30 2024-03-14T12:16:06.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.117+0000 7f43affff640 1 -- 172.21.15.82:0/2769999170 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f43b8018070 con 0x7f43c80cfe30 2024-03-14T12:16:06.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.118+0000 7f43affff640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f43b406bbd0 0x7f43b406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.118+0000 7f43affff640 1 -- 172.21.15.82:0/2769999170 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43b808a820 con 0x7f43c80cfe30 2024-03-14T12:16:06.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.118+0000 7f43c6ffd640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f43b406bbd0 0x7f43b406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.118+0000 7f43c6ffd640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f43b406bbd0 0x7f43b406e090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f43bc036000 tx=0x7f43bc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:06.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.122+0000 7f43affff640 1 -- 172.21.15.82:0/2769999170 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f43b8059310 con 0x7f43c80cfe30 2024-03-14T12:16:06.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.359+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f43c806d300 con 0x7f43c80cfe30 2024-03-14T12:16:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.362+0000 7f43affff640 1 -- 172.21.15.82:0/2769999170 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f43b8058c10 con 0x7f43c80cfe30 2024-03-14T12:16:06.361 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-03-14T12:16:06.361 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg== 2024-03-14T12:16:06.361 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-03-14T12:16:06.361 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-03-14T12:16:06.361 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-14T12:16:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-03-14T12:16:06.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f43b406bbd0 msgr2=0x7f43b406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f43b406bbd0 0x7f43b406e090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f43bc036000 tx=0x7f43bc008030 comp rx=0 tx=0).stop 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80cfe30 msgr2=0x7f43c80b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80cfe30 0x7f43c80b2610 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c570 tx=0x7f43b800ca30 comp rx=0 tx=0).stop 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 shutdown_connections 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43c815e770 0x7f43c815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f43b406bbd0 0x7f43b406e090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43c80d0210 0x7f43c81433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 --2- 172.21.15.82:0/2769999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43c80cfe30 0x7f43c80b2610 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 >> 172.21.15.82:0/2769999170 conn(0x7f43c806aef0 msgr2=0x7f43c81541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 shutdown_connections 2024-03-14T12:16:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.364+0000 7f43ce189640 1 -- 172.21.15.82:0/2769999170 wait complete. 2024-03-14T12:16:06.374 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:06.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.475+0000 7f0087fb9640 1 Processor -- start 2024-03-14T12:16:06.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.475+0000 7f0087fb9640 1 -- start start 2024-03-14T12:16:06.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0087fb9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00800b5710 0x7f008015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0087fb9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008015dea0 con 0x7f008006ea40 2024-03-14T12:16:06.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0087fb9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008015e150 con 0x7f00800b5af0 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0087fb9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008015e430 con 0x7f00800b5710 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0085d55640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00800b5710 0x7f008015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0086556640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f008006ea40 0x7f008006ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36404/0 (socket says 172.21.15.82:36404) 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0085d55640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00800b5710 0x7f008015f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37242/0 (socket says 172.21.15.82:37242) 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0086556640 1 -- 172.21.15.82:0/580409625 learned_addr learned my addr 172.21.15.82:0/580409625 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0085d55640 1 -- 172.21.15.82:0/580409625 >> v1:172.21.15.164:6789/0 conn(0x7f00800b5af0 legacy=0x7f008015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0085d55640 1 -- 172.21.15.82:0/580409625 >> v1:172.21.15.82:6789/0 conn(0x7f008006ea40 legacy=0x7f008006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0085d55640 1 -- 172.21.15.82:0/580409625 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008015e710 con 0x7f00800b5710 2024-03-14T12:16:06.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.476+0000 7f0084d53640 1 -- 172.21.15.82:0/580409625 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321734154 0 0) 0x7f008015dea0 con 0x7f008006ea40 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0085d55640 1 --2- 172.21.15.82:0/580409625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800b5710 0x7f008015f1f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f007c0097a0 tx=0x7f007c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba9f6409824e03b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0084d53640 1 -- 172.21.15.82:0/580409625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c035020 con 0x7f00800b5710 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0084d53640 1 -- 172.21.15.82:0/580409625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c031830 con 0x7f00800b5710 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0084d53640 1 -- 172.21.15.82:0/580409625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c031c00 con 0x7f00800b5710 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0087fb9640 1 -- 172.21.15.82:0/580409625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800b5710 msgr2=0x7f008015f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0087fb9640 1 --2- 172.21.15.82:0/580409625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800b5710 0x7f008015f1f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f007c0097a0 tx=0x7f007c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0087fb9640 1 -- 172.21.15.82:0/580409625 shutdown_connections 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0087fb9640 1 --2- 172.21.15.82:0/580409625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00800b5710 0x7f008015f1f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.477+0000 7f0087fb9640 1 -- 172.21.15.82:0/580409625 >> 172.21.15.82:0/580409625 conn(0x7f0080072ee0 msgr2=0x7f008009a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.478+0000 7f0087fb9640 1 -- 172.21.15.82:0/580409625 shutdown_connections 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.478+0000 7f0087fb9640 1 -- 172.21.15.82:0/580409625 wait complete. 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 Processor -- start 2024-03-14T12:16:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 -- start start 2024-03-14T12:16:06.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008006ea40 0x7f008014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800b5710 0x7f0080152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800b5af0 0x7f00800ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800b5f80 con 0x7f008006ea40 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008015ecd0 con 0x7f00800b5af0 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0087fb9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0080156650 con 0x7f00800b5710 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.479+0000 7f0085d55640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008006ea40 0x7f008014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008006ea40 0x7f008014f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37244/0 (socket says 172.21.15.82:37244) 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 -- 172.21.15.82:0/316045583 learned_addr learned my addr 172.21.15.82:0/316045583 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 -- 172.21.15.82:0/316045583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800b5710 msgr2=0x7f0080152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800b5710 0x7f0080152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 -- 172.21.15.82:0/316045583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800b5af0 msgr2=0x7f00800ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800b5af0 0x7f00800ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 -- 172.21.15.82:0/316045583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080150be0 con 0x7f008006ea40 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0085d55640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008006ea40 0x7f008014f200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f007c0098c0 tx=0x7f007c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f00767fc640 1 -- 172.21.15.82:0/316045583 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c03ba80 con 0x7f008006ea40 2024-03-14T12:16:06.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f00767fc640 1 -- 172.21.15.82:0/316045583 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c03bc20 con 0x7f008006ea40 2024-03-14T12:16:06.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f00767fc640 1 -- 172.21.15.82:0/316045583 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0453f0 con 0x7f008006ea40 2024-03-14T12:16:06.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0080150ec0 con 0x7f008006ea40 2024-03-14T12:16:06.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.480+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0080151480 con 0x7f008006ea40 2024-03-14T12:16:06.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.481+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008004f050 con 0x7f008006ea40 2024-03-14T12:16:06.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.482+0000 7f00767fc640 1 -- 172.21.15.82:0/316045583 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f007c04c060 con 0x7f008006ea40 2024-03-14T12:16:06.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.482+0000 7f00767fc640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006406bbd0 0x7f006406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:06.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.482+0000 7f00767fc640 1 -- 172.21.15.82:0/316045583 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f007c0ac7f0 con 0x7f008006ea40 2024-03-14T12:16:06.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.482+0000 7f0085554640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006406bbd0 0x7f006406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:06.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.482+0000 7f0085554640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006406bbd0 0x7f006406e090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070006bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:06.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.486+0000 7f00767fc640 1 -- 172.21.15.82:0/316045583 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f007c040070 con 0x7f008006ea40 2024-03-14T12:16:06.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.697+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00800d29d0 con 0x7f008006ea40 2024-03-14T12:16:06.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.698+0000 7f00767fc640 1 -- 172.21.15.82:0/316045583 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f007c02f2e0 con 0x7f008006ea40 2024-03-14T12:16:06.697 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:06.697 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1414844632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":1414844632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1414844632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/456693586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":456693586},{"type":"v1","addr":"172.21.15.164:6834","nonce":456693586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/58260684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":58260684},{"type":"v1","addr":"172.21.15.82:6836","nonce":58260684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:02.017303+0000","modified":"2024-03-14T12:16:04.014283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2708572159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2708572159},{"type":"v1","addr":"172.21.15.82:6837","nonce":2708572159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T12:16:06.697 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 21 2024-03-14T12:16:06.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.699+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006406bbd0 msgr2=0x7f006406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006406bbd0 0x7f006406e090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0070006fd0 tx=0x7f0070006bc0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008006ea40 msgr2=0x7f008014f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:06.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008006ea40 0x7f008014f200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f007c0098c0 tx=0x7f007c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 shutdown_connections 2024-03-14T12:16:06.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800b5af0 0x7f00800ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f006406bbd0 0x7f006406e090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800b5710 0x7f0080152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 --2- 172.21.15.82:0/316045583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008006ea40 0x7f008014f200 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:06.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 >> 172.21.15.82:0/316045583 conn(0x7f0080072ee0 msgr2=0x7f00800b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:06.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 shutdown_connections 2024-03-14T12:16:06.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:06.700+0000 7f0087fb9640 1 -- 172.21.15.82:0/316045583 wait complete. 2024-03-14T12:16:07.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T12:16:02.017303+0000', 'modified': '2024-03-14T12:16:04.014283+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5544}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5544': {'gid': 5544, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/2708572159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 2708572159}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2708572159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T12:16:07.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T12:16:07.113 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-14T12:16:07.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.218+0000 7f020dbdf640 1 Processor -- start 2024-03-14T12:16:07.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.218+0000 7f020dbdf640 1 -- start start 2024-03-14T12:16:07.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f020dbdf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02080b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f020dbdf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0208143f60 0x7f020815b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f020dbdf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020815ebe0 con 0x7f0208071740 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f020dbdf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020815eec0 con 0x7f0208143f60 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f020dbdf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020815f1a0 con 0x7f020815e670 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0207fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0208143f60 0x7f020815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0207fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0208143f60 0x7f020815b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37252/0 (socket says 172.21.15.82:37252) 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0207fff640 1 -- 172.21.15.82:0/2237044624 learned_addr learned my addr 172.21.15.82:0/2237044624 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0206ffd640 1 --2- 172.21.15.82:0/2237044624 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02080b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0207fff640 1 -- 172.21.15.82:0/2237044624 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 msgr2=0x7f02080b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0207fff640 1 --2- 172.21.15.82:0/2237044624 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02080b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0207fff640 1 -- 172.21.15.82:0/2237044624 >> v1:172.21.15.164:6789/0 conn(0x7f0208071740 legacy=0x7f0208071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.219+0000 7f0207fff640 1 -- 172.21.15.82:0/2237044624 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020815f480 con 0x7f0208143f60 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.220+0000 7f0207fff640 1 --2- 172.21.15.82:0/2237044624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f020815b610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0200009e50 tx=0x7f020002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6587ec44e73309b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.220+0000 7f02067fc640 1 -- 172.21.15.82:0/2237044624 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020002cb00 con 0x7f0208143f60 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.220+0000 7f02067fc640 1 -- 172.21.15.82:0/2237044624 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020002cca0 con 0x7f0208143f60 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.220+0000 7f02067fc640 1 -- 172.21.15.82:0/2237044624 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200034610 con 0x7f0208143f60 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 -- 172.21.15.82:0/2237044624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 msgr2=0x7f020815b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 --2- 172.21.15.82:0/2237044624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f020815b610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0200009e50 tx=0x7f020002bf60 comp rx=0 tx=0).stop 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 -- 172.21.15.82:0/2237044624 shutdown_connections 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 --2- 172.21.15.82:0/2237044624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f020815b610 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 --2- 172.21.15.82:0/2237044624 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02080b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 -- 172.21.15.82:0/2237044624 >> 172.21.15.82:0/2237044624 conn(0x7f02080b1c70 msgr2=0x7f02080cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 -- 172.21.15.82:0/2237044624 shutdown_connections 2024-03-14T12:16:07.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.221+0000 7f020dbdf640 1 -- 172.21.15.82:0/2237044624 wait complete. 2024-03-14T12:16:07.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.222+0000 7f020dbdf640 1 Processor -- start 2024-03-14T12:16:07.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.222+0000 7f020dbdf640 1 -- start start 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f020dbdf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0208071740 0x7f0208133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f020dbdf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f0208135380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f020dbdf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02081306a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f020dbdf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0208144670 con 0x7f0208143f60 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f020dbdf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020815b080 con 0x7f0208071740 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f020dbdf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02081447f0 con 0x7f020815e670 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f0208135380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f0208135380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37254/0 (socket says 172.21.15.82:37254) 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 -- 172.21.15.82:0/1225200454 learned_addr learned my addr 172.21.15.82:0/1225200454 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f02077fe640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0208071740 0x7f0208133b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0207fff640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02081306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 -- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 msgr2=0x7f02081306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02081306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 -- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0208071740 msgr2=0x7f0208133b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0208071740 0x7f0208133b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 -- 172.21.15.82:0/1225200454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208134920 con 0x7f0208143f60 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0207fff640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02081306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.223+0000 7f0206ffd640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f0208135380 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d830 tx=0x7f01fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.224+0000 7f01effff640 1 -- 172.21.15.82:0/1225200454 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00b6c0 con 0x7f0208143f60 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.224+0000 7f01effff640 1 -- 172.21.15.82:0/1225200454 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc019070 con 0x7f0208143f60 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.224+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0208134c00 con 0x7f0208143f60 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.224+0000 7f01effff640 1 -- 172.21.15.82:0/1225200454 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00ba80 con 0x7f0208143f60 2024-03-14T12:16:07.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.224+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020815cbd0 con 0x7f0208143f60 2024-03-14T12:16:07.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.225+0000 7f01effff640 1 -- 172.21.15.82:0/1225200454 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f01fc01e460 con 0x7f0208143f60 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.226+0000 7f01effff640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f01f406bbd0 0x7f01f406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.226+0000 7f01effff640 1 -- 172.21.15.82:0/1225200454 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01fc08b020 con 0x7f0208143f60 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.226+0000 7f02077fe640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f01f406bbd0 0x7f01f406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.226+0000 7f02077fe640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f01f406bbd0 0x7f01f406e090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0208136290 tx=0x7f01f8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.227+0000 7f02107c0740 1 Processor -- start 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.227+0000 7f02107c0740 1 -- start start 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.227+0000 7f02107c0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab75e0 0x55fef5ade110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ade650 0x55fef5ab6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab79c0 0x55fef5ab6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02107c0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef5a1b7c0 con 0x55fef5ab75e0 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef59c4b50 con 0x55fef5ab79c0 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef5a17860 con 0x55fef5ade650 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab75e0 0x55fef5ade110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab75e0 0x55fef5ade110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37270/0 (socket says 172.21.15.82:37270) 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 -- 172.21.15.82:0/2001130259 learned_addr learned my addr 172.21.15.82:0/2001130259 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 -- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ade650 msgr2=0x55fef5ab6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ade650 0x55fef5ab6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 -- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab79c0 msgr2=0x55fef5ab6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f0207fff640 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab79c0 0x55fef5ab6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab79c0 0x55fef5ab6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 -- 172.21.15.82:0/2001130259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fef59d89a0 con 0x55fef5ab75e0 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f0207fff640 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab79c0 0x55fef5ab6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f02077fe640 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab75e0 0x55fef5ade110 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f01f800f950 tx=0x7f01f8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b540acfca885 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f01ed7fa640 1 -- 172.21.15.82:0/2001130259 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8025070 con 0x55fef5ab75e0 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f01ed7fa640 1 -- 172.21.15.82:0/2001130259 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f801b050 con 0x55fef5ab75e0 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.228+0000 7f01ed7fa640 1 -- 172.21.15.82:0/2001130259 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8014de0 con 0x55fef5ab75e0 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 -- 172.21.15.82:0/2001130259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab75e0 msgr2=0x55fef5ade110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab75e0 0x55fef5ade110 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f01f800f950 tx=0x7f01f8012000 comp rx=0 tx=0).stop 2024-03-14T12:16:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 -- 172.21.15.82:0/2001130259 shutdown_connections 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab79c0 0x55fef5ab6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ade650 0x55fef5ab6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 --2- 172.21.15.82:0/2001130259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab75e0 0x55fef5ade110 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 -- 172.21.15.82:0/2001130259 >> 172.21.15.82:0/2001130259 conn(0x55fef5a7e400 msgr2=0x55fef5afc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 -- 172.21.15.82:0/2001130259 shutdown_connections 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.229+0000 7f02107c0740 1 -- 172.21.15.82:0/2001130259 wait complete. 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 -- start start 2024-03-14T12:16:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 Processor -- start 2024-03-14T12:16:07.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 0x55fef5ae4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5ae4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef5addc80 con 0x55fef5ad5a20 2024-03-14T12:16:07.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef59d7f90 con 0x55fef5ab75e0 2024-03-14T12:16:07.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef5ad6a00 con 0x55fef5ab79c0 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f02107c0740 10 client.0 _ll_register_callbacks cb 0x55fef5a13400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f0207fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5ae4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f0207fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5ae4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37286/0 (socket says 172.21.15.82:37286) 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.230+0000 7f0207fff640 1 -- 172.21.15.82:0/709930960 learned_addr learned my addr 172.21.15.82:0/709930960 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f02077fe640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f0206ffd640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 0x55fef5ae4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f0207fff640 1 -- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 msgr2=0x55fef5ae4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f0207fff640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 0x55fef5ae4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f0207fff640 1 -- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 msgr2=0x55fef5b0fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f0207fff640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b0fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f0207fff640 1 -- 172.21.15.82:0/709930960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fef57302d0 con 0x55fef5ad5a20 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f02077fe640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b0fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f0207fff640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5ae4c50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0200036040 tx=0x7f0200004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354a390f62a826f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f01e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f01e67fc640 1 -- 172.21.15.82:0/709930960 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200009510 con 0x55fef5ad5a20 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f01e67fc640 1 -- 172.21.15.82:0/709930960 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0200034c30 con 0x55fef5ad5a20 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01f801b050 con 0x55fef5ad5a20 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f01e67fc640 1 -- 172.21.15.82:0/709930960 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0200050050 con 0x55fef5ad5a20 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f02107c0740 10 client.5694 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55fef57fa6b0 con 0x55fef5ad5a20 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f01e67fc640 1 -- 172.21.15.82:0/709930960 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f801b050 con 0x55fef5ad5a20 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.231+0000 7f01e67fc640 1 -- 172.21.15.82:0/709930960 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0200053020 con 0x55fef5ad5a20 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02107c0740 10 client.5694 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02107c0740 10 client.5694 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02107c0740 20 client.5694 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02107c0740 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5b0cb90 0x55fef5b0cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02107c0740 4 client.5694 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:16:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fef5a28d00 con 0x55fef5b0cb90 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02077fe640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5b0cb90 0x55fef5b0cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f02077fe640 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5b0cb90 0x55fef5b0cf70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f4db519b05e775 server_cookie=4e7c5b4a52aada24 in_seq=0 out_seq=0 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.232+0000 7f01e67fc640 10 client.5694 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f01e67fc640 1 -- 172.21.15.82:0/709930960 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fef5a28d00 con 0x55fef5b0cb90 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f01e67fc640 10 client.5694 handle_command_reply: tid=0 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 client.5694 shutdown 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 msgr2=0x55fef5ae4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5ae4c50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0200036040 tx=0x7f0200004ab0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 shutdown_connections 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5ae4c50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 0x55fef5ae4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5b0cb90 0x55fef5b0cf70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 --2- 172.21.15.82:0/709930960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b0fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f01e67fc640 0 client.5694 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 >> 172.21.15.82:0/709930960 conn(0x55fef5a7e400 msgr2=0x55fef5add290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 shutdown_connections 2024-03-14T12:16:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 1 -- 172.21.15.82:0/709930960 wait complete. 2024-03-14T12:16:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.233+0000 7f02107c0740 20 client.5694 trim_cache size 0 max 16384 2024-03-14T12:16:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.248+0000 7f02107c0740 1 Processor -- start 2024-03-14T12:16:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.248+0000 7f02107c0740 1 -- start start 2024-03-14T12:16:07.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5ae1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 0x55fef5affb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f02107c0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5b000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f02107c0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020815b550 con 0x55fef5ad5a20 2024-03-14T12:16:07.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02081f6970 con 0x55fef5ab75e0 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020815d930 con 0x55fef5ab79c0 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f0207fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5b000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f0207fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5b000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37292/0 (socket says 172.21.15.82:37292) 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.249+0000 7f0207fff640 1 -- 172.21.15.82:0/404918217 learned_addr learned my addr 172.21.15.82:0/404918217 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f0207fff640 1 -- 172.21.15.82:0/404918217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 msgr2=0x55fef5affb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f0207fff640 1 --2- 172.21.15.82:0/404918217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 0x55fef5affb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f0207fff640 1 -- 172.21.15.82:0/404918217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 msgr2=0x55fef5ae1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f0207fff640 1 --2- 172.21.15.82:0/404918217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5ae1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f0207fff640 1 -- 172.21.15.82:0/404918217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fef5ab27b0 con 0x55fef5ad5a20 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f0207fff640 1 --2- 172.21.15.82:0/404918217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5b000a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0200009c70 tx=0x7f020003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b695ec23ff2f76e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f01ecff9640 1 -- 172.21.15.82:0/404918217 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020002cca0 con 0x55fef5ad5a20 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f01ecff9640 1 -- 172.21.15.82:0/404918217 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0200058070 con 0x55fef5ad5a20 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f01ecff9640 1 -- 172.21.15.82:0/404918217 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200034a30 con 0x55fef5ad5a20 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f02107c0740 1 -- 172.21.15.82:0/404918217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 msgr2=0x55fef5b000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.250+0000 7f02107c0740 1 --2- 172.21.15.82:0/404918217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5b000a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0200009c70 tx=0x7f020003f000 comp rx=0 tx=0).stop 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 -- 172.21.15.82:0/404918217 shutdown_connections 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 --2- 172.21.15.82:0/404918217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ad5a20 0x55fef5b000a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 --2- 172.21.15.82:0/404918217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ab79c0 0x55fef5affb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 --2- 172.21.15.82:0/404918217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5ae1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 -- 172.21.15.82:0/404918217 >> 172.21.15.82:0/404918217 conn(0x55fef5a7e400 msgr2=0x55fef5b0cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 -- 172.21.15.82:0/404918217 shutdown_connections 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 -- 172.21.15.82:0/404918217 wait complete. 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 -- start start 2024-03-14T12:16:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.251+0000 7f02107c0740 1 Processor -- start 2024-03-14T12:16:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02107c0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab79c0 0x55fef5ad9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02107c0740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ad5a20 0x55fef5affb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02107c0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef5ae2450 con 0x55fef5ab79c0 2024-03-14T12:16:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef5af9e10 con 0x55fef5ab75e0 2024-03-14T12:16:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02107c0740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef5afad10 con 0x55fef5ad5a20 2024-03-14T12:16:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02107c0740 10 client.0 _ll_register_callbacks cb 0x7f01f8017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab79c0 0x55fef5ad9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab79c0 0x55fef5ad9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37296/0 (socket says 172.21.15.82:37296) 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 -- 172.21.15.82:0/2760142379 learned_addr learned my addr 172.21.15.82:0/2760142379 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f02077fe640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0207fff640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ad5a20 0x55fef5affb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 -- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ad5a20 msgr2=0x55fef5affb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ad5a20 0x55fef5affb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 -- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 msgr2=0x55fef5b653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0206ffd640 1 -- 172.21.15.82:0/2760142379 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fef5ab3030 con 0x55fef5ab79c0 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.252+0000 7f0207fff640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ad5a20 0x55fef5affb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f0206ffd640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab79c0 0x55fef5ad9df0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f01fc089600 tx=0x7f01fc058ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a80a0a4db9ad0e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f01e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f01e67fc640 1 -- 172.21.15.82:0/2760142379 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc05d5f0 con 0x55fef5ab79c0 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f01e67fc640 1 -- 172.21.15.82:0/2760142379 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01fc01fe00 con 0x55fef5ab79c0 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0200058070 con 0x55fef5ab79c0 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f01e67fc640 1 -- 172.21.15.82:0/2760142379 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01fc061bb0 con 0x55fef5ab79c0 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 10 client.5700 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55fef5ada330 con 0x55fef5ab79c0 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f01e67fc640 1 -- 172.21.15.82:0/2760142379 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0200058070 con 0x55fef5ab79c0 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f01e67fc640 1 -- 172.21.15.82:0/2760142379 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01fc013070 con 0x55fef5ab79c0 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 10 client.5700 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 10 client.5700 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 20 client.5700 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5af2a70 0x55fef5b76c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 4 client.5700 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fef5a23050 con 0x55fef5af2a70 2024-03-14T12:16:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02077fe640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5af2a70 0x55fef5b76c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f01e67fc640 10 client.5700 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.253+0000 7f02077fe640 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5af2a70 0x55fef5b76c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc583dbd9947f64 server_cookie=ac13444f998b7a1 in_seq=0 out_seq=0 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f01e67fc640 1 -- 172.21.15.82:0/2760142379 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fef5a23050 con 0x55fef5af2a70 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f01e67fc640 10 client.5700 handle_command_reply: tid=0 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 client.5700 shutdown 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab79c0 msgr2=0x55fef5ad9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab79c0 0x55fef5ad9df0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f01fc089600 tx=0x7f01fc058ef0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 shutdown_connections 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55fef5ad5a20 0x55fef5affb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fef5ab79c0 0x55fef5ad9df0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x55fef5af2a70 0x55fef5b76c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 --2- 172.21.15.82:0/2760142379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fef5ab75e0 0x55fef5b653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f01e67fc640 0 client.5700 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 >> 172.21.15.82:0/2760142379 conn(0x55fef5a7e400 msgr2=0x55fef5afaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 shutdown_connections 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 1 -- 172.21.15.82:0/2760142379 wait complete. 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f02107c0740 20 client.5700 trim_cache size 0 max 16384 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2", 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 5544, 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.5827372869999996, 2024-03-14T12:16:07.260 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 21, 2024-03-14T12:16:07.261 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 46, 2024-03-14T12:16:07.261 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 46, 2024-03-14T12:16:07.261 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 9.6591133730000003 2024-03-14T12:16:07.261 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f01f406bbd0 msgr2=0x7f01f406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f01f406bbd0 0x7f01f406e090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0208136290 tx=0x7f01f8009040 comp rx=0 tx=0).stop 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 msgr2=0x7f0208135380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f0208135380 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d830 tx=0x7f01fc00dcf0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 shutdown_connections 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f01f406bbd0 0x7f01f406e090 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f020815e670 0x7f02081306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0208143f60 0x7f0208135380 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 --2- 172.21.15.82:0/1225200454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0208071740 0x7f0208133b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.254+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 >> 172.21.15.82:0/1225200454 conn(0x7f02080b1c70 msgr2=0x7f02080ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.255+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 shutdown_connections 2024-03-14T12:16:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.255+0000 7f020dbdf640 1 -- 172.21.15.82:0/1225200454 wait complete. 2024-03-14T12:16:07.264 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:16:07.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.364+0000 7f5974819640 1 Processor -- start 2024-03-14T12:16:07.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.364+0000 7f5974819640 1 -- start start 2024-03-14T12:16:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.365+0000 7f5974819640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 0x7f597015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.365+0000 7f5974819640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59700b3050 con 0x7f5970071740 2024-03-14T12:16:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.365+0000 7f5974819640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597015ebe0 con 0x7f597015e670 2024-03-14T12:16:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.365+0000 7f5974819640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597015eec0 con 0x7f5970143f60 2024-03-14T12:16:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.365+0000 7f596e59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5970071740 0x7f5970071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36408/0 (socket says 172.21.15.82:36408) 2024-03-14T12:16:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.365+0000 7f596e59c640 1 -- 172.21.15.82:0/3608539336 learned_addr learned my addr 172.21.15.82:0/3608539336 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141780083 0 0) 0x7f59700b3050 con 0x7f5970071740 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596ed9d640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 0x7f597015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5958003520 con 0x7f5970071740 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730266762 0 0) 0x7f5958003520 con 0x7f5970071740 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 msgr2=0x7f597015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 0x7f597015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 >> v1:172.21.15.164:6789/0 conn(0x7f597015e670 legacy=0x7f59700b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597015f1a0 con 0x7f5970071740 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f59600019f0 con 0x7f5970071740 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 >> v1:172.21.15.82:6789/0 conn(0x7f5970071740 legacy=0x7f5970071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 0x7f59580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5958004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5958004d40 0x7f59580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597015eec0 con 0x7f5958003f60 2024-03-14T12:16:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595800baa0 con 0x7f5958004d40 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595800bca0 con 0x7f5970143f60 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596dd9b640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5958004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596e59c640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 0x7f59580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596ed9d640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5958004d40 0x7f59580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596dd9b640 1 -- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 msgr2=0x7f59580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596dd9b640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970143f60 0x7f59580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596dd9b640 1 -- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5958004d40 msgr2=0x7f59580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596dd9b640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5958004d40 0x7f59580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596dd9b640 1 -- 172.21.15.82:0/3608539336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958003520 con 0x7f5958003f60 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.366+0000 7f596ed9d640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5958004d40 0x7f59580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.367+0000 7f596dd9b640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5958004800 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5964001750 tx=0x7f5958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6dc18ccdc67d1d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.367+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964001370 con 0x7f5958003f60 2024-03-14T12:16:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.367+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59640052d0 con 0x7f5958003f60 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.367+0000 7f596d59a640 1 -- 172.21.15.82:0/3608539336 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964003830 con 0x7f5958003f60 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.368+0000 7f5974819640 1 -- 172.21.15.82:0/3608539336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 msgr2=0x7f5958004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.368+0000 7f5974819640 1 --2- 172.21.15.82:0/3608539336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5958004800 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5964001750 tx=0x7f5958005830 comp rx=0 tx=0).stop 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.368+0000 7f596ed9d640 1 -- 172.21.15.82:0/3608539336 reap_dead start 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.368+0000 7f5974819640 1 -- 172.21.15.82:0/3608539336 shutdown_connections 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.368+0000 7f5974819640 1 -- 172.21.15.82:0/3608539336 >> 172.21.15.82:0/3608539336 conn(0x7f59700b1c70 msgr2=0x7f59700cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.368+0000 7f5974819640 1 -- 172.21.15.82:0/3608539336 shutdown_connections 2024-03-14T12:16:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.368+0000 7f5974819640 1 -- 172.21.15.82:0/3608539336 wait complete. 2024-03-14T12:16:07.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.369+0000 7f5974819640 1 Processor -- start 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.369+0000 7f5974819640 1 -- start start 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f5974819640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5970135690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f5974819640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970158570 0x7f59701f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f5974819640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59701f29d0 0x7f59701f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f5974819640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597015b080 con 0x7f5958003f60 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f5974819640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5970143ce0 con 0x7f59701f29d0 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f5974819640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5970144670 con 0x7f5970158570 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5970135690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5970135690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37308/0 (socket says 172.21.15.82:37308) 2024-03-14T12:16:07.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 -- 172.21.15.82:0/2786685244 learned_addr learned my addr 172.21.15.82:0/2786685244 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:07.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596dd9b640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970158570 0x7f59701f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596ed9d640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59701f29d0 0x7f59701f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 -- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970158570 msgr2=0x7f59701f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970158570 0x7f59701f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 -- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59701f29d0 msgr2=0x7f59701f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59701f29d0 0x7f59701f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 -- 172.21.15.82:0/2786685244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59701f5d60 con 0x7f5958003f60 2024-03-14T12:16:07.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596ed9d640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59701f29d0 0x7f59701f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.370+0000 7f596e59c640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5970135690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f59600079d0 tx=0x7f596000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.371+0000 7f595effd640 1 -- 172.21.15.82:0/2786685244 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960007ac0 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.371+0000 7f595effd640 1 -- 172.21.15.82:0/2786685244 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5960018070 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.371+0000 7f595effd640 1 -- 172.21.15.82:0/2786685244 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960013610 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.371+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59701f2200 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.371+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59701f62e0 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.372+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597004f050 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.372+0000 7f595effd640 1 -- 172.21.15.82:0/2786685244 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f59600139c0 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.373+0000 7f595effd640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f595006bda0 0x7f595006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.373+0000 7f595effd640 1 -- 172.21.15.82:0/2786685244 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f596008b470 con 0x7f5958003f60 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.373+0000 7f596dd9b640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f595006bda0 0x7f595006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.373+0000 7f596dd9b640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f595006bda0 0x7f595006e260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5964033000 tx=0x7f5964009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:07.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.377+0000 7f595effd640 1 -- 172.21.15.82:0/2786685244 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5960059ff0 con 0x7f5958003f60 2024-03-14T12:16:07.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.587+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59700ab900 con 0x7f5958003f60 2024-03-14T12:16:07.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.587+0000 7f595effd640 1 -- 172.21.15.82:0/2786685244 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59600598f0 con 0x7f5958003f60 2024-03-14T12:16:07.587 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f595006bda0 msgr2=0x7f595006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f595006bda0 0x7f595006e260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5964033000 tx=0x7f5964009030 comp rx=0 tx=0).stop 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 msgr2=0x7f5970135690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5970135690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f59600079d0 tx=0x7f596000a730 comp rx=0 tx=0).stop 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 shutdown_connections 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59701f29d0 0x7f59701f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f595006bda0 0x7f595006e260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5970158570 0x7f59701f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 --2- 172.21.15.82:0/2786685244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5958003f60 0x7f5970135690 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:07.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 >> 172.21.15.82:0/2786685244 conn(0x7f59700b1c70 msgr2=0x7f5970071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:07.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 shutdown_connections 2024-03-14T12:16:07.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:07.589+0000 7f5974819640 1 -- 172.21.15.82:0/2786685244 wait complete. 2024-03-14T12:16:07.600 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:16:07.600 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:16:07.601 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:16:07.601 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T12:16:07.601 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:07.601 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:16:07.601 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:16:07.601 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:16:07.601 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:07.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:07.601 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:07.617 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:07.618 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c443:69ff:fec0:3488/64 scope link 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::887e:60ff:fe47:4333/64 scope link 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a6:e3:19:81:dc:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a4e3:19ff:fe81:dc41/64 scope link 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:07.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:07.620 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:16:07.682 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:16:07.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:07.684 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:16:07.739 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:16:07.740 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:16:07.740 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:16:07.740 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:16:07.740 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:16:07.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:07.740 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:16:07.740 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:16:07.740 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:16:07.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:16:07.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:16:07.741 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:16:07.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:07.835 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:16:07.850 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-14T12:16:07.850 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:16:07.850 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:07.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:07.850 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-14T12:16:07.916 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:07.917 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:07.924 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-03-14T12:16:07.925+0000 7fc3372cd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:16:07.938 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:07.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:07.938 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:07.993 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T12:16:07.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:07.994 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:08.078 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:08.078 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:08.085 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:08.085 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:08.085 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:16:08.085 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:16:08.085 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:16:08.086 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T12:16:08.086 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:16:08.086 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:16:08.086 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:16:08.086 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:16:08.087 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:16:08.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.087 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:08.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.114 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:16:08.175 INFO:teuthology.orchestra.run.smithi082.stdout:client5263 2024-03-14T12:16:08.175 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:08.175 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:08.230 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:16:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.370+0000 7f5da3881640 1 Processor -- start 2024-03-14T12:16:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.370+0000 7f5da3881640 1 -- start start 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.371+0000 7f5da3881640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c06efe0 0x7f5d9c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.371+0000 7f5da3881640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d9c0716b0 0x7f5d9c071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.371+0000 7f5da3881640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0b53f0 con 0x7f5d9c158750 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.371+0000 7f5da3881640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c072040 con 0x7f5d9c0716b0 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.371+0000 7f5da3881640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c158b30 con 0x7f5d9c06efe0 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da1e1e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d9c0716b0 0x7f5d9c071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da0e1c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c06efe0 0x7f5d9c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da161d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5d9c158750 0x7f5d9c15bba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:45730/0 (socket says 172.21.15.82:45730) 2024-03-14T12:16:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da161d640 1 -- 172.21.15.82:0/2660567768 learned_addr learned my addr 172.21.15.82:0/2660567768 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da0e1c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c06efe0 0x7f5d9c0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:34536/0 (socket says 172.21.15.82:34536) 2024-03-14T12:16:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da1e1e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d9c0716b0 0x7f5d9c071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:39592/0 (socket says 172.21.15.82:39592) 2024-03-14T12:16:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da0e1c640 1 -- 172.21.15.82:0/2660567768 >> v2:172.21.15.164:3300/0 conn(0x7f5d9c0716b0 msgr2=0x7f5d9c071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da0e1c640 1 --2- 172.21.15.82:0/2660567768 >> v2:172.21.15.164:3300/0 conn(0x7f5d9c0716b0 0x7f5d9c071b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da0e1c640 1 -- 172.21.15.82:0/2660567768 >> v1:172.21.15.164:6789/0 conn(0x7f5d9c158750 legacy=0x7f5d9c15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.372+0000 7f5da0e1c640 1 -- 172.21.15.82:0/2660567768 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0b4e30 con 0x7f5d9c06efe0 2024-03-14T12:16:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.373+0000 7f5da0e1c640 1 --2- 172.21.15.82:0/2660567768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c06efe0 0x7f5d9c0b47e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097a0 tx=0x7f5d8c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed47d6e259e3be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.373+0000 7f5d93fff640 1 -- 172.21.15.82:0/2660567768 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c035020 con 0x7f5d9c06efe0 2024-03-14T12:16:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.373+0000 7f5d93fff640 1 -- 172.21.15.82:0/2660567768 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c031830 con 0x7f5d9c06efe0 2024-03-14T12:16:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.373+0000 7f5d93fff640 1 -- 172.21.15.82:0/2660567768 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c031c00 con 0x7f5d9c06efe0 2024-03-14T12:16:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.373+0000 7f5da3881640 1 -- 172.21.15.82:0/2660567768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c06efe0 msgr2=0x7f5d9c0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.373+0000 7f5da3881640 1 --2- 172.21.15.82:0/2660567768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c06efe0 0x7f5d9c0b47e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097a0 tx=0x7f5d8c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.374+0000 7f5da3881640 1 -- 172.21.15.82:0/2660567768 shutdown_connections 2024-03-14T12:16:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.374+0000 7f5da3881640 1 --2- 172.21.15.82:0/2660567768 >> v2:172.21.15.164:3300/0 conn(0x7f5d9c0716b0 0x7f5d9c071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.374+0000 7f5da3881640 1 --2- 172.21.15.82:0/2660567768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c06efe0 0x7f5d9c0b47e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.374+0000 7f5da3881640 1 -- 172.21.15.82:0/2660567768 >> 172.21.15.82:0/2660567768 conn(0x7f5d9c0d0a20 msgr2=0x7f5d9c0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.374+0000 7f5da3881640 1 -- 172.21.15.82:0/2660567768 shutdown_connections 2024-03-14T12:16:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.374+0000 7f5da3881640 1 -- 172.21.15.82:0/2660567768 wait complete. 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 Processor -- start 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 -- start start 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c06efe0 0x7f5d9c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0716b0 0x7f5d9c0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c158750 0x7f5d9c147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06dab0 con 0x7f5d9c06efe0 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06d7d0 con 0x7f5d9c0716b0 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.375+0000 7f5da3881640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c164740 con 0x7f5d9c158750 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c06efe0 0x7f5d9c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c06efe0 0x7f5d9c1579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37320/0 (socket says 172.21.15.82:37320) 2024-03-14T12:16:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 -- 172.21.15.82:0/488081336 learned_addr learned my addr 172.21.15.82:0/488081336 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da0e1c640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0716b0 0x7f5d9c0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da1e1e640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c158750 0x7f5d9c147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 -- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c158750 msgr2=0x7f5d9c147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c158750 0x7f5d9c147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 -- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0716b0 msgr2=0x7f5d9c0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0716b0 0x7f5d9c0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 -- 172.21.15.82:0/488081336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c156950 con 0x7f5d9c06efe0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da1e1e640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c158750 0x7f5d9c147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da161d640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c06efe0 0x7f5d9c1579f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800ca70 tx=0x7f5d9800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5d91ffb640 1 -- 172.21.15.82:0/488081336 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98017070 con 0x7f5d9c06efe0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5d91ffb640 1 -- 172.21.15.82:0/488081336 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9800abf0 con 0x7f5d9c06efe0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c156c30 con 0x7f5d9c06efe0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5d91ffb640 1 -- 172.21.15.82:0/488081336 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98012400 con 0x7f5d9c06efe0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.376+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c1574d0 con 0x7f5d9c06efe0 2024-03-14T12:16:08.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.378+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c157020 con 0x7f5d9c06efe0 2024-03-14T12:16:08.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.378+0000 7f5d91ffb640 1 -- 172.21.15.82:0/488081336 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5d9801a070 con 0x7f5d9c06efe0 2024-03-14T12:16:08.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.378+0000 7f5d91ffb640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:08.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.378+0000 7f5da0e1c640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:08.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.378+0000 7f5d91ffb640 1 -- 172.21.15.82:0/488081336 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d9808a6d0 con 0x7f5d9c06efe0 2024-03-14T12:16:08.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.379+0000 7f5da0e1c640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c036000 tx=0x7f5d8c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:08.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.383+0000 7f5d91ffb640 1 -- 172.21.15.82:0/488081336 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5d98058b70 con 0x7f5d9c06efe0 2024-03-14T12:16:08.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.592+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d9c04f050 con 0x7f5d9c06efe0 2024-03-14T12:16:08.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.592+0000 7f5d91ffb640 1 -- 172.21.15.82:0/488081336 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5d9805cc10 con 0x7f5d9c06efe0 2024-03-14T12:16:08.592 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 msgr2=0x7f5d8006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c036000 tx=0x7f5d8c008030 comp rx=0 tx=0).stop 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c06efe0 msgr2=0x7f5d9c1579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c06efe0 0x7f5d9c1579f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800ca70 tx=0x7f5d9800cf30 comp rx=0 tx=0).stop 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 shutdown_connections 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c158750 0x7f5d9c147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c0716b0 0x7f5d9c0c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 --2- 172.21.15.82:0/488081336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c06efe0 0x7f5d9c1579f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:08.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.594+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 >> 172.21.15.82:0/488081336 conn(0x7f5d9c0d0a20 msgr2=0x7f5d9c0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:08.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.595+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 shutdown_connections 2024-03-14T12:16:08.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:08.595+0000 7f5da3881640 1 -- 172.21.15.82:0/488081336 wait complete. 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:16:08.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:16:08.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.606 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-14T12:16:08.625 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-14T12:16:08.626 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42587sec preferred_lft 42587sec 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::882:aaff:fed8:c2be/64 scope link 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T12:16:08.628 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::dc5a:2ff:fe9f:88ac/64 scope link 2024-03-14T12:16:08.628 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:08.628 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:08.628 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether f6:de:82:8b:5d:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-14T12:16:08.628 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::f4de:82ff:fe8b:5dfb/64 scope link 2024-03-14T12:16:08.628 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:08.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.628 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:16:08.698 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-14T12:16:08.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.700 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-14T12:16:08.759 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-14T12:16:08.759 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:16:08.759 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:16:08.759 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:16:08.759 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:16:08.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.759 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:16:08.760 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:16:08.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:16:08.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:16:08.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:16:08.760 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:16:08.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.860 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:16:08.875 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-14T12:16:08.875 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T12:16:08.875 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:16:08.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:08.875 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-03-14T12:16:08.943 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:08.943 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:08.999 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:16:08.999 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:08.999 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:09.054 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T12:16:09.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:09.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:09.138 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:09.139 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:09.146 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with new device syntax: 1@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:16:09.147 INFO:teuthology.orchestra.run.smithi164.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:16:09.148 INFO:teuthology.orchestra.run.smithi164.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:16:09.149 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:16:09.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:09.149 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:09.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:09.175 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:16:09.233 INFO:teuthology.orchestra.run.smithi164.stdout:client4976 2024-03-14T12:16:09.233 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:09.233 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:09.289 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:09.289 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:09.308 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:09.308 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:09.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:09.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.516+0000 7fd5d813b640 1 Processor -- start 2024-03-14T12:16:09.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.516+0000 7fd5d813b640 1 -- start start 2024-03-14T12:16:09.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d813b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd5d015be40 0x7fd5d00d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d813b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 0x7fd5d006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d813b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d00ccb40 con 0x7fd5d015c220 2024-03-14T12:16:09.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d813b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d00cd750 con 0x7fd5d015be40 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d813b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d00cce20 con 0x7fd5d0066b00 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d66d8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd5d015c220 0x7fd5d00cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36424/0 (socket says 172.21.15.82:36424) 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d66d8640 1 -- 172.21.15.82:0/1558688960 learned_addr learned my addr 172.21.15.82:0/1558688960 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d5ed7640 1 --2- 172.21.15.82:0/1558688960 >> v2:172.21.15.164:3300/0 conn(0x7fd5d015be40 0x7fd5d00d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 0x7fd5d006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040968455 0 0) 0x7fd5d00ccb40 con 0x7fd5d015c220 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5bc003520 con 0x7fd5d015c220 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033635499 0 0) 0x7fd5bc003520 con 0x7fd5d015c220 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 msgr2=0x7fd5d006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 0x7fd5d006ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 >> v2:172.21.15.164:3300/0 conn(0x7fd5d015be40 msgr2=0x7fd5d00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 --2- 172.21.15.82:0/1558688960 >> v2:172.21.15.164:3300/0 conn(0x7fd5d015be40 0x7fd5d00d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00cd100 con 0x7fd5d015c220 2024-03-14T12:16:09.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fd5c80019f0 con 0x7fd5d015c220 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.517+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 >> v1:172.21.15.82:6789/0 conn(0x7fd5d015c220 legacy=0x7fd5d00cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d4ed5640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5bc0039b0 0x7fd5bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d4ed5640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5bc004800 0x7fd5bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d4ed5640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 0x7fd5bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d00cd750 con 0x7fd5bc004800 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d00cce20 con 0x7fd5bc0039b0 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc00b9c0 con 0x7fd5d0066b00 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d5ed7640 1 --2- 172.21.15.82:0/1558688960 >> v2:172.21.15.164:3300/0 conn(0x7fd5d015be40 0x7fd5d00d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5bc004800 0x7fd5bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d66d8640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 0x7fd5bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d5ed7640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5bc0039b0 0x7fd5bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d56d6640 1 -- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 msgr2=0x7fd5bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:09.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5d0066b00 0x7fd5bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d56d6640 1 -- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5bc0039b0 msgr2=0x7fd5bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5bc0039b0 0x7fd5bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d56d6640 1 -- 172.21.15.82:0/1558688960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc003520 con 0x7fd5bc004800 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.518+0000 7fd5d5ed7640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5bc0039b0 0x7fd5bc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5bc004800 0x7fd5bc004c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009ac0 tx=0x7fd5bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec6877bbcb86189 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0039070 con 0x7fd5bc004800 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c002c6b0 con 0x7fd5bc004800 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d4ed5640 1 -- 172.21.15.82:0/1558688960 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0009690 con 0x7fd5bc004800 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d813b640 1 -- 172.21.15.82:0/1558688960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5bc004800 msgr2=0x7fd5bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d813b640 1 --2- 172.21.15.82:0/1558688960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5bc004800 0x7fd5bc004c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009ac0 tx=0x7fd5bc005830 comp rx=0 tx=0).stop 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d66d8640 1 -- 172.21.15.82:0/1558688960 reap_dead start 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d813b640 1 -- 172.21.15.82:0/1558688960 shutdown_connections 2024-03-14T12:16:09.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.519+0000 7fd5d813b640 1 -- 172.21.15.82:0/1558688960 >> 172.21.15.82:0/1558688960 conn(0x7fd5d0072310 msgr2=0x7fd5d0072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:09.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.520+0000 7fd5d813b640 1 -- 172.21.15.82:0/1558688960 shutdown_connections 2024-03-14T12:16:09.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.520+0000 7fd5d813b640 1 -- 172.21.15.82:0/1558688960 wait complete. 2024-03-14T12:16:09.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 Processor -- start 2024-03-14T12:16:09.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 -- start start 2024-03-14T12:16:09.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5bc004800 0x7fd5d015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5d01576d0 0x7fd5d0157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d015c900 0x7fd5d015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d00cbf70 con 0x7fd5d01576d0 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d00d0f70 con 0x7fd5d015c900 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d813b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d00cc0f0 con 0x7fd5bc004800 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.521+0000 7fd5d56d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5d01576d0 0x7fd5d0157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5d01576d0 0x7fd5d0157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37332/0 (socket says 172.21.15.82:37332) 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 -- 172.21.15.82:0/3292960028 learned_addr learned my addr 172.21.15.82:0/3292960028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d66d8640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d015c900 0x7fd5d015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d5ed7640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5bc004800 0x7fd5d015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 -- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5bc004800 msgr2=0x7fd5d015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5bc004800 0x7fd5d015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 -- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d015c900 msgr2=0x7fd5d015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d015c900 0x7fd5d015cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 -- 172.21.15.82:0/3292960028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d015d670 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d56d6640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5d01576d0 0x7fd5d0157b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0000c00 tx=0x7fd5c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5c67fc640 1 -- 172.21.15.82:0/3292960028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0042020 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5c67fc640 1 -- 172.21.15.82:0/3292960028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c003a040 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d015d950 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5c67fc640 1 -- 172.21.15.82:0/3292960028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0039070 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.522+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d015df10 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.523+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d004f050 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.524+0000 7fd5c67fc640 1 -- 172.21.15.82:0/3292960028 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd5c0045020 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.524+0000 7fd5c67fc640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5b406bc00 0x7fd5b406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.524+0000 7fd5c67fc640 1 -- 172.21.15.82:0/3292960028 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5c00abc70 con 0x7fd5d01576d0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.524+0000 7fd5d5ed7640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5b406bc00 0x7fd5b406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.525+0000 7fd5d5ed7640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5b406bc00 0x7fd5b406e0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006f80 tx=0x7fd5cc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:09.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.529+0000 7fd5c67fc640 1 -- 172.21.15.82:0/3292960028 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd5c004c070 con 0x7fd5d01576d0 2024-03-14T12:16:09.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.705+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5d00b2d40 con 0x7fd5d01576d0 2024-03-14T12:16:09.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.706+0000 7fd5c67fc640 1 -- 172.21.15.82:0/3292960028 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fd5c007a1a0 con 0x7fd5d01576d0 2024-03-14T12:16:09.705 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:09.705 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:03.671135+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:09.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5b406bc00 msgr2=0x7fd5b406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5b406bc00 0x7fd5b406e0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006f80 tx=0x7fd5cc008040 comp rx=0 tx=0).stop 2024-03-14T12:16:09.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5d01576d0 msgr2=0x7fd5d0157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5d01576d0 0x7fd5d0157b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0000c00 tx=0x7fd5c0004730 comp rx=0 tx=0).stop 2024-03-14T12:16:09.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 shutdown_connections 2024-03-14T12:16:09.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5d015c900 0x7fd5d015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5d01576d0 0x7fd5d0157b50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5b406bc00 0x7fd5b406e0c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 --2- 172.21.15.82:0/3292960028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5bc004800 0x7fd5d015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 >> 172.21.15.82:0/3292960028 conn(0x7fd5d0072310 msgr2=0x7fd5d00ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:09.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 shutdown_connections 2024-03-14T12:16:09.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.708+0000 7fd5d813b640 1 -- 172.21.15.82:0/3292960028 wait complete. 2024-03-14T12:16:09.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:00.232652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:00.983002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:09.718 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:09.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.822+0000 7f1034668640 1 Processor -- start 2024-03-14T12:16:09.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.822+0000 7f1034668640 1 -- start start 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1034668640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f102c140e70 0x7f102c0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1034668640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f102c1626f0 0x7f102c15eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1034668640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102c0b41c0 con 0x7f102c143550 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1034668640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102c15f3f0 con 0x7f102c140e70 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1034668640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102c161c10 con 0x7f102c1626f0 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1032404640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f102c140e70 0x7f102c0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1032404640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f102c140e70 0x7f102c0b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37338/0 (socket says 172.21.15.82:37338) 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1032404640 1 -- 172.21.15.82:0/81498814 learned_addr learned my addr 172.21.15.82:0/81498814 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1031c03640 1 --2- 172.21.15.82:0/81498814 >> v2:172.21.15.164:3300/0 conn(0x7f102c1626f0 0x7f102c15eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1032404640 1 -- 172.21.15.82:0/81498814 >> v2:172.21.15.164:3300/0 conn(0x7f102c1626f0 msgr2=0x7f102c15eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1032404640 1 --2- 172.21.15.82:0/81498814 >> v2:172.21.15.164:3300/0 conn(0x7f102c1626f0 0x7f102c15eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1032404640 1 -- 172.21.15.82:0/81498814 >> v1:172.21.15.164:6789/0 conn(0x7f102c143550 legacy=0x7f102c0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.823+0000 7f1032404640 1 -- 172.21.15.82:0/81498814 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c161ef0 con 0x7f102c140e70 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.824+0000 7f1032404640 1 --2- 172.21.15.82:0/81498814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c140e70 0x7f102c0b3b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f102c0b4a20 tx=0x7f102402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f565af41faa7fb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.824+0000 7f1031402640 1 -- 172.21.15.82:0/81498814 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024034070 con 0x7f102c140e70 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.824+0000 7f1031402640 1 -- 172.21.15.82:0/81498814 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024030860 con 0x7f102c140e70 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.824+0000 7f1031402640 1 -- 172.21.15.82:0/81498814 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024030c30 con 0x7f102c140e70 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 -- 172.21.15.82:0/81498814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c140e70 msgr2=0x7f102c0b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 --2- 172.21.15.82:0/81498814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c140e70 0x7f102c0b3b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f102c0b4a20 tx=0x7f102402b9a0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 -- 172.21.15.82:0/81498814 shutdown_connections 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 --2- 172.21.15.82:0/81498814 >> v2:172.21.15.164:3300/0 conn(0x7f102c1626f0 0x7f102c15eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 --2- 172.21.15.82:0/81498814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c140e70 0x7f102c0b3b30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 -- 172.21.15.82:0/81498814 >> 172.21.15.82:0/81498814 conn(0x7f102c0b2830 msgr2=0x7f102c0b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:09.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 -- 172.21.15.82:0/81498814 shutdown_connections 2024-03-14T12:16:09.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.825+0000 7f1034668640 1 -- 172.21.15.82:0/81498814 wait complete. 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.826+0000 7f1034668640 1 Processor -- start 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.826+0000 7f1034668640 1 -- start start 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1034668640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f102c140e70 0x7f102c1e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1034668640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c143550 0x7f102c13fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1034668640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102c1626f0 0x7f102c0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1034668640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c143b40 con 0x7f102c143550 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1034668640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c1439c0 con 0x7f102c1626f0 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1034668640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102c143cc0 con 0x7f102c140e70 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c143550 0x7f102c13fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c143550 0x7f102c13fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37350/0 (socket says 172.21.15.82:37350) 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 -- 172.21.15.82:0/3282228827 learned_addr learned my addr 172.21.15.82:0/3282228827 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1032404640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f102c140e70 0x7f102c1e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1032c05640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102c1626f0 0x7f102c0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 -- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f102c140e70 msgr2=0x7f102c1e8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f102c140e70 0x7f102c1e8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 -- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102c1626f0 msgr2=0x7f102c0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102c1626f0 0x7f102c0ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 -- 172.21.15.82:0/3282228827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c0cf800 con 0x7f102c143550 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.827+0000 7f1031c03640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c143550 0x7f102c13fdd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d5c0 tx=0x7f101c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.828+0000 7f1022ffd640 1 -- 172.21.15.82:0/3282228827 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0125f0 con 0x7f102c143550 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.828+0000 7f1022ffd640 1 -- 172.21.15.82:0/3282228827 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c019070 con 0x7f102c143550 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.828+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102c0cfac0 con 0x7f102c143550 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.828+0000 7f1022ffd640 1 -- 172.21.15.82:0/3282228827 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0129d0 con 0x7f102c143550 2024-03-14T12:16:09.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.828+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102c1e8690 con 0x7f102c143550 2024-03-14T12:16:09.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.829+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102c04f050 con 0x7f102c143550 2024-03-14T12:16:09.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.829+0000 7f1022ffd640 1 -- 172.21.15.82:0/3282228827 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f101c012b70 con 0x7f102c143550 2024-03-14T12:16:09.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.829+0000 7f1022ffd640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f101006bbd0 0x7f101006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:09.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.830+0000 7f1022ffd640 1 -- 172.21.15.82:0/3282228827 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f101c08ad30 con 0x7f102c143550 2024-03-14T12:16:09.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.830+0000 7f1032404640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f101006bbd0 0x7f101006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:09.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.830+0000 7f1032404640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f101006bbd0 0x7f101006e090 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f1024008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:09.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:09.834+0000 7f1022ffd640 1 -- 172.21.15.82:0/3282228827 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f101c0591d0 con 0x7f102c143550 2024-03-14T12:16:10.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.005+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f102c0b3700 con 0x7f102c143550 2024-03-14T12:16:10.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.005+0000 7f1022ffd640 1 -- 172.21.15.82:0/3282228827 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f101c05d270 con 0x7f102c143550 2024-03-14T12:16:10.004 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:10.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f101006bbd0 msgr2=0x7f101006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f101006bbd0 0x7f101006e090 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f1024008030 comp rx=0 tx=0).stop 2024-03-14T12:16:10.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c143550 msgr2=0x7f102c13fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c143550 0x7f102c13fdd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d5c0 tx=0x7f101c00da80 comp rx=0 tx=0).stop 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 shutdown_connections 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102c1626f0 0x7f102c0ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f102c143550 0x7f102c13fdd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f101006bbd0 0x7f101006e090 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 --2- 172.21.15.82:0/3282228827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f102c140e70 0x7f102c1e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 >> 172.21.15.82:0/3282228827 conn(0x7f102c0b2830 msgr2=0x7f102c073e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 shutdown_connections 2024-03-14T12:16:10.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.007+0000 7f1034668640 1 -- 172.21.15.82:0/3282228827 wait complete. 2024-03-14T12:16:10.017 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:16:10.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.121+0000 7f0ed1d2b640 1 Processor -- start 2024-03-14T12:16:10.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.121+0000 7f0ed1d2b640 1 -- start start 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.122+0000 7f0ed1d2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 0x7f0ecc0d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.122+0000 7f0ed1d2b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0ecc0d0e40 0x7f0ecc0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.122+0000 7f0ed1d2b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0ce970 con 0x7f0ecc0cf6f0 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.122+0000 7f0ed1d2b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc0cf260 con 0x7f0ecc0d0e40 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.122+0000 7f0ed1d2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0d4a80 con 0x7f0ecc15e2c0 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.122+0000 7f0ecaffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0ecc0cf6f0 0x7f0ecc0d0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36440/0 (socket says 172.21.15.82:36440) 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.122+0000 7f0ecaffd640 1 -- 172.21.15.82:0/4196813419 learned_addr learned my addr 172.21.15.82:0/4196813419 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecbfff640 1 --2- 172.21.15.82:0/4196813419 >> v2:172.21.15.164:3300/0 conn(0x7f0ecc0d0e40 0x7f0ecc0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081503375 0 0) 0x7f0ecc0ce970 con 0x7f0ecc0cf6f0 2024-03-14T12:16:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb4003520 con 0x7f0ecc0cf6f0 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecb7fe640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 0x7f0ecc0d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1191070218 0 0) 0x7f0eb4003520 con 0x7f0ecc0cf6f0 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 msgr2=0x7f0ecc0d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 0x7f0ecc0d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 >> v2:172.21.15.164:3300/0 conn(0x7f0ecc0d0e40 msgr2=0x7f0ecc0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 --2- 172.21.15.82:0/4196813419 >> v2:172.21.15.164:3300/0 conn(0x7f0ecc0d0e40 0x7f0ecc0b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0cec10 con 0x7f0ecc0cf6f0 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f0ec0001670 con 0x7f0ecc0cf6f0 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 >> v1:172.21.15.82:6789/0 conn(0x7f0ecc0cf6f0 legacy=0x7f0ecc0d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 0x7f0eb40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4003f30 0x7f0eb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eb4004d40 0x7f0eb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0cf260 con 0x7f0eb4004d40 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0d4a80 con 0x7f0eb4003f30 2024-03-14T12:16:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb400b9c0 con 0x7f0ecc15e2c0 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecbfff640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eb4004d40 0x7f0eb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecb7fe640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 0x7f0eb40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecaffd640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4003f30 0x7f0eb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecbfff640 1 -- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 msgr2=0x7f0eb40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecbfff640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc15e2c0 0x7f0eb40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecbfff640 1 -- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4003f30 msgr2=0x7f0eb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecbfff640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4003f30 0x7f0eb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.123+0000 7f0ecbfff640 1 -- 172.21.15.82:0/4196813419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb4003520 con 0x7f0eb4004d40 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.124+0000 7f0ecbfff640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eb4004d40 0x7f0eb40051c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009fc0 tx=0x7f0eb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7ad070522c30bd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.124+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4039070 con 0x7f0eb4004d40 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.124+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec402cc40 con 0x7f0eb4004d40 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.124+0000 7f0eca7fc640 1 -- 172.21.15.82:0/4196813419 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4009c10 con 0x7f0eb4004d40 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.125+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/4196813419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eb4004d40 msgr2=0x7f0eb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.125+0000 7f0ed1d2b640 1 --2- 172.21.15.82:0/4196813419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eb4004d40 0x7f0eb40051c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009fc0 tx=0x7f0eb4005830 comp rx=0 tx=0).stop 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.125+0000 7f0ecbfff640 1 -- 172.21.15.82:0/4196813419 reap_dead start 2024-03-14T12:16:10.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.125+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/4196813419 shutdown_connections 2024-03-14T12:16:10.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.125+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/4196813419 >> 172.21.15.82:0/4196813419 conn(0x7f0ecc09d0d0 msgr2=0x7f0ecc0d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:10.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.125+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/4196813419 shutdown_connections 2024-03-14T12:16:10.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.125+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/4196813419 wait complete. 2024-03-14T12:16:10.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.126+0000 7f0ed1d2b640 1 Processor -- start 2024-03-14T12:16:10.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.126+0000 7f0ed1d2b640 1 -- start start 2024-03-14T12:16:10.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ed1d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4004d40 0x7f0ecc0b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ed1d2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ecc0b5b40 0x7f0ecc152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ed1d2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc152e60 0x7f0ecc1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ed1d2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0d4820 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ed1d2b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0d46a0 con 0x7f0eb4004d40 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ed1d2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0d1a70 con 0x7f0ecc152e60 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ecc0b5b40 0x7f0ecc152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ecc0b5b40 0x7f0ecc152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37376/0 (socket says 172.21.15.82:37376) 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 -- 172.21.15.82:0/1863900909 learned_addr learned my addr 172.21.15.82:0/1863900909 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecb7fe640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4004d40 0x7f0ecc0b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecbfff640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc152e60 0x7f0ecc1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 -- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc152e60 msgr2=0x7f0ecc1f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc152e60 0x7f0ecc1f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 -- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4004d40 msgr2=0x7f0ecc0b8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4004d40 0x7f0ecc0b8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 -- 172.21.15.82:0/1863900909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0cd700 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecb7fe640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4004d40 0x7f0ecc0b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.127+0000 7f0ecaffd640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ecc0b5b40 0x7f0ecc152920 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00073a0 tx=0x7f0ec0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.128+0000 7f0eb3fff640 1 -- 172.21.15.82:0/1863900909 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec001a070 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.128+0000 7f0eb3fff640 1 -- 172.21.15.82:0/1863900909 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec000f9c0 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.128+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc0cb5b0 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.128+0000 7f0eb3fff640 1 -- 172.21.15.82:0/1863900909 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000fd90 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.128+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc0cb2d0 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.129+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc08cdb0 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.129+0000 7f0eb3fff640 1 -- 172.21.15.82:0/1863900909 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0ec001d070 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.130+0000 7f0eb3fff640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0eb806bc00 0x7f0eb806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.130+0000 7f0eb3fff640 1 -- 172.21.15.82:0/1863900909 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ec008a4b0 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.130+0000 7f0ecb7fe640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0eb806bc00 0x7f0eb806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.130+0000 7f0ecb7fe640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0eb806bc00 0x7f0eb806e0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009bc0 tx=0x7f0ebc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.134+0000 7f0eb3fff640 1 -- 172.21.15.82:0/1863900909 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0ecc08cdb0 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.302+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ecc04f050 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.303+0000 7f0eb3fff640 1 -- 172.21.15.82:0/1863900909 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ec0058fa0 con 0x7f0ecc0b5b40 2024-03-14T12:16:10.302 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:16:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0eb806bc00 msgr2=0x7f0eb806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0eb806bc00 0x7f0eb806e0c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009bc0 tx=0x7f0ebc008040 comp rx=0 tx=0).stop 2024-03-14T12:16:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ecc0b5b40 msgr2=0x7f0ecc152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ecc0b5b40 0x7f0ecc152920 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00073a0 tx=0x7f0ec0007860 comp rx=0 tx=0).stop 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 shutdown_connections 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0eb806bc00 0x7f0eb806e0c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0ecc152e60 0x7f0ecc1f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ecc0b5b40 0x7f0ecc152920 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 --2- 172.21.15.82:0/1863900909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0eb4004d40 0x7f0ecc0b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 >> 172.21.15.82:0/1863900909 conn(0x7f0ecc09d0d0 msgr2=0x7f0ecc15d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 shutdown_connections 2024-03-14T12:16:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.305+0000 7f0ed1d2b640 1 -- 172.21.15.82:0/1863900909 wait complete. 2024-03-14T12:16:10.315 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:16:10.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.419+0000 7fa344b1b640 1 Processor -- start 2024-03-14T12:16:10.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.419+0000 7fa344b1b640 1 -- start start 2024-03-14T12:16:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa344b1b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3400d12a0 0x7fa3400d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa344b1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400d0c40 0x7fa3400cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa344b1b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400d25a0 con 0x7fa34015dff0 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa344b1b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3400d2350 con 0x7fa3400d12a0 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa344b1b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400d2880 con 0x7fa3400d0c40 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa335d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3400d12a0 0x7fa3400d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa33e59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa34015dff0 0x7fa3400d4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36442/0 (socket says 172.21.15.82:36442) 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa335d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3400d12a0 0x7fa3400d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37378/0 (socket says 172.21.15.82:37378) 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.420+0000 7fa33e59c640 1 -- 172.21.15.82:0/3293682685 learned_addr learned my addr 172.21.15.82:0/3293682685 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.421+0000 7fa335d9b640 1 -- 172.21.15.82:0/3293682685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400d0c40 msgr2=0x7fa3400cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.421+0000 7fa335d9b640 1 --2- 172.21.15.82:0/3293682685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400d0c40 0x7fa3400cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.421+0000 7fa335d9b640 1 -- 172.21.15.82:0/3293682685 >> v1:172.21.15.82:6789/0 conn(0x7fa34015dff0 legacy=0x7fa3400d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.421+0000 7fa33dd9b640 1 -- 172.21.15.82:0/3293682685 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240610364 0 0) 0x7fa3400d25a0 con 0x7fa34015dff0 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.421+0000 7fa335d9b640 1 -- 172.21.15.82:0/3293682685 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400d2b60 con 0x7fa3400d12a0 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.421+0000 7fa335d9b640 1 --2- 172.21.15.82:0/3293682685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400d0700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3280097a0 tx=0x7fa32802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223b0d461cdd4ea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa33dd9b640 1 -- 172.21.15.82:0/3293682685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328035020 con 0x7fa3400d12a0 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa33dd9b640 1 -- 172.21.15.82:0/3293682685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa328031830 con 0x7fa3400d12a0 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa33dd9b640 1 -- 172.21.15.82:0/3293682685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328031c00 con 0x7fa3400d12a0 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 -- 172.21.15.82:0/3293682685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 msgr2=0x7fa3400d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 --2- 172.21.15.82:0/3293682685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400d0700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3280097a0 tx=0x7fa32802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 -- 172.21.15.82:0/3293682685 shutdown_connections 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 --2- 172.21.15.82:0/3293682685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400d0c40 0x7fa3400cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 --2- 172.21.15.82:0/3293682685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400d0700 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 -- 172.21.15.82:0/3293682685 >> 172.21.15.82:0/3293682685 conn(0x7fa34009d0d0 msgr2=0x7fa3400d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 -- 172.21.15.82:0/3293682685 shutdown_connections 2024-03-14T12:16:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.422+0000 7fa344b1b640 1 -- 172.21.15.82:0/3293682685 wait complete. 2024-03-14T12:16:10.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 Processor -- start 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 -- start start 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400d0c40 0x7fa34014a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa34015dff0 0x7fa3400b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400cb9e0 con 0x7fa3400d12a0 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400cb860 con 0x7fa3400d0c40 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa344b1b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400cbb60 con 0x7fa34015dff0 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa335d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa335d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37388/0 (socket says 172.21.15.82:37388) 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.424+0000 7fa335d9b640 1 -- 172.21.15.82:0/2353574625 learned_addr learned my addr 172.21.15.82:0/2353574625 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa33e59c640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400d0c40 0x7fa34014a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa335d9b640 1 -- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa34015dff0 msgr2=0x7fa3400b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa33ed9d640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa34015dff0 0x7fa3400b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa335d9b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa34015dff0 0x7fa3400b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa335d9b640 1 -- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400d0c40 msgr2=0x7fa34014a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa335d9b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400d0c40 0x7fa34014a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa335d9b640 1 -- 172.21.15.82:0/2353574625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34014a6b0 con 0x7fa3400d12a0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa33e59c640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400d0c40 0x7fa34014a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa33ed9d640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa34015dff0 0x7fa3400b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa335d9b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400b7370 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa3280098c0 tx=0x7fa3280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa3377fe640 1 -- 172.21.15.82:0/2353574625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32803ba80 con 0x7fa3400d12a0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa3377fe640 1 -- 172.21.15.82:0/2353574625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32803bc20 con 0x7fa3400d12a0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3400b8170 con 0x7fa3400d12a0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa3377fe640 1 -- 172.21.15.82:0/2353574625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3280453f0 con 0x7fa3400d12a0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.425+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3400b8730 con 0x7fa3400d12a0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.426+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34004f050 con 0x7fa3400d12a0 2024-03-14T12:16:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.427+0000 7fa3377fe640 1 -- 172.21.15.82:0/2353574625 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa328038020 con 0x7fa3400d12a0 2024-03-14T12:16:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.427+0000 7fa3377fe640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32006bbd0 0x7fa32006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.427+0000 7fa3377fe640 1 -- 172.21.15.82:0/2353574625 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3280ac860 con 0x7fa3400d12a0 2024-03-14T12:16:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.427+0000 7fa33e59c640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32006bbd0 0x7fa32006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.427+0000 7fa33e59c640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32006bbd0 0x7fa32006e090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa330000f60 tx=0x7fa330009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.431+0000 7fa3377fe640 1 -- 172.21.15.82:0/2353574625 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa34014a6b0 con 0x7fa3400d12a0 2024-03-14T12:16:10.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.603+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3400cf460 con 0x7fa3400d12a0 2024-03-14T12:16:10.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.603+0000 7fa3377fe640 1 -- 172.21.15.82:0/2353574625 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa3280ac020 con 0x7fa3400d12a0 2024-03-14T12:16:10.603 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.605+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32006bbd0 msgr2=0x7fa32006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.605+0000 7fa344b1b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32006bbd0 0x7fa32006e090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa330000f60 tx=0x7fa330009040 comp rx=0 tx=0).stop 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.605+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 msgr2=0x7fa3400b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.605+0000 7fa344b1b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400b7370 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa3280098c0 tx=0x7fa3280046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 shutdown_connections 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32006bbd0 0x7fa32006e090 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa34015dff0 0x7fa3400b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d12a0 0x7fa3400b7370 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 --2- 172.21.15.82:0/2353574625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400d0c40 0x7fa34014a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 >> 172.21.15.82:0/2353574625 conn(0x7fa34009d0d0 msgr2=0x7fa3400cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 shutdown_connections 2024-03-14T12:16:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.606+0000 7fa344b1b640 1 -- 172.21.15.82:0/2353574625 wait complete. 2024-03-14T12:16:10.616 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:16:10.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.721+0000 7f9fe93b2640 1 Processor -- start 2024-03-14T12:16:10.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.721+0000 7f9fe93b2640 1 -- start start 2024-03-14T12:16:10.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.721+0000 7f9fe93b2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9fe415e0e0 0x7f9fe40d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.721+0000 7f9fe93b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40b3e10 0x7f9fe40b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.721+0000 7f9fe93b2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe415e4c0 con 0x7f9fe40d4a20 2024-03-14T12:16:10.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.721+0000 7f9fe93b2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe40d1640 con 0x7f9fe415e0e0 2024-03-14T12:16:10.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe93b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40cf2f0 con 0x7f9fe40b3e10 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9fe415e0e0 0x7f9fe40d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe27fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9fe40d4a20 0x7f9fe40d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36452/0 (socket says 172.21.15.82:36452) 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe27fc640 1 -- 172.21.15.82:0/671997306 learned_addr learned my addr 172.21.15.82:0/671997306 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe2ffd640 1 -- 172.21.15.82:0/671997306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40b3e10 msgr2=0x7f9fe40b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe2ffd640 1 --2- 172.21.15.82:0/671997306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40b3e10 0x7f9fe40b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe2ffd640 1 -- 172.21.15.82:0/671997306 >> v1:172.21.15.82:6789/0 conn(0x7f9fe40d4a20 legacy=0x7f9fe40d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe1ffb640 1 -- 172.21.15.82:0/671997306 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974696998 0 0) 0x7f9fe415e4c0 con 0x7f9fe40d4a20 2024-03-14T12:16:10.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.722+0000 7f9fe2ffd640 1 -- 172.21.15.82:0/671997306 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40cf5d0 con 0x7f9fe415e0e0 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe2ffd640 1 --2- 172.21.15.82:0/671997306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40d1910 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40097a0 tx=0x7f9fd402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab43c3a52eb6b98f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe1ffb640 1 -- 172.21.15.82:0/671997306 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4035020 con 0x7f9fe415e0e0 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe1ffb640 1 -- 172.21.15.82:0/671997306 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd4031830 con 0x7f9fe415e0e0 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe1ffb640 1 -- 172.21.15.82:0/671997306 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4031c00 con 0x7f9fe415e0e0 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 -- 172.21.15.82:0/671997306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe415e0e0 msgr2=0x7f9fe40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/671997306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40d1910 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40097a0 tx=0x7f9fd402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 -- 172.21.15.82:0/671997306 shutdown_connections 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/671997306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40b3e10 0x7f9fe40b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/671997306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40d1910 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 -- 172.21.15.82:0/671997306 >> 172.21.15.82:0/671997306 conn(0x7f9fe40d3d40 msgr2=0x7f9fe40d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 -- 172.21.15.82:0/671997306 shutdown_connections 2024-03-14T12:16:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.723+0000 7f9fe93b2640 1 -- 172.21.15.82:0/671997306 wait complete. 2024-03-14T12:16:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 Processor -- start 2024-03-14T12:16:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 -- start start 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe40b3e10 0x7f9fe41607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40d4a20 0x7f9fe4152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40b3610 con 0x7f9fe40b3e10 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40cfb90 con 0x7f9fe415e0e0 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe93b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40b38f0 con 0x7f9fe40d4a20 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe40b3e10 0x7f9fe41607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe40b3e10 0x7f9fe41607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37408/0 (socket says 172.21.15.82:37408) 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.725+0000 7f9fe2ffd640 1 -- 172.21.15.82:0/769381363 learned_addr learned my addr 172.21.15.82:0/769381363 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:10.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe37fe640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe2ffd640 1 -- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40d4a20 msgr2=0x7f9fe4152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe2ffd640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40d4a20 0x7f9fe4152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe2ffd640 1 -- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fe415e0e0 msgr2=0x7f9fe40b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe2ffd640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe2ffd640 1 -- 172.21.15.82:0/769381363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe4153470 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe37fe640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe2ffd640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe40b3e10 0x7f9fe41607e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40098c0 tx=0x7f9fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fcb7fe640 1 -- 172.21.15.82:0/769381363 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd403ba80 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fcb7fe640 1 -- 172.21.15.82:0/769381363 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd403bc20 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe415f740 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fcb7fe640 1 -- 172.21.15.82:0/769381363 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40453f0 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.726+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe415fd00 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.727+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe404f050 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.727+0000 7f9fcb7fe640 1 -- 172.21.15.82:0/769381363 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9fd404c060 con 0x7f9fe40b3e10 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.728+0000 7f9fcb7fe640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fd006bbd0 0x7f9fd006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:10.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.728+0000 7f9fe27fc640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fd006bbd0 0x7f9fd006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:10.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.728+0000 7f9fcb7fe640 1 -- 172.21.15.82:0/769381363 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fd40ac780 con 0x7f9fe40b3e10 2024-03-14T12:16:10.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.728+0000 7f9fe27fc640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fd006bbd0 0x7f9fd006e090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8007fd0 tx=0x7f9fd8007920 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:10.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.732+0000 7f9fcb7fe640 1 -- 172.21.15.82:0/769381363 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9fd4040070 con 0x7f9fe40b3e10 2024-03-14T12:16:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.913+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9fe406c770 con 0x7f9fe40b3e10 2024-03-14T12:16:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.913+0000 7f9fcb7fe640 1 -- 172.21.15.82:0/769381363 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9fd402f2e0 con 0x7f9fe40b3e10 2024-03-14T12:16:10.912 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fd006bbd0 msgr2=0x7f9fd006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fd006bbd0 0x7f9fd006e090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8007fd0 tx=0x7f9fd8007920 comp rx=0 tx=0).stop 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe40b3e10 msgr2=0x7f9fe41607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe40b3e10 0x7f9fe41607e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40098c0 tx=0x7f9fd40046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 shutdown_connections 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fe415e0e0 0x7f9fe40b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fd006bbd0 0x7f9fd006e090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fe40d4a20 0x7f9fe4152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 --2- 172.21.15.82:0/769381363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fe40b3e10 0x7f9fe41607e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 >> 172.21.15.82:0/769381363 conn(0x7f9fe40d3d40 msgr2=0x7f9fe406cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 shutdown_connections 2024-03-14T12:16:10.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:10.915+0000 7f9fe93b2640 1 -- 172.21.15.82:0/769381363 wait complete. 2024-03-14T12:16:10.925 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T12:16:10.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:10.925 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:10.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:16:10.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:10.956 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T12:16:11.011 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:16:11.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.152+0000 7f1feea9a640 1 Processor -- start 2024-03-14T12:16:11.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.152+0000 7f1feea9a640 1 -- start start 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.152+0000 7f1feea9a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1fe80b4b70 0x7f1fe8070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.152+0000 7f1feea9a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1fe815dde0 0x7f1fe815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.152+0000 7f1feea9a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80b3da0 con 0x7f1fe8151230 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.152+0000 7f1feea9a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe806c3b0 con 0x7f1fe815dde0 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.152+0000 7f1feea9a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe806c690 con 0x7f1fe80b4b70 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.153+0000 7f1fed037640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1fe815dde0 0x7f1fe815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.153+0000 7f1fec836640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1fe8151230 0x7f1fe80cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36466/0 (socket says 172.21.15.82:36466) 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.153+0000 7f1fec836640 1 -- 172.21.15.82:0/2748377295 learned_addr learned my addr 172.21.15.82:0/2748377295 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.153+0000 7f1fed037640 1 -- 172.21.15.82:0/2748377295 >> v2:172.21.15.164:3300/0 conn(0x7f1fe80b4b70 msgr2=0x7f1fe8070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.153+0000 7f1fed037640 1 --2- 172.21.15.82:0/2748377295 >> v2:172.21.15.164:3300/0 conn(0x7f1fe80b4b70 0x7f1fe8070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.153+0000 7f1fed037640 1 -- 172.21.15.82:0/2748377295 >> v1:172.21.15.82:6789/0 conn(0x7f1fe8151230 legacy=0x7f1fe80cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:11.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.153+0000 7f1fed037640 1 -- 172.21.15.82:0/2748377295 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80b4080 con 0x7f1fe815dde0 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1fed037640 1 --2- 172.21.15.82:0/2748377295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe815e210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806eb40 tx=0x7f1fe002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f362705bbd7e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1fe77fe640 1 -- 172.21.15.82:0/2748377295 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0039070 con 0x7f1fe815dde0 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1fe77fe640 1 -- 172.21.15.82:0/2748377295 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe002cc80 con 0x7f1fe815dde0 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1fe77fe640 1 -- 172.21.15.82:0/2748377295 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0034660 con 0x7f1fe815dde0 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 -- 172.21.15.82:0/2748377295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 msgr2=0x7f1fe815e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 --2- 172.21.15.82:0/2748377295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe815e210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806eb40 tx=0x7f1fe002bed0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 -- 172.21.15.82:0/2748377295 shutdown_connections 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 --2- 172.21.15.82:0/2748377295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe815e210 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 --2- 172.21.15.82:0/2748377295 >> v2:172.21.15.164:3300/0 conn(0x7f1fe80b4b70 0x7f1fe8070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 -- 172.21.15.82:0/2748377295 >> 172.21.15.82:0/2748377295 conn(0x7f1fe80b1c70 msgr2=0x7f1fe80cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 -- 172.21.15.82:0/2748377295 shutdown_connections 2024-03-14T12:16:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.154+0000 7f1feea9a640 1 -- 172.21.15.82:0/2748377295 wait complete. 2024-03-14T12:16:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.155+0000 7f1feea9a640 1 Processor -- start 2024-03-14T12:16:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.155+0000 7f1feea9a640 1 -- start start 2024-03-14T12:16:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1feea9a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe80b4b70 0x7f1fe8140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1feea9a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fe8151230 0x7f1fe80cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1feea9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe81539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1feea9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8151610 con 0x7f1fe815dde0 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1feea9a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe815e750 con 0x7f1fe80b4b70 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1feea9a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8070b70 con 0x7f1fe8151230 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1fed037640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe81539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1fed037640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe81539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37424/0 (socket says 172.21.15.82:37424) 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.156+0000 7f1fed037640 1 -- 172.21.15.82:0/4178865022 learned_addr learned my addr 172.21.15.82:0/4178865022 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fec836640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe80b4b70 0x7f1fe8140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fe7fff640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fe8151230 0x7f1fe80cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fed037640 1 -- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fe8151230 msgr2=0x7f1fe80cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fed037640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fe8151230 0x7f1fe80cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fed037640 1 -- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe80b4b70 msgr2=0x7f1fe8140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fed037640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe80b4b70 0x7f1fe8140680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fed037640 1 -- 172.21.15.82:0/4178865022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe814a0f0 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fec836640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe80b4b70 0x7f1fe8140680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fe7fff640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fe8151230 0x7f1fe80cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fed037640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe81539a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009e80 tx=0x7f1fe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fe57fa640 1 -- 172.21.15.82:0/4178865022 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0039050 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fe57fa640 1 -- 172.21.15.82:0/4178865022 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe0045070 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe8149850 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1fe57fa640 1 -- 172.21.15.82:0/4178865022 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe003e850 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.157+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe8149e10 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.158+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe806e6f0 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.159+0000 7f1fe57fa640 1 -- 172.21.15.82:0/4178865022 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1fe003e9f0 con 0x7f1fe815dde0 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.159+0000 7f1fe57fa640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fcc06bbd0 0x7f1fcc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.159+0000 7f1fe57fa640 1 -- 172.21.15.82:0/4178865022 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fe00ad860 con 0x7f1fe815dde0 2024-03-14T12:16:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.159+0000 7f1fec836640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fcc06bbd0 0x7f1fcc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.159+0000 7f1fec836640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fcc06bbd0 0x7f1fcc06e090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000f60 tx=0x7f1fdc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.163+0000 7f1fe57fa640 1 -- 172.21.15.82:0/4178865022 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1fe806e6f0 con 0x7f1fe815dde0 2024-03-14T12:16:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.370+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1fe806cf00 con 0x7f1fe815dde0 2024-03-14T12:16:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.370+0000 7f1fe57fa640 1 -- 172.21.15.82:0/4178865022 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1fe806cf00 con 0x7f1fe815dde0 2024-03-14T12:16:11.370 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fcc06bbd0 msgr2=0x7f1fcc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fcc06bbd0 0x7f1fcc06e090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000f60 tx=0x7f1fdc009040 comp rx=0 tx=0).stop 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 msgr2=0x7f1fe81539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe81539a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009e80 tx=0x7f1fe0004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 shutdown_connections 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fe815dde0 0x7f1fe81539a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fcc06bbd0 0x7f1fcc06e090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fe8151230 0x7f1fe80cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.372+0000 7f1feea9a640 1 --2- 172.21.15.82:0/4178865022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fe80b4b70 0x7f1fe8140680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:11.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.373+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 >> 172.21.15.82:0/4178865022 conn(0x7f1fe80b1c70 msgr2=0x7f1fe80ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:11.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.373+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 shutdown_connections 2024-03-14T12:16:11.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:11.373+0000 7f1feea9a640 1 -- 172.21.15.82:0/4178865022 wait complete. 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:16:11.383 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:11.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.383 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:16:11.399 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42549sec preferred_lft 42549sec 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:11.400 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c443:69ff:fec0:3488/64 scope link 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::887e:60ff:fe47:4333/64 scope link 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a6:e3:19:81:dc:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a4e3:19ff:fe81:dc41/64 scope link 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:11.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.402 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:16:11.464 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:16:11.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.466 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:16:11.522 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:16:11.522 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:16:11.522 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:16:11.522 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:16:11.522 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:16:11.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.522 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:16:11.523 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:16:11.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:16:11.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:16:11.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:16:11.523 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:16:11.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.617 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:16:11.632 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-14T12:16:11.632 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:16:11.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:11.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.633 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-14T12:16:11.701 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:11.701 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:11.756 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:11.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:11.756 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:11.810 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T12:16:11.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:11.894 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:11.894 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:11.901 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:11.901 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:11.901 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:16:11.902 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:16:11.902 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:16:11.902 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T12:16:11.902 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:16:11.902 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:16:11.902 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:16:11.902 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:16:11.904 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:16:11.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:11.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:11.930 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:16:11.986 INFO:teuthology.orchestra.run.smithi082.stdout:client5293 2024-03-14T12:16:11.986 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:11.986 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:12.041 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:12.041 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:12.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:12.097 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:16:12.153 INFO:teuthology.orchestra.run.smithi082.stdout:client5293 2024-03-14T12:16:12.153 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:16:12.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 counter dump 2024-03-14T12:16:12.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.295+0000 7fb02a909640 1 Processor -- start 2024-03-14T12:16:12.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.295+0000 7fb02a909640 1 -- start start 2024-03-14T12:16:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb02a909640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb02415e200 0x7fb0240d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb02a909640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb0240d28e0 0x7fb0240b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb02a909640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0240d2cf0 con 0x7fb02415de20 2024-03-14T12:16:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb02a909640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0240cfd80 con 0x7fb0240d28e0 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb02a909640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0240cece0 con 0x7fb02415e200 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb028ea6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb0240d28e0 0x7fb0240b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb028ea6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb0240d28e0 0x7fb0240b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49562/0 (socket says 172.21.15.82:49562) 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb028ea6640 1 -- 172.21.15.82:0/2985961426 learned_addr learned my addr 172.21.15.82:0/2985961426 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb0237fe640 1 --2- 172.21.15.82:0/2985961426 >> v2:172.21.15.164:3300/0 conn(0x7fb02415e200 0x7fb0240d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb028ea6640 1 -- 172.21.15.82:0/2985961426 >> v2:172.21.15.164:3300/0 conn(0x7fb02415e200 msgr2=0x7fb0240d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2985961426 >> v2:172.21.15.164:3300/0 conn(0x7fb02415e200 0x7fb0240d23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb028ea6640 1 -- 172.21.15.82:0/2985961426 >> v1:172.21.15.164:6789/0 conn(0x7fb02415de20 legacy=0x7fb0240d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.296+0000 7fb028ea6640 1 -- 172.21.15.82:0/2985961426 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240cefc0 con 0x7fb0240d28e0 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.297+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2985961426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240b4960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb02408e500 tx=0x7fb01c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c89c34bfee5765 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.297+0000 7fb022ffd640 1 -- 172.21.15.82:0/2985961426 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c02ca80 con 0x7fb0240d28e0 2024-03-14T12:16:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.297+0000 7fb022ffd640 1 -- 172.21.15.82:0/2985961426 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c02cc20 con 0x7fb0240d28e0 2024-03-14T12:16:12.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.297+0000 7fb022ffd640 1 -- 172.21.15.82:0/2985961426 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0343f0 con 0x7fb0240d28e0 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 -- 172.21.15.82:0/2985961426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 msgr2=0x7fb0240b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 --2- 172.21.15.82:0/2985961426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240b4960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb02408e500 tx=0x7fb01c02bee0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 -- 172.21.15.82:0/2985961426 shutdown_connections 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 --2- 172.21.15.82:0/2985961426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240b4960 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 --2- 172.21.15.82:0/2985961426 >> v2:172.21.15.164:3300/0 conn(0x7fb02415e200 0x7fb0240d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 -- 172.21.15.82:0/2985961426 >> 172.21.15.82:0/2985961426 conn(0x7fb02409d030 msgr2=0x7fb0240d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 -- 172.21.15.82:0/2985961426 shutdown_connections 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.298+0000 7fb02a909640 1 -- 172.21.15.82:0/2985961426 wait complete. 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.299+0000 7fb02a909640 1 Processor -- start 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.299+0000 7fb02a909640 1 -- start start 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb02a909640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb02a909640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb02415de20 0x7fb024148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb02a909640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb02415e200 0x7fb024158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb02a909640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240ce940 con 0x7fb0240d28e0 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb02a909640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240cf580 con 0x7fb02415de20 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb02a909640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240ceac0 con 0x7fb02415e200 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49574/0 (socket says 172.21.15.82:49574) 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 -- 172.21.15.82:0/2239851917 learned_addr learned my addr 172.21.15.82:0/2239851917 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb0237fe640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb02415de20 0x7fb024148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb02415e200 0x7fb024158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 -- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb02415e200 msgr2=0x7fb024158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb02415e200 0x7fb024158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 -- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb02415de20 msgr2=0x7fb024148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb02415de20 0x7fb024148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 -- 172.21.15.82:0/2239851917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024159200 con 0x7fb0240d28e0 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb02415e200 0x7fb024158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.300+0000 7fb023fff640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240cbee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb01400c400 tx=0x7fb01400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.301+0000 7fb020ff9640 1 -- 172.21.15.82:0/2239851917 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140052d0 con 0x7fb0240d28e0 2024-03-14T12:16:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.301+0000 7fb020ff9640 1 -- 172.21.15.82:0/2239851917 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014005490 con 0x7fb0240d28e0 2024-03-14T12:16:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.301+0000 7fb020ff9640 1 -- 172.21.15.82:0/2239851917 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014007260 con 0x7fb0240d28e0 2024-03-14T12:16:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.301+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0240cc4c0 con 0x7fb0240d28e0 2024-03-14T12:16:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.301+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0240cca80 con 0x7fb0240d28e0 2024-03-14T12:16:12.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.302+0000 7fb020ff9640 1 -- 172.21.15.82:0/2239851917 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb014007400 con 0x7fb0240d28e0 2024-03-14T12:16:12.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.303+0000 7fb020ff9640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb01006bbd0 0x7fb01006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.303+0000 7fb0237fe640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb01006bbd0 0x7fb01006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.303+0000 7fb020ff9640 1 -- 172.21.15.82:0/2239851917 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb01408a080 con 0x7fb0240d28e0 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.303+0000 7fb0237fe640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb01006bbd0 0x7fb01006e090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0241f7410 tx=0x7fb018008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 Processor -- start 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 -- start start 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b89c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b89c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b897210 0x56019b8c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019b864360 con 0x56019b8e4a60 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019b800ec0 con 0x56019b897210 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019b7a9d30 con 0x56019b89aae0 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb023fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b89c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb023fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b89c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49584/0 (socket says 172.21.15.82:49584) 2024-03-14T12:16:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.304+0000 7fb023fff640 1 -- 172.21.15.82:0/2374010587 learned_addr learned my addr 172.21.15.82:0/2374010587 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb0237fe640 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b89c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b897210 0x56019b8c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb023fff640 1 -- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 msgr2=0x56019b89c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb023fff640 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b89c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb023fff640 1 -- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b897210 msgr2=0x56019b8c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb023fff640 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b897210 0x56019b8c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb023fff640 1 -- 172.21.15.82:0/2374010587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019b7a8bc0 con 0x56019b8e4a60 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb0237fe640 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b89c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b897210 0x56019b8c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb023fff640 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b89c090 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb014058360 tx=0x7fb01400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4042d540ca1717 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb009ffb640 1 -- 172.21.15.82:0/2374010587 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140566d0 con 0x56019b8e4a60 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb009ffb640 1 -- 172.21.15.82:0/2374010587 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014054a40 con 0x56019b8e4a60 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb009ffb640 1 -- 172.21.15.82:0/2374010587 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014057820 con 0x56019b8e4a60 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2374010587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 msgr2=0x56019b89c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.305+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b89c090 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb014058360 tx=0x7fb01400a000 comp rx=0 tx=0).stop 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2374010587 shutdown_connections 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b897210 0x56019b8c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b89c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2374010587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b89c090 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2374010587 >> 172.21.15.82:0/2374010587 conn(0x56019b899480 msgr2=0x56019b8e1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2374010587 shutdown_connections 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2374010587 wait complete. 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 -- start start 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.306+0000 7fb02d5c7740 1 Processor -- start 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019b79f4a0 con 0x56019b8bf1b0 2024-03-14T12:16:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019b7fcf60 con 0x56019b8e4a60 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019b89aff0 con 0x56019b89aae0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb02d5c7740 10 client.0 _ll_register_callbacks cb 0x56019b7fad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8e99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49596/0 (socket says 172.21.15.82:49596) 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 -- 172.21.15.82:0/2681933673 learned_addr learned my addr 172.21.15.82:0/2681933673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb0237fe640 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb023fff640 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 -- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 msgr2=0x56019b8beba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8beba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 -- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 msgr2=0x56019b8f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.307+0000 7fb028ea6640 1 -- 172.21.15.82:0/2681933673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56019b5153f0 con 0x56019b8bf1b0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8e99e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb01c02c4f0 tx=0x7fb01c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afecac8df6b94cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb002ffd640 1 -- 172.21.15.82:0/2681933673 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c044070 con 0x56019b8bf1b0 2024-03-14T12:16:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb002ffd640 1 -- 172.21.15.82:0/2681933673 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb01c034b30 con 0x56019b8bf1b0 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb014054a40 con 0x56019b8bf1b0 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb002ffd640 1 -- 172.21.15.82:0/2681933673 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb01c04f050 con 0x56019b8bf1b0 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 10 client.5766 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x56019b5df7d0 con 0x56019b8bf1b0 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb002ffd640 1 -- 172.21.15.82:0/2681933673 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014054a40 con 0x56019b8bf1b0 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb002ffd640 1 -- 172.21.15.82:0/2681933673 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb01c052020 con 0x56019b8bf1b0 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 10 client.5766 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 10 client.5766 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 20 client.5766 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8c5940 0x56019b8c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 4 client.5766 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56019b80e400 con 0x56019b8c5940 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.308+0000 7fb023fff640 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8c5940 0x56019b8c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb002ffd640 10 client.5766 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:12.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb023fff640 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8c5940 0x56019b8c5d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c1c90adde7b82 server_cookie=baa1c8dbbc8f7039 in_seq=0 out_seq=0 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb002ffd640 1 -- 172.21.15.82:0/2681933673 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56019b80e400 con 0x56019b8c5940 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb002ffd640 10 client.5766 handle_command_reply: tid=0 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 client.5766 shutdown 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 msgr2=0x56019b8e99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8e99e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb01c02c4f0 tx=0x7fb01c004ab0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 shutdown_connections 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8e99e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8c5940 0x56019b8c5d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2681933673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb002ffd640 0 client.5766 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.309+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 >> 172.21.15.82:0/2681933673 conn(0x56019b897210 msgr2=0x56019b89cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.310+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 shutdown_connections 2024-03-14T12:16:12.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.310+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2681933673 wait complete. 2024-03-14T12:16:12.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.310+0000 7fb02d5c7740 20 client.5766 trim_cache size 0 max 16384 2024-03-14T12:16:12.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.321+0000 7fb02d5c7740 1 Processor -- start 2024-03-14T12:16:12.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.321+0000 7fb02d5c7740 1 -- start start 2024-03-14T12:16:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8c2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb024200120 con 0x56019b8bf1b0 2024-03-14T12:16:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02415b3b0 con 0x56019b8e4a60 2024-03-14T12:16:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02415a7b0 con 0x56019b89aae0 2024-03-14T12:16:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49606/0 (socket says 172.21.15.82:49606) 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 -- 172.21.15.82:0/3114900620 learned_addr learned my addr 172.21.15.82:0/3114900620 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb028ea6640 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb023fff640 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 -- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 msgr2=0x56019b8c2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8c2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 -- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 msgr2=0x56019b8975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 -- 172.21.15.82:0/3114900620 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019b893190 con 0x56019b8bf1b0 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb023fff640 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8c2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb028ea6640 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.322+0000 7fb0237fe640 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8c25b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb018011d10 tx=0x7fb018014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98214e6504c0bfc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb0097fa640 1 -- 172.21.15.82:0/3114900620 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018026070 con 0x56019b8bf1b0 2024-03-14T12:16:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb0097fa640 1 -- 172.21.15.82:0/3114900620 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01801a040 con 0x56019b8bf1b0 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb0097fa640 1 -- 172.21.15.82:0/3114900620 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01802b460 con 0x56019b8bf1b0 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 -- 172.21.15.82:0/3114900620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 msgr2=0x56019b8c25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8c25b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb018011d10 tx=0x7fb018014000 comp rx=0 tx=0).stop 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 -- 172.21.15.82:0/3114900620 shutdown_connections 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b8e4a60 0x56019b8975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8bf1b0 0x56019b8c25b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/3114900620 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b89aae0 0x56019b8c2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 -- 172.21.15.82:0/3114900620 >> 172.21.15.82:0/3114900620 conn(0x56019b897210 msgr2=0x56019b8c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 -- 172.21.15.82:0/3114900620 shutdown_connections 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.323+0000 7fb02d5c7740 1 -- 172.21.15.82:0/3114900620 wait complete. 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.324+0000 7fb02d5c7740 1 -- start start 2024-03-14T12:16:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.324+0000 7fb02d5c7740 1 Processor -- start 2024-03-14T12:16:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b89aae0 0x56019b8e5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b8bf1b0 0x56019b8bc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b8c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02415adb0 con 0x56019b8e4a60 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019b905930 con 0x56019b89aae0 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019b8bd450 con 0x56019b8bf1b0 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 10 client.0 _ll_register_callbacks cb 0x7fb014054670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b8c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b8c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49622/0 (socket says 172.21.15.82:49622) 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 -- 172.21.15.82:0/2200216275 learned_addr learned my addr 172.21.15.82:0/2200216275 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 -- 172.21.15.82:0/2200216275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b8bf1b0 msgr2=0x56019b8bc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b8bf1b0 0x56019b8bc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb023fff640 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b89aae0 0x56019b8e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 -- 172.21.15.82:0/2200216275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b89aae0 msgr2=0x56019b8e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b89aae0 0x56019b8e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 -- 172.21.15.82:0/2200216275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56019b89af60 con 0x56019b8e4a60 2024-03-14T12:16:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb028ea6640 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b8c4c50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb01c02c4c0 tx=0x7fb01c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec29f71dbdc8ff server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb002ffd640 1 -- 172.21.15.82:0/2200216275 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c044070 con 0x56019b8e4a60 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb002ffd640 1 -- 172.21.15.82:0/2200216275 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb01c003ba0 con 0x56019b8e4a60 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb01801a040 con 0x56019b8e4a60 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb002ffd640 1 -- 172.21.15.82:0/2200216275 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb01c009af0 con 0x56019b8e4a60 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.325+0000 7fb02d5c7740 10 client.5772 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x56019b8e5b90 con 0x56019b8e4a60 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb002ffd640 1 -- 172.21.15.82:0/2200216275 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01801a040 con 0x56019b8e4a60 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb002ffd640 1 -- 172.21.15.82:0/2200216275 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb01c034ce0 con 0x56019b8e4a60 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb02d5c7740 10 client.5772 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb02d5c7740 10 client.5772 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb02d5c7740 20 client.5772 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8be260 0x56019b8c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb02d5c7740 4 client.5772 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x56019b808750 con 0x56019b8be260 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb023fff640 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8be260 0x56019b8c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:12.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb002ffd640 10 client.5772 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.326+0000 7fb023fff640 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8be260 0x56019b8c5190 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca7f166091333998 server_cookie=8b3eebd772e1cd2b in_seq=0 out_seq=0 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb002ffd640 1 -- 172.21.15.82:0/2200216275 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25993 (crc 0 0 0) 0x56019b808750 con 0x56019b8be260 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb002ffd640 10 client.5772 handle_command_reply: tid=0 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 client.5772 shutdown 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 msgr2=0x56019b8c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b8c4c50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb01c02c4c0 tx=0x7fb01c009810 comp rx=0 tx=0).stop 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 shutdown_connections 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56019b8e4a60 0x56019b8c4c50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56019b8bf1b0 0x56019b8bc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x56019b8be260 0x56019b8c5190 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 --2- 172.21.15.82:0/2200216275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019b89aae0 0x56019b8e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb002ffd640 0 client.5772 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 >> 172.21.15.82:0/2200216275 conn(0x56019b897210 msgr2=0x56019b8975f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 shutdown_connections 2024-03-14T12:16:12.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 1 -- 172.21.15.82:0/2200216275 wait complete. 2024-03-14T12:16:12.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02d5c7740 20 client.5772 trim_cache size 0 max 16384 2024-03-14T12:16:12.333 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:12.333 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.334 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:16:12.335 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 68, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 62, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 115025, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 34678, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008550570, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001552942, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.009234633, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000311463, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 62, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.047011846, 2024-03-14T12:16:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000758255 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000163, 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000081 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112512, 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 11440 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.337 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 52, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 53, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13288, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 135895, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 9, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.007161695, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001827893, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005405201, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000385293, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 52, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.077767917, 2024-03-14T12:16:12.338 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001495536 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 65, 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000010538, 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000162 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 90560 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.339 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 29, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 29, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 8003, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 15264, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.003950899, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000499123, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002642060, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000658590, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 27, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.044235489, 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001638351 2024-03-14T12:16:12.340 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 9, 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000001318, 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000146 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.341 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 41, 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001828515, 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000044597 2024-03-14T12:16:12.342 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000023738, 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000011869 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 31, 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 31, 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 31, 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.004132863, 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000133318 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:16:12.344 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 10, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:16:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 61, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 61, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:16:12.346 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:16:12.347 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 1 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:16:12.348 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:16:12.349 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:16:12.350 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "5544" 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 1 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4976", 2024-03-14T12:16:12.351 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 23, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:16:12.352 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 8, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:16:12.353 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 8, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 2, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197140, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:16:12.354 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.008887449, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001777489 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:16:12.355 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 2, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 3, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 42256, 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.356 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 31, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 31, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 5, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 30, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003872370, 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000129079 2024-03-14T12:16:12.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.358 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:16:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:16:12.364 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000260493, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000260493 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.365 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:16:12.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.367 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 29, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 14, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 0, 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:16:12.368 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:16:12.369 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:16:12.370 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 220919, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 61, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:16:12.371 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 51472, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 205, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:16:12.372 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 43, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 13956, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 43, 2024-03-14T12:16:12.373 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 43, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.599988840, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.013953228 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 43, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 67 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 43, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 43, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:16:12.374 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 5, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 7, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:16:12.375 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:16:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 46, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:16:12.377 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 15, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:16:12.378 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.379 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.380 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 142, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 118530, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 142, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 142, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 118530, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:12.382 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 43, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 13956, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 43, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 13956, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:16:12.383 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 43, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 43, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 43, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 43, 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:12.384 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:12.385 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:12.386 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55d2d11906e0": [ 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 8, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2836, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 8, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 5, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2836, 2024-03-14T12:16:12.387 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:16:12.388 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:12.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb01006bbd0 msgr2=0x7fb01006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02a909640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb01006bbd0 0x7fb01006e090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0241f7410 tx=0x7fb018008040 comp rx=0 tx=0).stop 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 msgr2=0x7fb0240cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.327+0000 7fb02a909640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240cbee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb01400c400 tx=0x7fb01400c8c0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 shutdown_connections 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb01006bbd0 0x7fb01006e090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb02415e200 0x7fb024158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb02415de20 0x7fb024148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 --2- 172.21.15.82:0/2239851917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0240d28e0 0x7fb0240cbee0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 >> 172.21.15.82:0/2239851917 conn(0x7fb02409d030 msgr2=0x7fb0240c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 shutdown_connections 2024-03-14T12:16:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:12.328+0000 7fb02a909640 1 -- 172.21.15.82:0/2239851917 wait complete. 2024-03-14T12:16:13.391 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:16:13.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 counter dump 2024-03-14T12:16:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.506+0000 7f7568989640 1 Processor -- start 2024-03-14T12:16:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.506+0000 7f7568989640 1 -- start start 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7568989640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7564145400 0x7f75640723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7568989640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f756406d690 0x7f756415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7568989640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75641457e0 con 0x7f756415f800 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7568989640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75641601a0 con 0x7f756406d690 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7568989640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640ce800 con 0x7f7564145400 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7562d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f756406d690 0x7f756415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f755bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f756415f800 0x7f75640706c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44610/0 (socket says 172.21.15.82:44610) 2024-03-14T12:16:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f755bfff640 1 -- 172.21.15.82:0/2968391168 learned_addr learned my addr 172.21.15.82:0/2968391168 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f756259c640 1 --2- 172.21.15.82:0/2968391168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7564145400 0x7f75640723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7562d9d640 1 -- 172.21.15.82:0/2968391168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7564145400 msgr2=0x7f75640723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7562d9d640 1 --2- 172.21.15.82:0/2968391168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7564145400 0x7f75640723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7562d9d640 1 -- 172.21.15.82:0/2968391168 >> v1:172.21.15.82:6789/0 conn(0x7f756415f800 legacy=0x7f75640706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7562d9d640 1 -- 172.21.15.82:0/2968391168 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640cea50 con 0x7f756406d690 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f7561d9b640 1 -- 172.21.15.82:0/2968391168 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497193982 0 0) 0x7f75641457e0 con 0x7f756415f800 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.507+0000 7f756259c640 1 --2- 172.21.15.82:0/2968391168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7564145400 0x7f75640723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.508+0000 7f7562d9d640 1 --2- 172.21.15.82:0/2968391168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f756406d690 0x7f756415fc60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d964143910863d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.508+0000 7f7561d9b640 1 -- 172.21.15.82:0/2968391168 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c039070 con 0x7f756406d690 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.508+0000 7f7561d9b640 1 -- 172.21.15.82:0/2968391168 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c02cc50 con 0x7f756406d690 2024-03-14T12:16:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.508+0000 7f7561d9b640 1 -- 172.21.15.82:0/2968391168 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c034440 con 0x7f756406d690 2024-03-14T12:16:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.508+0000 7f7568989640 1 -- 172.21.15.82:0/2968391168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f756406d690 msgr2=0x7f756415fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.508+0000 7f7568989640 1 --2- 172.21.15.82:0/2968391168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f756406d690 0x7f756415fc60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c02bea0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.509+0000 7f7568989640 1 -- 172.21.15.82:0/2968391168 shutdown_connections 2024-03-14T12:16:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.509+0000 7f7568989640 1 --2- 172.21.15.82:0/2968391168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f756406d690 0x7f756415fc60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.509+0000 7f7568989640 1 --2- 172.21.15.82:0/2968391168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7564145400 0x7f75640723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.509+0000 7f7568989640 1 -- 172.21.15.82:0/2968391168 >> 172.21.15.82:0/2968391168 conn(0x7f75640cf910 msgr2=0x7f75640cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.509+0000 7f7568989640 1 -- 172.21.15.82:0/2968391168 shutdown_connections 2024-03-14T12:16:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.509+0000 7f7568989640 1 -- 172.21.15.82:0/2968391168 wait complete. 2024-03-14T12:16:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.510+0000 7f7568989640 1 Processor -- start 2024-03-14T12:16:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.510+0000 7f7568989640 1 -- start start 2024-03-14T12:16:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f7568989640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f756406d690 0x7f756413bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f7568989640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7564145400 0x7f7564143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f7568989640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756415f800 0x7f75640b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f7568989640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640cf2f0 con 0x7f7564145400 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f7568989640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640cf010 con 0x7f756415f800 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f7568989640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640cf5d0 con 0x7f756406d690 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7564145400 0x7f7564143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7564145400 0x7f7564143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49644/0 (socket says 172.21.15.82:49644) 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 -- 172.21.15.82:0/3930468753 learned_addr learned my addr 172.21.15.82:0/3930468753 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f756259c640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f756406d690 0x7f756413bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f7562d9d640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756415f800 0x7f75640b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 -- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f756406d690 msgr2=0x7f756413bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f756406d690 0x7f756413bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 -- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756415f800 msgr2=0x7f75640b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756415f800 0x7f75640b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 -- 172.21.15.82:0/3930468753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756413c430 con 0x7f7564145400 2024-03-14T12:16:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755bfff640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7564145400 0x7f7564143e30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f754c00c430 tx=0x7f754c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755affd640 1 -- 172.21.15.82:0/3930468753 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c0074e0 con 0x7f7564145400 2024-03-14T12:16:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.511+0000 7f755affd640 1 -- 172.21.15.82:0/3930468753 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c0047e0 con 0x7f7564145400 2024-03-14T12:16:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.512+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7564157e40 con 0x7f7564145400 2024-03-14T12:16:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.512+0000 7f755affd640 1 -- 172.21.15.82:0/3930468753 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c004bd0 con 0x7f7564145400 2024-03-14T12:16:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.512+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75641586a0 con 0x7f7564145400 2024-03-14T12:16:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.513+0000 7f755affd640 1 -- 172.21.15.82:0/3930468753 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f754c011070 con 0x7f7564145400 2024-03-14T12:16:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.513+0000 7f755affd640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f754406bbd0 0x7f754406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.514+0000 7f755affd640 1 -- 172.21.15.82:0/3930468753 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f754c08a6d0 con 0x7f7564145400 2024-03-14T12:16:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.514+0000 7f756259c640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f754406bbd0 0x7f754406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.514+0000 7f756259c640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f754406bbd0 0x7f754406e090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7554006fd0 tx=0x7f7554008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 Processor -- start 2024-03-14T12:16:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 -- start start 2024-03-14T12:16:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd502090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd5025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd4fd210 0x5648dd52cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648dd4ca360 con 0x5648dd500ae0 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648dd466ec0 con 0x5648dd54aa60 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.515+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648dd40fd30 con 0x5648dd4fd210 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd5025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd5025d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49650/0 (socket says 172.21.15.82:49650) 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 -- 172.21.15.82:0/1090242371 learned_addr learned my addr 172.21.15.82:0/1090242371 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f756259c640 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd502090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f7562d9d640 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd4fd210 0x5648dd52cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 -- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd4fd210 msgr2=0x5648dd52cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd4fd210 0x5648dd52cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 -- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 msgr2=0x5648dd502090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd502090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 -- 172.21.15.82:0/1090242371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648dd40ebc0 con 0x5648dd500ae0 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f755bfff640 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd5025d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f754c088c30 tx=0x7f754c059250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18ee00b4ff61dae server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f7543fff640 1 -- 172.21.15.82:0/1090242371 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c05cca0 con 0x5648dd500ae0 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f7543fff640 1 -- 172.21.15.82:0/1090242371 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c061260 con 0x5648dd500ae0 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.516+0000 7f7543fff640 1 -- 172.21.15.82:0/1090242371 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c004d70 con 0x5648dd500ae0 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 -- 172.21.15.82:0/1090242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 msgr2=0x5648dd5025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd5025d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f754c088c30 tx=0x7f754c059250 comp rx=0 tx=0).stop 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 -- 172.21.15.82:0/1090242371 shutdown_connections 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd4fd210 0x5648dd52cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd5025d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 --2- 172.21.15.82:0/1090242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd502090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 -- 172.21.15.82:0/1090242371 >> 172.21.15.82:0/1090242371 conn(0x5648dd4ff480 msgr2=0x5648dd547d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 -- 172.21.15.82:0/1090242371 shutdown_connections 2024-03-14T12:16:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.517+0000 7f756b56a740 1 -- 172.21.15.82:0/1090242371 wait complete. 2024-03-14T12:16:13.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.518+0000 7f756b56a740 1 -- start start 2024-03-14T12:16:13.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.518+0000 7f756b56a740 1 Processor -- start 2024-03-14T12:16:13.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd500ae0 0x5648dd528700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd54e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54e8b0 0x5648dd558830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648dd4054a0 con 0x5648dd54e8b0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648dd462f60 con 0x5648dd54aa60 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648dd4ca360 con 0x5648dd500ae0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 10 client.0 _ll_register_callbacks cb 0x5648dd460d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54e8b0 0x5648dd558830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54e8b0 0x5648dd558830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49662/0 (socket says 172.21.15.82:49662) 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 -- 172.21.15.82:0/926977306 learned_addr learned my addr 172.21.15.82:0/926977306 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756259c640 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd500ae0 0x5648dd528700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f755bfff640 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd54e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 -- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd500ae0 msgr2=0x5648dd528700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd500ae0 0x5648dd528700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 -- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 msgr2=0x5648dd54e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd54e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 -- 172.21.15.82:0/926977306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648dd17b3f0 con 0x5648dd54e8b0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f7562d9d640 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54e8b0 0x5648dd558830 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7933b17bfcc58669 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f75417fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f75417fa640 1 -- 172.21.15.82:0/926977306 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c045070 con 0x5648dd54e8b0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f75417fa640 1 -- 172.21.15.82:0/926977306 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f755c0346d0 con 0x5648dd54e8b0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f754c061260 con 0x5648dd54e8b0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f75417fa640 1 -- 172.21.15.82:0/926977306 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755c039040 con 0x5648dd54e8b0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 10 client.5787 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.519+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5648dd2457d0 con 0x5648dd54e8b0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f75417fa640 1 -- 172.21.15.82:0/926977306 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c061260 con 0x5648dd54e8b0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f75417fa640 1 -- 172.21.15.82:0/926977306 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f755c003990 con 0x5648dd54e8b0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756b56a740 10 client.5787 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756b56a740 10 client.5787 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756b56a740 20 client.5787 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756b56a740 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd54c910 0x5648dd54cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756b56a740 4 client.5787 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648dd474400 con 0x5648dd54c910 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756259c640 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd54c910 0x5648dd54cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f75417fa640 10 client.5787 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.520+0000 7f756259c640 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd54c910 0x5648dd54cd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbca1c95d51066e3 server_cookie=319111848fe693f1 in_seq=0 out_seq=0 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f75417fa640 1 -- 172.21.15.82:0/926977306 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648dd474400 con 0x5648dd54c910 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f75417fa640 10 client.5787 handle_command_reply: tid=0 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 client.5787 shutdown 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54e8b0 msgr2=0x5648dd558830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54e8b0 0x5648dd558830 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c004ab0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 shutdown_connections 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54e8b0 0x5648dd558830 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd54c910 0x5648dd54cd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54aa60 0x5648dd54e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 --2- 172.21.15.82:0/926977306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd500ae0 0x5648dd528700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f75417fa640 0 client.5787 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 >> 172.21.15.82:0/926977306 conn(0x5648dd4fd210 msgr2=0x5648dd4fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 shutdown_connections 2024-03-14T12:16:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 1 -- 172.21.15.82:0/926977306 wait complete. 2024-03-14T12:16:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.521+0000 7f756b56a740 20 client.5787 trim_cache size 0 max 16384 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.533+0000 7f756b56a740 1 Processor -- start 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.533+0000 7f756b56a740 1 -- start start 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756b56a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd56b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54aa60 0x5648dd525120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54e8b0 0x5648dd525660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756b56a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564200050 con 0x5648dd500ae0 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640ae5a0 con 0x5648dd54e8b0 2024-03-14T12:16:13.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640ad9a0 con 0x5648dd54aa60 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd56b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd56b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49664/0 (socket says 172.21.15.82:49664) 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 -- 172.21.15.82:0/5698804 learned_addr learned my addr 172.21.15.82:0/5698804 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f7562d9d640 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54e8b0 0x5648dd525660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f755bfff640 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54aa60 0x5648dd525120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 -- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54aa60 msgr2=0x5648dd525120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54aa60 0x5648dd525120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 -- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54e8b0 msgr2=0x5648dd525660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54e8b0 0x5648dd525660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 -- 172.21.15.82:0/5698804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648dd54ec90 con 0x5648dd500ae0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f7562d9d640 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54e8b0 0x5648dd525660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.534+0000 7f756259c640 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd56b4e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5648dd54d350 tx=0x7f7554016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e83772517f9024b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f75437fe640 1 -- 172.21.15.82:0/5698804 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755400f040 con 0x5648dd500ae0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f75437fe640 1 -- 172.21.15.82:0/5698804 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554015a40 con 0x5648dd500ae0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f75437fe640 1 -- 172.21.15.82:0/5698804 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554015e10 con 0x5648dd500ae0 2024-03-14T12:16:13.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 -- 172.21.15.82:0/5698804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 msgr2=0x5648dd56b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd56b4e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x5648dd54d350 tx=0x7f7554016000 comp rx=0 tx=0).stop 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 -- 172.21.15.82:0/5698804 shutdown_connections 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd54e8b0 0x5648dd525660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54aa60 0x5648dd525120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 --2- 172.21.15.82:0/5698804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd500ae0 0x5648dd56b4e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 -- 172.21.15.82:0/5698804 >> 172.21.15.82:0/5698804 conn(0x5648dd4fd210 msgr2=0x5648dd528510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 -- 172.21.15.82:0/5698804 shutdown_connections 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.535+0000 7f756b56a740 1 -- 172.21.15.82:0/5698804 wait complete. 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.536+0000 7f756b56a740 1 -- start start 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.536+0000 7f756b56a740 1 Processor -- start 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd500ae0 0x5648dd53e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756b56a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54aa60 0x5648dd5b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756b56a740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54e8b0 0x5648dd547cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756b56a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640adfa0 con 0x5648dd54aa60 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648dd5484a0 con 0x5648dd500ae0 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756b56a740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648dd521f90 con 0x5648dd54e8b0 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756b56a740 10 client.0 _ll_register_callbacks cb 0x7f754c0076a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54aa60 0x5648dd5b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54aa60 0x5648dd5b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49672/0 (socket says 172.21.15.82:49672) 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 -- 172.21.15.82:0/3287510099 learned_addr learned my addr 172.21.15.82:0/3287510099 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f756259c640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd500ae0 0x5648dd53e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f7562d9d640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54e8b0 0x5648dd547cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 -- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54e8b0 msgr2=0x5648dd547cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54e8b0 0x5648dd547cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 -- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd500ae0 msgr2=0x5648dd53e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd500ae0 0x5648dd53e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 -- 172.21.15.82:0/3287510099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648dd562e20 con 0x5648dd54aa60 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f7562d9d640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54e8b0 0x5648dd547cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:13.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f755bfff640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54aa60 0x5648dd5b2ac0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f754c058ae0 tx=0x7f754c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9769b2dedb183851 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.537+0000 7f75417fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f75417fa640 1 -- 172.21.15.82:0/3287510099 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c075060 con 0x5648dd54aa60 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f75417fa640 1 -- 172.21.15.82:0/3287510099 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f754c05dbc0 con 0x5648dd54aa60 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7554015a40 con 0x5648dd54aa60 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f75417fa640 1 -- 172.21.15.82:0/3287510099 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f754c05cca0 con 0x5648dd54aa60 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 10 client.5793 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5648dd521cd0 con 0x5648dd54aa60 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f75417fa640 1 -- 172.21.15.82:0/3287510099 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554015a40 con 0x5648dd54aa60 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f75417fa640 1 -- 172.21.15.82:0/3287510099 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f754c057fa0 con 0x5648dd54aa60 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 10 client.5793 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 10 client.5793 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 20 client.5793 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd53d700 0x5648dd53db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 4 client.5793 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x5648dd52d1e0 con 0x5648dd53d700 2024-03-14T12:16:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756259c640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd53d700 0x5648dd53db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f75417fa640 10 client.5793 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.538+0000 7f756259c640 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd53d700 0x5648dd53db10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3bdd60b396ad973 server_cookie=379ff9a270fd0cd4 in_seq=0 out_seq=0 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f75417fa640 1 -- 172.21.15.82:0/3287510099 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25995 (crc 0 0 0) 0x5648dd52d1e0 con 0x5648dd53d700 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f75417fa640 10 client.5793 handle_command_reply: tid=0 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 client.5793 shutdown 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54aa60 msgr2=0x5648dd5b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54aa60 0x5648dd5b2ac0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f754c058ae0 tx=0x7f754c000ed0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 shutdown_connections 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5648dd54e8b0 0x5648dd547cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648dd54aa60 0x5648dd5b2ac0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x5648dd53d700 0x5648dd53db10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 --2- 172.21.15.82:0/3287510099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5648dd500ae0 0x5648dd53e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f75417fa640 0 client.5793 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 >> 172.21.15.82:0/3287510099 conn(0x5648dd4fd210 msgr2=0x5648dd4fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 shutdown_connections 2024-03-14T12:16:13.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 1 -- 172.21.15.82:0/3287510099 wait complete. 2024-03-14T12:16:13.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.539+0000 7f756b56a740 20 client.5793 trim_cache size 0 max 16384 2024-03-14T12:16:13.546 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:13.546 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:16:13.546 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.546 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:13.546 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:13.547 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.548 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 70, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 64, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 115317, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 79940, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 10, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.009238688, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001628636, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.009720318, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000311463, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 64, 2024-03-14T12:16:13.549 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.047032748, 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000734886 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000341, 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000085 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112512, 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 11440 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:16:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 53, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 54, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13706, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 137687, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 9, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.007326830, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001896000, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005438051, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000437452, 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:13.551 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 53, 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.077820676, 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001468314 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 66, 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000010652, 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000161 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 92352 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.552 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 31, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 29, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 8839, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 15264, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.004108479, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000542278, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002683046, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000727898, 2024-03-14T12:16:13.553 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 27, 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.044235489, 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001638351 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 11, 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000001559, 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000141 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.554 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:13.555 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 41, 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001828515, 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000044597 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:16:13.556 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000023738, 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000011869 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 31, 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 31, 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 31, 2024-03-14T12:16:13.557 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.004132863, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000133318 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 10, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:16:13.558 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 61, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 61, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:16:13.559 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:16:13.560 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 1 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.561 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:16:13.562 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:16:13.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "5544" 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 1 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:16:13.564 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4976", 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 23, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:16:13.565 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:16:13.566 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 8, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 8, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 2, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197140, 2024-03-14T12:16:13.567 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.008887449, 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001777489 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:16:13.568 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 2, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 3, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 42896, 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.569 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 31, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 31, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 5, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:16:13.570 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 30, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003872370, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000129079 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:16:13.571 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.572 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.574 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.575 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.576 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000260493, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000260493 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.578 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.579 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.580 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 29, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 14, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 2, 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:16:13.581 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:16:13.582 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:16:13.583 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:16:13.584 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 224651, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 87, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:16:13.585 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 52240, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 213, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.586 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 43, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 13956, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 43, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 43, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.599988840, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.013953228 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:16:13.587 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 43, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 67 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 43, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 43, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 5, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 7, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:16:13.588 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:16:13.589 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:16:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 46, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 15, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.591 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.592 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:16:13.593 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 147, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 119626, 2024-03-14T12:16:13.594 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 147, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 147, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 119626, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.595 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 43, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 13956, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 43, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 13956, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.596 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 43, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 43, 2024-03-14T12:16:13.597 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 43, 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 43, 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:13.598 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.599 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55d2d11906e0": [ 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 8, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2836, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 8, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 5, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2836, 2024-03-14T12:16:13.600 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:16:13.601 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f754406bbd0 msgr2=0x7f754406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f754406bbd0 0x7f754406e090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7554006fd0 tx=0x7f7554008040 comp rx=0 tx=0).stop 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7564145400 msgr2=0x7f7564143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7564145400 0x7f7564143e30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f754c00c430 tx=0x7f754c00c8f0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 shutdown_connections 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f756415f800 0x7f75640b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7564145400 0x7f7564143e30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f754406bbd0 0x7f754406e090 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 --2- 172.21.15.82:0/3930468753 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f756406d690 0x7f756413bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 >> 172.21.15.82:0/3930468753 conn(0x7f75640cf910 msgr2=0x7f7564070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 shutdown_connections 2024-03-14T12:16:13.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:13.540+0000 7f7568989640 1 -- 172.21.15.82:0/3930468753 wait complete. 2024-03-14T12:16:14.605 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:16:14.605 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 counter dump 2024-03-14T12:16:14.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.720+0000 7f00a9d5b640 1 Processor -- start 2024-03-14T12:16:14.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.720+0000 7f00a9d5b640 1 -- start start 2024-03-14T12:16:14.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.721+0000 7f00a9d5b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 0x7f00a40b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.721+0000 7f00a9d5b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f00a40d2610 0x7f00a40b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.721+0000 7f00a9d5b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a40b0910 con 0x7f00a415c1c0 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.721+0000 7f00a9d5b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a40d0170 con 0x7f00a40d2610 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.721+0000 7f00a9d5b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40d0450 con 0x7f00a40b1b60 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.721+0000 7f00a37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f00a415c1c0 0x7f00a40b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44612/0 (socket says 172.21.15.82:44612) 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.721+0000 7f00a37fe640 1 -- 172.21.15.82:0/2529604364 learned_addr learned my addr 172.21.15.82:0/2529604364 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476197531 0 0) 0x7f00a40b0910 con 0x7f00a415c1c0 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f008c003520 con 0x7f00a415c1c0 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a3fff640 1 --2- 172.21.15.82:0/2529604364 >> v2:172.21.15.164:3300/0 conn(0x7f00a40d2610 0x7f00a40b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 0x7f00a40b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029918683 0 0) 0x7f008c003520 con 0x7f00a415c1c0 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 msgr2=0x7f00a40b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 0x7f00a40b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 >> v2:172.21.15.164:3300/0 conn(0x7f00a40d2610 msgr2=0x7f00a40b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 --2- 172.21.15.82:0/2529604364 >> v2:172.21.15.164:3300/0 conn(0x7f00a40d2610 0x7f00a40b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40d0730 con 0x7f00a415c1c0 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f0094001670 con 0x7f00a415c1c0 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 >> v1:172.21.15.82:6789/0 conn(0x7f00a415c1c0 legacy=0x7f00a40b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c0039b0 0x7f008c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 0x7f008c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c004d40 0x7f008c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40d0170 con 0x7f008c0039b0 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40d0450 con 0x7f008c004d40 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c00bb20 con 0x7f00a40b1b60 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a3fff640 1 --2- 172.21.15.82:0/2529604364 >> v2:172.21.15.164:3300/0 conn(0x7f00a40d2610 0x7f00a40b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a37fe640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c0039b0 0x7f008c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 0x7f008c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a3fff640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c004d40 0x7f008c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a37fe640 1 -- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 msgr2=0x7f008c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a37fe640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40b1b60 0x7f008c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a37fe640 1 -- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c004d40 msgr2=0x7f008c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a37fe640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c004d40 0x7f008c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a37fe640 1 -- 172.21.15.82:0/2529604364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c003520 con 0x7f008c0039b0 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.722+0000 7f00a3fff640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c004d40 0x7f008c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.723+0000 7f00a37fe640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c0039b0 0x7f008c003dd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f009400c240 tx=0x7f008c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f888fa7bf11855 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.723+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094007d60 con 0x7f008c0039b0 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.723+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094030070 con 0x7f008c0039b0 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.723+0000 7f00a27fc640 1 -- 172.21.15.82:0/2529604364 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009402cd70 con 0x7f008c0039b0 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.724+0000 7f00a9d5b640 1 -- 172.21.15.82:0/2529604364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c0039b0 msgr2=0x7f008c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.724+0000 7f00a9d5b640 1 --2- 172.21.15.82:0/2529604364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c0039b0 0x7f008c003dd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f009400c240 tx=0x7f008c005830 comp rx=0 tx=0).stop 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.724+0000 7f00a3fff640 1 -- 172.21.15.82:0/2529604364 reap_dead start 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.724+0000 7f00a9d5b640 1 -- 172.21.15.82:0/2529604364 shutdown_connections 2024-03-14T12:16:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.724+0000 7f00a9d5b640 1 -- 172.21.15.82:0/2529604364 >> 172.21.15.82:0/2529604364 conn(0x7f00a40d1af0 msgr2=0x7f00a40d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.724+0000 7f00a9d5b640 1 -- 172.21.15.82:0/2529604364 shutdown_connections 2024-03-14T12:16:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.724+0000 7f00a9d5b640 1 -- 172.21.15.82:0/2529604364 wait complete. 2024-03-14T12:16:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.725+0000 7f00a9d5b640 1 Processor -- start 2024-03-14T12:16:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.725+0000 7f00a9d5b640 1 -- start start 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a9d5b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c0039b0 0x7f00a406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a9d5b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40cb5a0 0x7f00a406d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a9d5b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40b68b0 0x7f00a40b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a9d5b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40c57c0 con 0x7f00a40b68b0 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a9d5b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40c5640 con 0x7f008c0039b0 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a9d5b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40b02a0 con 0x7f00a40cb5a0 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40b68b0 0x7f00a40b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40b68b0 0x7f00a40b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49682/0 (socket says 172.21.15.82:49682) 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 -- 172.21.15.82:0/4241218934 learned_addr learned my addr 172.21.15.82:0/4241218934 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c0039b0 0x7f00a406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40cb5a0 0x7f00a406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 -- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40cb5a0 msgr2=0x7f00a406d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40cb5a0 0x7f00a406d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 -- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c0039b0 msgr2=0x7f00a406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c0039b0 0x7f00a406d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.726+0000 7f00a3fff640 1 -- 172.21.15.82:0/4241218934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40c8f80 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.727+0000 7f00a3fff640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40b68b0 0x7f00a40b6d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f00a4071390 tx=0x7f009c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.727+0000 7f008bfff640 1 -- 172.21.15.82:0/4241218934 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c019070 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.727+0000 7f008bfff640 1 -- 172.21.15.82:0/4241218934 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c007cb0 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.727+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a415f0b0 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.727+0000 7f008bfff640 1 -- 172.21.15.82:0/4241218934 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c007e60 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.727+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a406d580 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.728+0000 7f008bfff640 1 -- 172.21.15.82:0/4241218934 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f009c01c070 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.729+0000 7f008bfff640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f009006bc00 0x7f009006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.729+0000 7f008bfff640 1 -- 172.21.15.82:0/4241218934 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f009c08a440 con 0x7f00a40b68b0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.729+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f009006bc00 0x7f009006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.729+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f009006bc00 0x7f009006e0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0094034000 tx=0x7f0094009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.730+0000 7f00aca19740 1 Processor -- start 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.730+0000 7f00aca19740 1 -- start start 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00aca19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2b10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b0eae0 0x558ff2b105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0b210 0x558ff2b3ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00aca19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2ad8360 con 0x558ff2b58a60 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2a74ec0 con 0x558ff2b0eae0 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ff2a1dd30 con 0x558ff2b0b210 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2b10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2b10090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49686/0 (socket says 172.21.15.82:49686) 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 -- 172.21.15.82:0/4174416315 learned_addr learned my addr 172.21.15.82:0/4174416315 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a3fff640 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0b210 0x558ff2b3ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b0eae0 0x558ff2b105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 -- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0b210 msgr2=0x558ff2b3ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0b210 0x558ff2b3ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 -- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b0eae0 msgr2=0x558ff2b105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b0eae0 0x558ff2b105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 -- 172.21.15.82:0/4174416315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ff2a1cbc0 con 0x558ff2b58a60 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2b10090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f009402c5a0 tx=0x7f0094007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c36e004dcb2444 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00897fa640 1 -- 172.21.15.82:0/4174416315 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094004cb0 con 0x558ff2b58a60 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00897fa640 1 -- 172.21.15.82:0/4174416315 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094007070 con 0x558ff2b58a60 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.731+0000 7f00897fa640 1 -- 172.21.15.82:0/4174416315 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094007460 con 0x558ff2b58a60 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 -- 172.21.15.82:0/4174416315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 msgr2=0x558ff2b10090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2b10090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f009402c5a0 tx=0x7f0094007000 comp rx=0 tx=0).stop 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 -- 172.21.15.82:0/4174416315 shutdown_connections 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0b210 0x558ff2b3ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b0eae0 0x558ff2b105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 --2- 172.21.15.82:0/4174416315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2b10090 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 -- 172.21.15.82:0/4174416315 >> 172.21.15.82:0/4174416315 conn(0x558ff2b0d480 msgr2=0x558ff2b55d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:14.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 -- 172.21.15.82:0/4174416315 shutdown_connections 2024-03-14T12:16:14.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.732+0000 7f00aca19740 1 -- 172.21.15.82:0/4174416315 wait complete. 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.733+0000 7f00aca19740 1 -- start start 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.733+0000 7f00aca19740 1 Processor -- start 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b5c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b58a60 0x558ff2b397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b255d0 0x558ff2b39d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2a134a0 con 0x558ff2b0eae0 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2a70f60 con 0x558ff2b58a60 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ff2b0eff0 con 0x558ff2b255d0 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b5c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 10 client.0 _ll_register_callbacks cb 0x558ff2a6ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b5c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49698/0 (socket says 172.21.15.82:49698) 2024-03-14T12:16:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 -- 172.21.15.82:0/869650037 learned_addr learned my addr 172.21.15.82:0/869650037 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b58a60 0x558ff2b397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a3fff640 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b255d0 0x558ff2b39d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 -- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b255d0 msgr2=0x558ff2b39d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b255d0 0x558ff2b39d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 -- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b58a60 msgr2=0x558ff2b397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b58a60 0x558ff2b397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 -- 172.21.15.82:0/869650037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ff27893f0 con 0x558ff2b0eae0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00a37fe640 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b5c750 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f00940017e0 tx=0x7f009402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f21640295d6e0a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00827fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00827fc640 1 -- 172.21.15.82:0/869650037 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094004cb0 con 0x558ff2b0eae0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00827fc640 1 -- 172.21.15.82:0/869650037 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f009404a050 con 0x558ff2b0eae0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.734+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0094007070 con 0x558ff2b0eae0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00827fc640 1 -- 172.21.15.82:0/869650037 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00940077c0 con 0x558ff2b0eae0 2024-03-14T12:16:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 10 client.5805 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x558ff28537d0 con 0x558ff2b0eae0 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00827fc640 1 -- 172.21.15.82:0/869650037 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094007070 con 0x558ff2b0eae0 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00827fc640 1 -- 172.21.15.82:0/869650037 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0094035680 con 0x558ff2b0eae0 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 10 client.5805 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 10 client.5805 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 20 client.5805 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2b3ba90 0x558ff2b3be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 4 client.5805 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ff2a82400 con 0x558ff2b3ba90 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2b3ba90 0x558ff2b3be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00827fc640 10 client.5805 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.735+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2b3ba90 0x558ff2b3be90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168ff4b42fd57279 server_cookie=103cc2a5c92b884b in_seq=0 out_seq=0 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00827fc640 1 -- 172.21.15.82:0/869650037 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ff2a82400 con 0x558ff2b3ba90 2024-03-14T12:16:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00827fc640 10 client.5805 handle_command_reply: tid=0 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 client.5805 shutdown 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 msgr2=0x558ff2b5c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b5c750 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f00940017e0 tx=0x7f009402e070 comp rx=0 tx=0).stop 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 shutdown_connections 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b255d0 0x558ff2b39d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2b3ba90 0x558ff2b3be90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b58a60 0x558ff2b397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 --2- 172.21.15.82:0/869650037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b5c750 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 >> 172.21.15.82:0/869650037 conn(0x558ff2b0b210 msgr2=0x558ff2b10ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00827fc640 0 client.5805 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 shutdown_connections 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 1 -- 172.21.15.82:0/869650037 wait complete. 2024-03-14T12:16:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.736+0000 7f00aca19740 20 client.5805 trim_cache size 0 max 16384 2024-03-14T12:16:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.748+0000 7f00aca19740 1 Processor -- start 2024-03-14T12:16:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.748+0000 7f00aca19740 1 -- start start 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00aca19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b58a60 0x558ff2b393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00aca19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a42007c0 con 0x558ff2b0eae0 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41ffb20 con 0x558ff2b255d0 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a415e0a0 con 0x558ff2b58a60 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b54ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49714/0 (socket says 172.21.15.82:49714) 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 -- 172.21.15.82:0/2298296573 learned_addr learned my addr 172.21.15.82:0/2298296573 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a3fff640 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b58a60 0x558ff2b393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 -- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b58a60 msgr2=0x558ff2b393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b58a60 0x558ff2b393a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 -- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 msgr2=0x558ff2b55000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b55000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 -- 172.21.15.82:0/2298296573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ff2a7a140 con 0x558ff2b0eae0 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f00a37fe640 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b54ac0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f009400c160 tx=0x7f0094053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47d264a16c8abe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f0088ff9640 1 -- 172.21.15.82:0/2298296573 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094004c30 con 0x558ff2b0eae0 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f0088ff9640 1 -- 172.21.15.82:0/2298296573 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094007530 con 0x558ff2b0eae0 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.749+0000 7f0088ff9640 1 -- 172.21.15.82:0/2298296573 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094035b80 con 0x558ff2b0eae0 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 -- 172.21.15.82:0/2298296573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 msgr2=0x558ff2b54ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b54ac0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f009400c160 tx=0x7f0094053000 comp rx=0 tx=0).stop 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 -- 172.21.15.82:0/2298296573 shutdown_connections 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b58a60 0x558ff2b393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b55000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 --2- 172.21.15.82:0/2298296573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b0eae0 0x558ff2b54ac0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 -- 172.21.15.82:0/2298296573 >> 172.21.15.82:0/2298296573 conn(0x558ff2b0b210 msgr2=0x558ff2b56190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:14.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 -- 172.21.15.82:0/2298296573 shutdown_connections 2024-03-14T12:16:14.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.750+0000 7f00aca19740 1 -- 172.21.15.82:0/2298296573 wait complete. 2024-03-14T12:16:14.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 -- start start 2024-03-14T12:16:14.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 Processor -- start 2024-03-14T12:16:14.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0eae0 0x558ff2b3c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b39bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2ba5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2b3b310 con 0x558ff2b58a60 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a415e6a0 con 0x558ff2b255d0 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ff2b69900 con 0x558ff2b0eae0 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00aca19740 10 client.0 _ll_register_callbacks cb 0x7f008c00b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2ba5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.751+0000 7f00a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2ba5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49726/0 (socket says 172.21.15.82:49726) 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a3fff640 1 -- 172.21.15.82:0/4208117370 learned_addr learned my addr 172.21.15.82:0/4208117370 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a2ffd640 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b39bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0eae0 0x558ff2b3c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a3fff640 1 -- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0eae0 msgr2=0x558ff2b3c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a3fff640 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0eae0 0x558ff2b3c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a3fff640 1 -- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 msgr2=0x558ff2b39bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a3fff640 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b39bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a3fff640 1 -- 172.21.15.82:0/4208117370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ff2b3a110 con 0x558ff2b58a60 2024-03-14T12:16:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00a3fff640 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2ba5fa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f009c002230 tx=0x7f009c058a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9950cbc4ac3981 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00827fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00827fc640 1 -- 172.21.15.82:0/4208117370 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c060ab0 con 0x558ff2b58a60 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00827fc640 1 -- 172.21.15.82:0/4208117370 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f009c060c50 con 0x558ff2b58a60 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0094007530 con 0x558ff2b58a60 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00827fc640 1 -- 172.21.15.82:0/4208117370 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f009c06cb80 con 0x558ff2b58a60 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 10 client.5811 fetch_fsmap learned FSMap version 21 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x558ff2b56490 con 0x558ff2b58a60 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00827fc640 1 -- 172.21.15.82:0/4208117370 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094007530 con 0x558ff2b58a60 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00827fc640 1 -- 172.21.15.82:0/4208117370 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f009c059b60 con 0x558ff2b58a60 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 10 client.5811 fetch_fsmap finished waiting for FSMap version 21 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 10 client.5811 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 20 client.5811 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2ba64e0 0x558ff2b36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 4 client.5811 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:16:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.752+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 --> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x558ff2a7c750 con 0x558ff2ba64e0 2024-03-14T12:16:14.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2ba64e0 0x558ff2b36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:14.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00827fc640 10 client.5811 ms_handle_connect on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:14.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00a37fe640 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2ba64e0 0x558ff2b36f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4a3ddebf2273f7 server_cookie=f7933be368388a59 in_seq=0 out_seq=0 2024-03-14T12:16:14.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00827fc640 1 -- 172.21.15.82:0/4208117370 <== mds.0 v2:172.21.15.82:6835/2708572159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26752 (crc 0 0 0) 0x558ff2a7c750 con 0x558ff2ba64e0 2024-03-14T12:16:14.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00827fc640 10 client.5811 handle_command_reply: tid=0 2024-03-14T12:16:14.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00aca19740 1 client.5811 shutdown 2024-03-14T12:16:14.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 msgr2=0x558ff2ba5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.753+0000 7f00aca19740 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2ba5fa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f009c002230 tx=0x7f009c058a60 comp rx=0 tx=0).stop 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 shutdown_connections 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ff2b58a60 0x558ff2ba5fa0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.82:6835/2708572159,v1:172.21.15.82:6837/2708572159] conn(0x558ff2ba64e0 0x558ff2b36f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00827fc640 0 client.5811 ms_handle_reset on v2:172.21.15.82:6835/2708572159 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ff2b255d0 0x558ff2b39bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 --2- 172.21.15.82:0/4208117370 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x558ff2b0eae0 0x558ff2b3c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 >> 172.21.15.82:0/4208117370 conn(0x558ff2b0b210 msgr2=0x558ff2b54cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 shutdown_connections 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 1 -- 172.21.15.82:0/4208117370 wait complete. 2024-03-14T12:16:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00aca19740 20 client.5811 trim_cache size 0 max 16384 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.764 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 73, 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 67, 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 115881, 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 125828, 2024-03-14T12:16:14.765 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 12, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.010048127, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001753255, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.010235621, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000329870, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 67, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.047075540, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000702620 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000516, 2024-03-14T12:16:14.766 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000086 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112784, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 12064 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 54, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 54, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 14124, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 137687, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 9, 2024-03-14T12:16:14.767 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.007545336, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001968136, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005498096, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000519340, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 53, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.077820676, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001468314 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 67, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000010852, 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000161 2024-03-14T12:16:14.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 92352 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 32, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 29, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 9257, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 15264, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:16:14.769 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.004270874, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000581933, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002732662, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000795837, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 27, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.044235489, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001638351 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 12, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000001773, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000147 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.770 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 41, 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001828515, 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000044597 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000023738, 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000011869 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 31, 2024-03-14T12:16:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 31, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 31, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.004132863, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000133318 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:16:14.774 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 10, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 61, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 61, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:16:14.775 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:16:14.776 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 1 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.777 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:16:14.778 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.779 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "5544" 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 2 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.4976", 2024-03-14T12:16:14.780 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 23, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:16:14.781 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.5293", 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 41, 2024-03-14T12:16:14.782 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.783 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 8, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 8, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 2, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:16:14.784 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197140, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.008887449, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001777489 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.785 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:16:14.786 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 2, 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 3, 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1, 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 43024, 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 31, 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:16:14.787 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 31, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 5, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 30, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003872370, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000129079 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:16:14.788 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:16:14.789 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.790 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.791 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.793 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.794 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000260493, 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000260493 2024-03-14T12:16:14.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:16:14.796 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:16:14.797 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-03-14T12:16:14.798 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 29, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 14, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 2, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:16:14.799 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:16:14.800 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 224651, 2024-03-14T12:16:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 87, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:16:14.802 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 52240, 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 213, 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.803 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 43, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 13956, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 43, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 43, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.599988840, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.013953228 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 43, 2024-03-14T12:16:14.804 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 67 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 43, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 43, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 5, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 7, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:16:14.805 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:16:14.806 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:16:14.807 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 46, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 15, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.808 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.809 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:16:14.810 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 152, 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 120524, 2024-03-14T12:16:14.811 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 152, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 152, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 120524, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:16:14.812 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 43, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 13956, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 43, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 13956, 2024-03-14T12:16:14.813 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:14.814 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 43, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 43, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 43, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 43, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.815 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:16:14.816 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55d2d11906e0": [ 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:14.817 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 8, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2836, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 8, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 5, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2836, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.818 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:14.819 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:16:14.819 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f009006bc00 msgr2=0x7f009006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f009006bc00 0x7f009006e0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0094034000 tx=0x7f0094009030 comp rx=0 tx=0).stop 2024-03-14T12:16:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40b68b0 msgr2=0x7f00a40b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40b68b0 0x7f00a40b6d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f00a4071390 tx=0x7f009c004840 comp rx=0 tx=0).stop 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 shutdown_connections 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a40b68b0 0x7f00a40b6d60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f009006bc00 0x7f009006e0c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00a40cb5a0 0x7f00a406d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 --2- 172.21.15.82:0/4241218934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f008c0039b0 0x7f00a406d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 >> 172.21.15.82:0/4241218934 conn(0x7f00a40d1af0 msgr2=0x7f00a40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 shutdown_connections 2024-03-14T12:16:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:14.754+0000 7f00a9d5b640 1 -- 172.21.15.82:0/4241218934 wait complete. 2024-03-14T12:16:14.822 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:16:14.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:14.863 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:14.908 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:16:14.908 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:14.908 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:14.924 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:14.924 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:14.979 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:15.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.120+0000 7f5da3f89640 1 Processor -- start 2024-03-14T12:16:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.121+0000 7f5da3f89640 1 -- start start 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.121+0000 7f5da3f89640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d9c15e670 0x7f5d9c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.121+0000 7f5da3f89640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.121+0000 7f5da3f89640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c15ebe0 con 0x7f5d9c071740 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.121+0000 7f5da3f89640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c15eec0 con 0x7f5d9c15e670 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.121+0000 7f5da3f89640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c15f1a0 con 0x7f5d9c143f60 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da1524640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d9c15e670 0x7f5d9c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da2526640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da1d25640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5d9c071740 0x7f5d9c071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:56542/0 (socket says 172.21.15.82:56542) 2024-03-14T12:16:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da1d25640 1 -- 172.21.15.82:0/1883991946 learned_addr learned my addr 172.21.15.82:0/1883991946 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da1524640 1 -- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 msgr2=0x7f5d9c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da1524640 1 --2- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da1524640 1 -- 172.21.15.82:0/1883991946 >> v1:172.21.15.164:6789/0 conn(0x7f5d9c071740 legacy=0x7f5d9c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da1524640 1 -- 172.21.15.82:0/1883991946 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c15f480 con 0x7f5d9c15e670 2024-03-14T12:16:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.122+0000 7f5da2526640 1 --2- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.123+0000 7f5da1524640 1 --2- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 0x7f5d9c0b2940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097d0 tx=0x7f5d8c02b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2719dcaf899e608 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.123+0000 7f5da0d23640 1 -- 172.21.15.82:0/1883991946 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c035020 con 0x7f5d9c15e670 2024-03-14T12:16:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.123+0000 7f5da0d23640 1 -- 172.21.15.82:0/1883991946 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c031800 con 0x7f5d9c15e670 2024-03-14T12:16:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.123+0000 7f5da0d23640 1 -- 172.21.15.82:0/1883991946 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c031bd0 con 0x7f5d9c15e670 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 -- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 msgr2=0x7f5d9c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 --2- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 0x7f5d9c0b2940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097d0 tx=0x7f5d8c02b8a0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 -- 172.21.15.82:0/1883991946 shutdown_connections 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 --2- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 --2- 172.21.15.82:0/1883991946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 0x7f5d9c0b2940 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 -- 172.21.15.82:0/1883991946 >> 172.21.15.82:0/1883991946 conn(0x7f5d9c0b1c70 msgr2=0x7f5d9c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 -- 172.21.15.82:0/1883991946 shutdown_connections 2024-03-14T12:16:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.124+0000 7f5da3f89640 1 -- 172.21.15.82:0/1883991946 wait complete. 2024-03-14T12:16:15.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.125+0000 7f5da3f89640 1 Processor -- start 2024-03-14T12:16:15.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.125+0000 7f5da3f89640 1 -- start start 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da3f89640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c071740 0x7f5d9c140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da3f89640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da3f89640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 0x7f5d9c1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da3f89640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c144670 con 0x7f5d9c071740 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da3f89640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c15b080 con 0x7f5d9c15e670 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da3f89640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1447f0 con 0x7f5d9c143f60 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c071740 0x7f5d9c140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c071740 0x7f5d9c140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49728/0 (socket says 172.21.15.82:49728) 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 -- 172.21.15.82:0/3257211047 learned_addr learned my addr 172.21.15.82:0/3257211047 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1524640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da2526640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 0x7f5d9c1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 -- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 msgr2=0x7f5d9c157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 -- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 msgr2=0x7f5d9c1575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 0x7f5d9c1575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 -- 172.21.15.82:0/3257211047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c149d40 con 0x7f5d9c071740 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.126+0000 7f5da1d25640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c071740 0x7f5d9c140e10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c3c0 tx=0x7f5d9800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.127+0000 7f5d927fc640 1 -- 172.21.15.82:0/3257211047 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98017070 con 0x7f5d9c071740 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.127+0000 7f5d927fc640 1 -- 172.21.15.82:0/3257211047 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d98005ce0 con 0x7f5d9c071740 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.127+0000 7f5d927fc640 1 -- 172.21.15.82:0/3257211047 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980072f0 con 0x7f5d9c071740 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.127+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c14a020 con 0x7f5d9c071740 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.127+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c14a5e0 con 0x7f5d9c071740 2024-03-14T12:16:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.128+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c04f050 con 0x7f5d9c071740 2024-03-14T12:16:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.128+0000 7f5d927fc640 1 -- 172.21.15.82:0/3257211047 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5d9801a070 con 0x7f5d9c071740 2024-03-14T12:16:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.128+0000 7f5d927fc640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.129+0000 7f5d927fc640 1 -- 172.21.15.82:0/3257211047 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d98089c80 con 0x7f5d9c071740 2024-03-14T12:16:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.129+0000 7f5da1524640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.129+0000 7f5da1524640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c036000 tx=0x7f5d8c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.133+0000 7f5d927fc640 1 -- 172.21.15.82:0/3257211047 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5d98058120 con 0x7f5d9c071740 2024-03-14T12:16:15.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.305+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d9c13b020 con 0x7f5d9c071740 2024-03-14T12:16:15.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.306+0000 7f5d927fc640 1 -- 172.21.15.82:0/3257211047 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f5d9805c1c0 con 0x7f5d9c071740 2024-03-14T12:16:15.305 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:15.305 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:03.671135+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:15.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 msgr2=0x7f5d8006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c036000 tx=0x7f5d8c008030 comp rx=0 tx=0).stop 2024-03-14T12:16:15.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c071740 msgr2=0x7f5d9c140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c071740 0x7f5d9c140e10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c3c0 tx=0x7f5d9800c7c0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 shutdown_connections 2024-03-14T12:16:15.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d9c15e670 0x7f5d9c1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5d8006bbd0 0x7f5d8006e090 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5d9c143f60 0x7f5d9c157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 --2- 172.21.15.82:0/3257211047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c071740 0x7f5d9c140e10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 >> 172.21.15.82:0/3257211047 conn(0x7f5d9c0b1c70 msgr2=0x7f5d9c158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:15.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 shutdown_connections 2024-03-14T12:16:15.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.308+0000 7f5da3f89640 1 -- 172.21.15.82:0/3257211047 wait complete. 2024-03-14T12:16:15.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:00.232652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:00.983002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:15.318 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:15.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.421+0000 7f159748e640 1 Processor -- start 2024-03-14T12:16:15.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f159748e640 1 -- start start 2024-03-14T12:16:15.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f159748e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f159015bde0 0x7f159015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f159748e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f15900cd250 0x7f15900cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f159748e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900ce380 con 0x7f15900d01b0 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f159748e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15900cef90 con 0x7f15900cd250 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f159748e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900b5a30 con 0x7f159015bde0 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f1595a2b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f15900d01b0 0x7f15900b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44616/0 (socket says 172.21.15.82:44616) 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.422+0000 7f1595a2b640 1 -- 172.21.15.82:0/2304698000 learned_addr learned my addr 172.21.15.82:0/2304698000 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1594a29640 1 --2- 172.21.15.82:0/2304698000 >> v2:172.21.15.164:3300/0 conn(0x7f15900cd250 0x7f15900cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f159522a640 1 --2- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f159015bde0 0x7f159015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1587fff640 1 -- 172.21.15.82:0/2304698000 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993524888 0 0) 0x7f15900ce380 con 0x7f15900d01b0 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1587fff640 1 -- 172.21.15.82:0/2304698000 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f157c003520 con 0x7f15900d01b0 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1594a29640 1 -- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f159015bde0 msgr2=0x7f159015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1587fff640 1 -- 172.21.15.82:0/2304698000 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1418577956 0 0) 0x7f157c003520 con 0x7f15900d01b0 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1594a29640 1 --2- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f159015bde0 0x7f159015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1594a29640 1 -- 172.21.15.82:0/2304698000 >> v1:172.21.15.82:6789/0 conn(0x7f15900d01b0 legacy=0x7f15900b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f1594a29640 1 -- 172.21.15.82:0/2304698000 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900ce660 con 0x7f15900cd250 2024-03-14T12:16:15.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.423+0000 7f159522a640 1 --2- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f159015bde0 0x7f159015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.424+0000 7f1594a29640 1 --2- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15900cd250 0x7f15900cd690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f15800097a0 tx=0x7f158002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bcc8159ef4f32a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.424+0000 7f1587fff640 1 -- 172.21.15.82:0/2304698000 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580038070 con 0x7f15900cd250 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.424+0000 7f1587fff640 1 -- 172.21.15.82:0/2304698000 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580031500 con 0x7f15900cd250 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.424+0000 7f1587fff640 1 -- 172.21.15.82:0/2304698000 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588001670 con 0x7f15900cd250 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 -- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15900cd250 msgr2=0x7f15900cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 --2- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15900cd250 0x7f15900cd690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f15800097a0 tx=0x7f158002b820 comp rx=0 tx=0).stop 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 -- 172.21.15.82:0/2304698000 shutdown_connections 2024-03-14T12:16:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 --2- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15900cd250 0x7f15900cd690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 --2- 172.21.15.82:0/2304698000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f159015bde0 0x7f159015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 -- 172.21.15.82:0/2304698000 >> 172.21.15.82:0/2304698000 conn(0x7f15900d1bd0 msgr2=0x7f15900d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 -- 172.21.15.82:0/2304698000 shutdown_connections 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.425+0000 7f159748e640 1 -- 172.21.15.82:0/2304698000 wait complete. 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 Processor -- start 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 -- start start 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15900cd250 0x7f15901f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15900d01b0 0x7f15901f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f159015bde0 0x7f15901f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900cf420 con 0x7f15900cd250 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900cec20 con 0x7f159015bde0 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.426+0000 7f159748e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900d0640 con 0x7f15900d01b0 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15900cd250 0x7f15901f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15900cd250 0x7f15901f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49730/0 (socket says 172.21.15.82:49730) 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 -- 172.21.15.82:0/1112759153 learned_addr learned my addr 172.21.15.82:0/1112759153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f1595a2b640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f159015bde0 0x7f15901f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f1594a29640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15900d01b0 0x7f15901f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 -- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15900d01b0 msgr2=0x7f15901f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15900d01b0 0x7f15901f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 -- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f159015bde0 msgr2=0x7f15901f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f159015bde0 0x7f15901f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 -- 172.21.15.82:0/1112759153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15901f3ac0 con 0x7f15900cd250 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f1594a29640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15900d01b0 0x7f15901f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159522a640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15900cd250 0x7f15901f4d70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f158c00efc0 tx=0x7f158c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f1585ffb640 1 -- 172.21.15.82:0/1112759153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c018020 con 0x7f15900cd250 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f1585ffb640 1 -- 172.21.15.82:0/1112759153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c014680 con 0x7f15900cd250 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f1585ffb640 1 -- 172.21.15.82:0/1112759153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c014a50 con 0x7f15900cd250 2024-03-14T12:16:15.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15901f3da0 con 0x7f15900cd250 2024-03-14T12:16:15.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.427+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901f4360 con 0x7f15900cd250 2024-03-14T12:16:15.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.428+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15900c8760 con 0x7f15900cd250 2024-03-14T12:16:15.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.429+0000 7f1585ffb640 1 -- 172.21.15.82:0/1112759153 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f158c005d10 con 0x7f15900cd250 2024-03-14T12:16:15.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.429+0000 7f1585ffb640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f157406bbd0 0x7f157406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.429+0000 7f1585ffb640 1 -- 172.21.15.82:0/1112759153 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f158c08b760 con 0x7f15900cd250 2024-03-14T12:16:15.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.429+0000 7f1594a29640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f157406bbd0 0x7f157406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.430+0000 7f1594a29640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f157406bbd0 0x7f157406e090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1580035000 tx=0x7f1580008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.434+0000 7f1585ffb640 1 -- 172.21.15.82:0/1112759153 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f158c022030 con 0x7f15900cd250 2024-03-14T12:16:15.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.605+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f159004f050 con 0x7f15900cd250 2024-03-14T12:16:15.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.605+0000 7f1585ffb640 1 -- 172.21.15.82:0/1112759153 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f158c010040 con 0x7f15900cd250 2024-03-14T12:16:15.605 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f157406bbd0 msgr2=0x7f157406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f157406bbd0 0x7f157406e090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1580035000 tx=0x7f1580008030 comp rx=0 tx=0).stop 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15900cd250 msgr2=0x7f15901f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15900cd250 0x7f15901f4d70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f158c00efc0 tx=0x7f158c00c490 comp rx=0 tx=0).stop 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 shutdown_connections 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f159015bde0 0x7f15901f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f157406bbd0 0x7f157406e090 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15900d01b0 0x7f15901f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 --2- 172.21.15.82:0/1112759153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15900cd250 0x7f15901f4d70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 >> 172.21.15.82:0/1112759153 conn(0x7f15900d1bd0 msgr2=0x7f15900b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 shutdown_connections 2024-03-14T12:16:15.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.607+0000 7f159748e640 1 -- 172.21.15.82:0/1112759153 wait complete. 2024-03-14T12:16:15.617 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:16:15.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.722+0000 7fbd5735b640 1 Processor -- start 2024-03-14T12:16:15.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd5735b640 1 -- start start 2024-03-14T12:16:15.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd5735b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbd50158290 0x7fbd500d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd5735b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd500b0030 0x7fbd500d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd5735b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5015e500 con 0x7fbd50158670 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd5735b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5015e7a0 con 0x7fbd500b0030 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd5735b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5015ea80 con 0x7fbd50158290 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd558f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd500b0030 0x7fbd500d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd558f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd500b0030 0x7fbd500d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49740/0 (socket says 172.21.15.82:49740) 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.723+0000 7fbd558f8640 1 -- 172.21.15.82:0/858917461 learned_addr learned my addr 172.21.15.82:0/858917461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.724+0000 7fbd558f8640 1 -- 172.21.15.82:0/858917461 >> v2:172.21.15.164:3300/0 conn(0x7fbd50158290 msgr2=0x7fbd500d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.724+0000 7fbd558f8640 1 --2- 172.21.15.82:0/858917461 >> v2:172.21.15.164:3300/0 conn(0x7fbd50158290 0x7fbd500d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.724+0000 7fbd558f8640 1 -- 172.21.15.82:0/858917461 >> v1:172.21.15.164:6789/0 conn(0x7fbd50158670 legacy=0x7fbd500af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.724+0000 7fbd558f8640 1 -- 172.21.15.82:0/858917461 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5015ed60 con 0x7fbd500b0030 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.724+0000 7fbd558f8640 1 --2- 172.21.15.82:0/858917461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd500b0030 0x7fbd500d1340 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbd500b5580 tx=0x7fbd4802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40810effaffc5d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd47fff640 1 -- 172.21.15.82:0/858917461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4802e070 con 0x7fbd500b0030 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd47fff640 1 -- 172.21.15.82:0/858917461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48037070 con 0x7fbd500b0030 2024-03-14T12:16:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd47fff640 1 -- 172.21.15.82:0/858917461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48031a00 con 0x7fbd500b0030 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 -- 172.21.15.82:0/858917461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd500b0030 msgr2=0x7fbd500d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 --2- 172.21.15.82:0/858917461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd500b0030 0x7fbd500d1340 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbd500b5580 tx=0x7fbd4802b880 comp rx=0 tx=0).stop 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 -- 172.21.15.82:0/858917461 shutdown_connections 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 --2- 172.21.15.82:0/858917461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd500b0030 0x7fbd500d1340 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 --2- 172.21.15.82:0/858917461 >> v2:172.21.15.164:3300/0 conn(0x7fbd50158290 0x7fbd500d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 -- 172.21.15.82:0/858917461 >> 172.21.15.82:0/858917461 conn(0x7fbd5006c000 msgr2=0x7fbd5006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 -- 172.21.15.82:0/858917461 shutdown_connections 2024-03-14T12:16:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.725+0000 7fbd5735b640 1 -- 172.21.15.82:0/858917461 wait complete. 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.726+0000 7fbd5735b640 1 Processor -- start 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.726+0000 7fbd5735b640 1 -- start start 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd5735b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd500b0030 0x7fbd501552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd5735b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd50158290 0x7fbd501e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd5735b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50158670 0x7fbd501e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd5735b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5006d890 con 0x7fbd50158670 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd5735b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5006d710 con 0x7fbd500b0030 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd5735b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500ae860 con 0x7fbd50158290 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50158670 0x7fbd501e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50158670 0x7fbd501e87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49752/0 (socket says 172.21.15.82:49752) 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 -- 172.21.15.82:0/2891875028 learned_addr learned my addr 172.21.15.82:0/2891875028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd550f7640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd500b0030 0x7fbd501552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd548f6640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd50158290 0x7fbd501e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 -- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd50158290 msgr2=0x7fbd501e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd50158290 0x7fbd501e82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 -- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd500b0030 msgr2=0x7fbd501552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd500b0030 0x7fbd501552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd558f8640 1 -- 172.21.15.82:0/2891875028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd501e9030 con 0x7fbd50158670 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.727+0000 7fbd550f7640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd500b0030 0x7fbd501552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.728+0000 7fbd558f8640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50158670 0x7fbd501e87f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbd480090b0 tx=0x7fbd480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.728+0000 7fbd45ffb640 1 -- 172.21.15.82:0/2891875028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48032880 con 0x7fbd50158670 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.728+0000 7fbd45ffb640 1 -- 172.21.15.82:0/2891875028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48032a40 con 0x7fbd50158670 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.728+0000 7fbd45ffb640 1 -- 172.21.15.82:0/2891875028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48032e30 con 0x7fbd50158670 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.728+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd501e9310 con 0x7fbd50158670 2024-03-14T12:16:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.728+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5015f560 con 0x7fbd50158670 2024-03-14T12:16:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.729+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5004f050 con 0x7fbd50158670 2024-03-14T12:16:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.729+0000 7fbd45ffb640 1 -- 172.21.15.82:0/2891875028 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbd480453f0 con 0x7fbd50158670 2024-03-14T12:16:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.730+0000 7fbd45ffb640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd3406bbd0 0x7fbd3406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.730+0000 7fbd45ffb640 1 -- 172.21.15.82:0/2891875028 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd480acba0 con 0x7fbd50158670 2024-03-14T12:16:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.730+0000 7fbd550f7640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd3406bbd0 0x7fbd3406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.730+0000 7fbd550f7640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd3406bbd0 0x7fbd3406e090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009fa0 tx=0x7fbd4c009570 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.734+0000 7fbd45ffb640 1 -- 172.21.15.82:0/2891875028 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbd48040070 con 0x7fbd50158670 2024-03-14T12:16:15.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.904+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd50157ab0 con 0x7fbd50158670 2024-03-14T12:16:15.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.905+0000 7fbd45ffb640 1 -- 172.21.15.82:0/2891875028 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd48031e90 con 0x7fbd50158670 2024-03-14T12:16:15.904 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:16:15.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd3406bbd0 msgr2=0x7fbd3406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd3406bbd0 0x7fbd3406e090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009fa0 tx=0x7fbd4c009570 comp rx=0 tx=0).stop 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50158670 msgr2=0x7fbd501e87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50158670 0x7fbd501e87f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbd480090b0 tx=0x7fbd480046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 shutdown_connections 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50158670 0x7fbd501e87f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd3406bbd0 0x7fbd3406e090 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd50158290 0x7fbd501e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 --2- 172.21.15.82:0/2891875028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd500b0030 0x7fbd501552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 >> 172.21.15.82:0/2891875028 conn(0x7fbd5006c000 msgr2=0x7fbd500d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 shutdown_connections 2024-03-14T12:16:15.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:15.907+0000 7fbd5735b640 1 -- 172.21.15.82:0/2891875028 wait complete. 2024-03-14T12:16:15.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:16:16.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.022+0000 7fe34a545640 1 Processor -- start 2024-03-14T12:16:16.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.022+0000 7fe34a545640 1 -- start start 2024-03-14T12:16:16.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe34a545640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3440b2510 0x7fe34415e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe34a545640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34406e8e0 con 0x7fe344147360 2024-03-14T12:16:16.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe34a545640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34406eb80 con 0x7fe3440d0db0 2024-03-14T12:16:16.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe34a545640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34406f100 con 0x7fe3440b2510 2024-03-14T12:16:16.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe348ae2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3440b2510 0x7fe34415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe3437fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe344147360 0x7fe3440b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44628/0 (socket says 172.21.15.82:44628) 2024-03-14T12:16:16.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe348ae2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3440b2510 0x7fe34415e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49766/0 (socket says 172.21.15.82:49766) 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe3437fe640 1 -- 172.21.15.82:0/3181696724 learned_addr learned my addr 172.21.15.82:0/3181696724 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe348ae2640 1 -- 172.21.15.82:0/3181696724 >> v1:172.21.15.164:6789/0 conn(0x7fe3440d0db0 legacy=0x7fe344162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe348ae2640 1 -- 172.21.15.82:0/3181696724 >> v1:172.21.15.82:6789/0 conn(0x7fe344147360 legacy=0x7fe3440b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe348ae2640 1 -- 172.21.15.82:0/3181696724 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34406ee20 con 0x7fe3440b2510 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.023+0000 7fe342ffd640 1 -- 172.21.15.82:0/3181696724 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334087241 0 0) 0x7fe34406e8e0 con 0x7fe344147360 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.024+0000 7fe348ae2640 1 --2- 172.21.15.82:0/3181696724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe34415e660 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe34406eab0 tx=0x7fe33c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2759413a0e6d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.024+0000 7fe342ffd640 1 -- 172.21.15.82:0/3181696724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c039070 con 0x7fe3440b2510 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.024+0000 7fe342ffd640 1 -- 172.21.15.82:0/3181696724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c02cbc0 con 0x7fe3440b2510 2024-03-14T12:16:16.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.024+0000 7fe342ffd640 1 -- 172.21.15.82:0/3181696724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c034440 con 0x7fe3440b2510 2024-03-14T12:16:16.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.024+0000 7fe34a545640 1 -- 172.21.15.82:0/3181696724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 msgr2=0x7fe34415e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.024+0000 7fe34a545640 1 --2- 172.21.15.82:0/3181696724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe34415e660 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe34406eab0 tx=0x7fe33c02be10 comp rx=0 tx=0).stop 2024-03-14T12:16:16.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.025+0000 7fe34a545640 1 -- 172.21.15.82:0/3181696724 shutdown_connections 2024-03-14T12:16:16.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.025+0000 7fe34a545640 1 --2- 172.21.15.82:0/3181696724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe34415e660 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.025+0000 7fe34a545640 1 -- 172.21.15.82:0/3181696724 >> 172.21.15.82:0/3181696724 conn(0x7fe3440b4fd0 msgr2=0x7fe3440b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:16.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.025+0000 7fe34a545640 1 -- 172.21.15.82:0/3181696724 shutdown_connections 2024-03-14T12:16:16.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.025+0000 7fe34a545640 1 -- 172.21.15.82:0/3181696724 wait complete. 2024-03-14T12:16:16.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.026+0000 7fe34a545640 1 Processor -- start 2024-03-14T12:16:16.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.026+0000 7fe34a545640 1 -- start start 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe3441461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3440d0db0 0x7fe34414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe344147360 0x7fe3441e9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34406d330 con 0x7fe3440b2510 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe344162040 con 0x7fe344147360 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34406d4b0 con 0x7fe3440d0db0 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe3441461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe3441461f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49778/0 (socket says 172.21.15.82:49778) 2024-03-14T12:16:16.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 -- 172.21.15.82:0/472573389 learned_addr learned my addr 172.21.15.82:0/472573389 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe348ae2640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe344147360 0x7fe3441e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 -- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3440d0db0 msgr2=0x7fe34414da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3440d0db0 0x7fe34414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 -- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe344147360 msgr2=0x7fe3441e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe344147360 0x7fe3441e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 -- 172.21.15.82:0/472573389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34409c770 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe348ae2640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe344147360 0x7fe3441e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe343fff640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe3441461f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c430 tx=0x7fe33400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe340ff9640 1 -- 172.21.15.82:0/472573389 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334011790 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe340ff9640 1 -- 172.21.15.82:0/472573389 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334017070 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34409ca50 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe340ff9640 1 -- 172.21.15.82:0/472573389 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334011b70 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.027+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34409d010 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.028+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34404f050 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.029+0000 7fe340ff9640 1 -- 172.21.15.82:0/472573389 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe334011d10 con 0x7fe3440b2510 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.029+0000 7fe340ff9640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe33006bbd0 0x7fe33006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.029+0000 7fe340ff9640 1 -- 172.21.15.82:0/472573389 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe33408a900 con 0x7fe3440b2510 2024-03-14T12:16:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.029+0000 7fe3437fe640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe33006bbd0 0x7fe33006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.029+0000 7fe3437fe640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe33006bbd0 0x7fe33006e090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe338007fd0 tx=0x7fe338007960 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.033+0000 7fe340ff9640 1 -- 172.21.15.82:0/472573389 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe3340593f0 con 0x7fe3440b2510 2024-03-14T12:16:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.204+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe34406c460 con 0x7fe3440b2510 2024-03-14T12:16:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.205+0000 7fe340ff9640 1 -- 172.21.15.82:0/472573389 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe334058cf0 con 0x7fe3440b2510 2024-03-14T12:16:16.204 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:16:16.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe33006bbd0 msgr2=0x7fe33006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe33006bbd0 0x7fe33006e090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe338007fd0 tx=0x7fe338007960 comp rx=0 tx=0).stop 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 msgr2=0x7fe3441461f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe3441461f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c430 tx=0x7fe33400c8f0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 shutdown_connections 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe344147360 0x7fe3441e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe33006bbd0 0x7fe33006e090 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3440d0db0 0x7fe34414da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 --2- 172.21.15.82:0/472573389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3440b2510 0x7fe3441461f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 >> 172.21.15.82:0/472573389 conn(0x7fe3440b4fd0 msgr2=0x7fe344160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 shutdown_connections 2024-03-14T12:16:16.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.207+0000 7fe34a545640 1 -- 172.21.15.82:0/472573389 wait complete. 2024-03-14T12:16:16.217 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:16:16.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.319+0000 7f673e675640 1 Processor -- start 2024-03-14T12:16:16.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.319+0000 7f673e675640 1 -- start start 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.319+0000 7f673e675640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f67380b0f60 0x7f67380cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.319+0000 7f673e675640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67380afff0 0x7f67380d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.319+0000 7f673e675640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67380b1340 con 0x7f673806ad10 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.319+0000 7f673e675640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67380d0ee0 con 0x7f67380afff0 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.319+0000 7f673e675640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67380b4ee0 con 0x7f67380b0f60 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f67377fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67380afff0 0x7f67380d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f673cc12640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f673806ad10 0x7f673806b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44632/0 (socket says 172.21.15.82:44632) 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f673cc12640 1 -- 172.21.15.82:0/40677627 learned_addr learned my addr 172.21.15.82:0/40677627 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f6737fff640 1 --2- 172.21.15.82:0/40677627 >> v2:172.21.15.164:3300/0 conn(0x7f67380b0f60 0x7f67380cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f67377fe640 1 -- 172.21.15.82:0/40677627 >> v2:172.21.15.164:3300/0 conn(0x7f67380b0f60 msgr2=0x7f67380cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f67377fe640 1 --2- 172.21.15.82:0/40677627 >> v2:172.21.15.164:3300/0 conn(0x7f67380b0f60 0x7f67380cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f67377fe640 1 -- 172.21.15.82:0/40677627 >> v1:172.21.15.82:6789/0 conn(0x7f673806ad10 legacy=0x7f673806b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f67377fe640 1 -- 172.21.15.82:0/40677627 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380cd030 con 0x7f67380afff0 2024-03-14T12:16:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.320+0000 7f6736ffd640 1 -- 172.21.15.82:0/40677627 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469193118 0 0) 0x7f67380b1340 con 0x7f673806ad10 2024-03-14T12:16:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f67377fe640 1 --2- 172.21.15.82:0/40677627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67380afff0 0x7f67380d09a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6738071ef0 tx=0x7f672802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ecd0f02f6084b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f6736ffd640 1 -- 172.21.15.82:0/40677627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728035020 con 0x7f67380afff0 2024-03-14T12:16:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f6736ffd640 1 -- 172.21.15.82:0/40677627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728031830 con 0x7f67380afff0 2024-03-14T12:16:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f6736ffd640 1 -- 172.21.15.82:0/40677627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728031c00 con 0x7f67380afff0 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 -- 172.21.15.82:0/40677627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67380afff0 msgr2=0x7f67380d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 --2- 172.21.15.82:0/40677627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67380afff0 0x7f67380d09a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6738071ef0 tx=0x7f672802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 -- 172.21.15.82:0/40677627 shutdown_connections 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 --2- 172.21.15.82:0/40677627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67380afff0 0x7f67380d09a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 --2- 172.21.15.82:0/40677627 >> v2:172.21.15.164:3300/0 conn(0x7f67380b0f60 0x7f67380cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 -- 172.21.15.82:0/40677627 >> 172.21.15.82:0/40677627 conn(0x7f6738072310 msgr2=0x7f6738072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 -- 172.21.15.82:0/40677627 shutdown_connections 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.321+0000 7f673e675640 1 -- 172.21.15.82:0/40677627 wait complete. 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.322+0000 7f673e675640 1 Processor -- start 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f673e675640 1 -- start start 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f673e675640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673806ad10 0x7f6738155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f673e675640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67380afff0 0x7f67380c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f673e675640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67380b0f60 0x7f67380c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f673e675640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67380cd8d0 con 0x7f673806ad10 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f673e675640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67380cd5f0 con 0x7f67380afff0 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f673e675640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67380b0450 con 0x7f67380b0f60 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f6737fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673806ad10 0x7f6738155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f6737fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673806ad10 0x7f6738155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49800/0 (socket says 172.21.15.82:49800) 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.323+0000 7f6737fff640 1 -- 172.21.15.82:0/4078954410 learned_addr learned my addr 172.21.15.82:0/4078954410 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f67377fe640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67380afff0 0x7f67380c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6737fff640 1 -- 172.21.15.82:0/4078954410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67380b0f60 msgr2=0x7f67380c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6737fff640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67380b0f60 0x7f67380c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6737fff640 1 -- 172.21.15.82:0/4078954410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67380afff0 msgr2=0x7f67380c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6737fff640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67380afff0 0x7f67380c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6737fff640 1 -- 172.21.15.82:0/4078954410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380c6020 con 0x7f673806ad10 2024-03-14T12:16:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6737fff640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673806ad10 0x7f6738155120 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f672c00d620 tx=0x7f672c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6734ff9640 1 -- 172.21.15.82:0/4078954410 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00b620 con 0x7f673806ad10 2024-03-14T12:16:16.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6734ff9640 1 -- 172.21.15.82:0/4078954410 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c019070 con 0x7f673806ad10 2024-03-14T12:16:16.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67380c6300 con 0x7f673806ad10 2024-03-14T12:16:16.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f6734ff9640 1 -- 172.21.15.82:0/4078954410 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00ba00 con 0x7f673806ad10 2024-03-14T12:16:16.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.324+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67380c68c0 con 0x7f673806ad10 2024-03-14T12:16:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.325+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673804f050 con 0x7f673806ad10 2024-03-14T12:16:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.325+0000 7f6734ff9640 1 -- 172.21.15.82:0/4078954410 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f672c00bba0 con 0x7f673806ad10 2024-03-14T12:16:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.326+0000 7f6734ff9640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f672406bbd0 0x7f672406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.326+0000 7f67377fe640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f672406bbd0 0x7f672406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.326+0000 7f6734ff9640 1 -- 172.21.15.82:0/4078954410 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f672c08af20 con 0x7f673806ad10 2024-03-14T12:16:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.326+0000 7f67377fe640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f672406bbd0 0x7f672406e090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6728036000 tx=0x7f6728008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.330+0000 7f6734ff9640 1 -- 172.21.15.82:0/4078954410 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f672c059980 con 0x7f673806ad10 2024-03-14T12:16:16.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.514+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6738071200 con 0x7f673806ad10 2024-03-14T12:16:16.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.514+0000 7f6734ff9640 1 -- 172.21.15.82:0/4078954410 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f672c059280 con 0x7f673806ad10 2024-03-14T12:16:16.514 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f672406bbd0 msgr2=0x7f672406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f672406bbd0 0x7f672406e090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6728036000 tx=0x7f6728008030 comp rx=0 tx=0).stop 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673806ad10 msgr2=0x7f6738155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673806ad10 0x7f6738155120 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f672c00d620 tx=0x7f672c00dae0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 shutdown_connections 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f672406bbd0 0x7f672406e090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67380b0f60 0x7f67380c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67380afff0 0x7f67380c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 --2- 172.21.15.82:0/4078954410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673806ad10 0x7f6738155120 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 >> 172.21.15.82:0/4078954410 conn(0x7f6738072310 msgr2=0x7f673806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 shutdown_connections 2024-03-14T12:16:16.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.516+0000 7f673e675640 1 -- 172.21.15.82:0/4078954410 wait complete. 2024-03-14T12:16:16.526 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T12:16:16.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:16.527 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:16.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:16:16.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:16.557 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T12:16:16.612 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:16.612 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:16.631 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:16.631 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:16.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:16.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.791+0000 7fc530019640 1 Processor -- start 2024-03-14T12:16:16.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.791+0000 7fc530019640 1 -- start start 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc530019640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 0x7fc5280d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc530019640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280cf8c0 con 0x7fc5280d0760 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc530019640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280cfba0 con 0x7fc5280b7720 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc530019640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280b6c40 con 0x7fc5280cf370 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52ddb5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc5280d0760 0x7fc5280d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44638/0 (socket says 172.21.15.82:44638) 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52ddb5640 1 -- 172.21.15.82:0/1281612524 learned_addr learned my addr 172.21.15.82:0/1281612524 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52d5b4640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 0x7fc5280d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945560027 0 0) 0x7fc5280cf8c0 con 0x7fc5280d0760 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc514003520 con 0x7fc5280d0760 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963818813 0 0) 0x7fc514003520 con 0x7fc5280d0760 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 msgr2=0x7fc5280d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 0x7fc5280d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 >> v1:172.21.15.164:6789/0 conn(0x7fc5280b7720 legacy=0x7fc5280b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280b6f20 con 0x7fc5280d0760 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fc524001670 con 0x7fc5280d0760 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 >> v1:172.21.15.82:6789/0 conn(0x7fc5280d0760 legacy=0x7fc5280d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5140039e0 0x7fc514003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc514004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 0x7fc5140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280b6c40 con 0x7fc514004800 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51400bb30 con 0x7fc5140039e0 2024-03-14T12:16:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.792+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51400bd30 con 0x7fc5280cf370 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52d5b4640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc514004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52ddb5640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5140039e0 0x7fc514003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52e5b6640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 0x7fc5140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52d5b4640 1 -- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 msgr2=0x7fc5140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52d5b4640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cf370 0x7fc5140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52d5b4640 1 -- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5140039e0 msgr2=0x7fc514003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52d5b4640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5140039e0 0x7fc514003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.793+0000 7fc52d5b4640 1 -- 172.21.15.82:0/1281612524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc514003520 con 0x7fc514004800 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc52d5b4640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc514004c80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc518009a90 tx=0x7fc514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa176316427afaac server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518039070 con 0x7fc514004800 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51802c680 con 0x7fc514004800 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc52cdb3640 1 -- 172.21.15.82:0/1281612524 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518009690 con 0x7fc514004800 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc530019640 1 -- 172.21.15.82:0/1281612524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 msgr2=0x7fc514004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc530019640 1 --2- 172.21.15.82:0/1281612524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc514004c80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc518009a90 tx=0x7fc514005830 comp rx=0 tx=0).stop 2024-03-14T12:16:16.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc52e5b6640 1 -- 172.21.15.82:0/1281612524 reap_dead start 2024-03-14T12:16:16.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc530019640 1 -- 172.21.15.82:0/1281612524 shutdown_connections 2024-03-14T12:16:16.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc530019640 1 -- 172.21.15.82:0/1281612524 >> 172.21.15.82:0/1281612524 conn(0x7fc52809d030 msgr2=0x7fc5280d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:16.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc530019640 1 -- 172.21.15.82:0/1281612524 shutdown_connections 2024-03-14T12:16:16.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.794+0000 7fc530019640 1 -- 172.21.15.82:0/1281612524 wait complete. 2024-03-14T12:16:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.795+0000 7fc530019640 1 Processor -- start 2024-03-14T12:16:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.795+0000 7fc530019640 1 -- start start 2024-03-14T12:16:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc530019640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc5280cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc530019640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cafb0 0x7fc5280cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc530019640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52815ecb0 0x7fc52815f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc530019640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528161920 con 0x7fc514004800 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc530019640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281617a0 con 0x7fc52815ecb0 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc530019640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280d4660 con 0x7fc5280cafb0 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc52ddb5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc5280cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc52ddb5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc5280cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49812/0 (socket says 172.21.15.82:49812) 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc52ddb5640 1 -- 172.21.15.82:0/4119610948 learned_addr learned my addr 172.21.15.82:0/4119610948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc52ddb5640 1 -- 172.21.15.82:0/4119610948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cafb0 msgr2=0x7fc5280cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc52e5b6640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52815ecb0 0x7fc52815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.796+0000 7fc52ddb5640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cafb0 0x7fc5280cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc52ddb5640 1 -- 172.21.15.82:0/4119610948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52815ecb0 msgr2=0x7fc52815f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc52ddb5640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52815ecb0 0x7fc52815f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc52ddb5640 1 -- 172.21.15.82:0/4119610948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280ce150 con 0x7fc514004800 2024-03-14T12:16:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc52ddb5640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc5280cdc10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc524001b80 tx=0x7fc524007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc51e7fc640 1 -- 172.21.15.82:0/4119610948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400f960 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc51e7fc640 1 -- 172.21.15.82:0/4119610948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524018070 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52815f680 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc51e7fc640 1 -- 172.21.15.82:0/4119610948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400fd20 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.797+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52815fc30 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.798+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52804f050 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.798+0000 7fc51e7fc640 1 -- 172.21.15.82:0/4119610948 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc52401d460 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.799+0000 7fc51e7fc640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50c06bbd0 0x7fc50c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.799+0000 7fc51e7fc640 1 -- 172.21.15.82:0/4119610948 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc52408aba0 con 0x7fc514004800 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.799+0000 7fc52d5b4640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50c06bbd0 0x7fc50c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.799+0000 7fc52d5b4640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50c06bbd0 0x7fc50c06e090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc518036000 tx=0x7fc51802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:16.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.803+0000 7fc51e7fc640 1 -- 172.21.15.82:0/4119610948 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc5280ce150 con 0x7fc514004800 2024-03-14T12:16:16.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.974+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5280d0540 con 0x7fc514004800 2024-03-14T12:16:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.975+0000 7fc51e7fc640 1 -- 172.21.15.82:0/4119610948 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fc524059690 con 0x7fc514004800 2024-03-14T12:16:16.975 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:16.975 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:03.671135+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:16.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50c06bbd0 msgr2=0x7fc50c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50c06bbd0 0x7fc50c06e090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc518036000 tx=0x7fc51802e000 comp rx=0 tx=0).stop 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 msgr2=0x7fc5280cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc5280cdc10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc524001b80 tx=0x7fc524007720 comp rx=0 tx=0).stop 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 shutdown_connections 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc52815ecb0 0x7fc52815f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50c06bbd0 0x7fc50c06e090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5280cafb0 0x7fc5280cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 --2- 172.21.15.82:0/4119610948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc514004800 0x7fc5280cdc10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 >> 172.21.15.82:0/4119610948 conn(0x7fc52809d030 msgr2=0x7fc5280b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 shutdown_connections 2024-03-14T12:16:16.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:16.977+0000 7fc530019640 1 -- 172.21.15.82:0/4119610948 wait complete. 2024-03-14T12:16:16.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:00.232652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:00.983002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:16.988 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:17.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.092+0000 7fed32d9f640 1 Processor -- start 2024-03-14T12:16:17.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.092+0000 7fed32d9f640 1 -- start start 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed32d9f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c161430 0x7fed2c0d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed32d9f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed2c0b2990 0x7fed2c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed32d9f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c0d24b0 con 0x7fed2c0d0590 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed32d9f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c0d2790 con 0x7fed2c0b2990 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed32d9f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0d2a70 con 0x7fed2c161430 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3159c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed2c0b2990 0x7fed2c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3159c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed2c0b2990 0x7fed2c0b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49824/0 (socket says 172.21.15.82:49824) 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3159c640 1 -- 172.21.15.82:0/1627656220 learned_addr learned my addr 172.21.15.82:0/1627656220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3259e640 1 --1- 172.21.15.82:0/1627656220 >> v1:172.21.15.82:6789/0 conn(0x7fed2c0d0590 0x7fed2c0cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44648/0 (socket says 172.21.15.82:44648) 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed31d9d640 1 --2- 172.21.15.82:0/1627656220 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c161430 0x7fed2c0d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3159c640 1 -- 172.21.15.82:0/1627656220 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c161430 msgr2=0x7fed2c0d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3159c640 1 --2- 172.21.15.82:0/1627656220 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c161430 0x7fed2c0d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3159c640 1 -- 172.21.15.82:0/1627656220 >> v1:172.21.15.82:6789/0 conn(0x7fed2c0d0590 legacy=0x7fed2c0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed30d9b640 1 -- 172.21.15.82:0/1627656220 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75701177 0 0) 0x7fed2c0d24b0 con 0x7fed2c0d0590 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed3159c640 1 -- 172.21.15.82:0/1627656220 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0ceac0 con 0x7fed2c0b2990 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.093+0000 7fed31d9d640 1 --2- 172.21.15.82:0/1627656220 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c161430 0x7fed2c0d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.094+0000 7fed3159c640 1 --2- 172.21.15.82:0/1627656220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c0b34c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009870 tx=0x7fed1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4d222f92cda0a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.094+0000 7fed30d9b640 1 -- 172.21.15.82:0/1627656220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c035020 con 0x7fed2c0b2990 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.094+0000 7fed30d9b640 1 -- 172.21.15.82:0/1627656220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c031640 con 0x7fed2c0b2990 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.094+0000 7fed30d9b640 1 -- 172.21.15.82:0/1627656220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c02d040 con 0x7fed2c0b2990 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.094+0000 7fed32d9f640 1 -- 172.21.15.82:0/1627656220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 msgr2=0x7fed2c0b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.094+0000 7fed32d9f640 1 --2- 172.21.15.82:0/1627656220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c0b34c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009870 tx=0x7fed1c02b8f0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.095+0000 7fed32d9f640 1 -- 172.21.15.82:0/1627656220 shutdown_connections 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.095+0000 7fed32d9f640 1 --2- 172.21.15.82:0/1627656220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c0b34c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.095+0000 7fed32d9f640 1 --2- 172.21.15.82:0/1627656220 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c161430 0x7fed2c0d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.095+0000 7fed32d9f640 1 -- 172.21.15.82:0/1627656220 >> 172.21.15.82:0/1627656220 conn(0x7fed2c09cf10 msgr2=0x7fed2c0d3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.095+0000 7fed32d9f640 1 -- 172.21.15.82:0/1627656220 shutdown_connections 2024-03-14T12:16:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.095+0000 7fed32d9f640 1 -- 172.21.15.82:0/1627656220 wait complete. 2024-03-14T12:16:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 Processor -- start 2024-03-14T12:16:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 -- start start 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c0d0590 0x7fed2c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed2c161430 0x7fed2c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c161810 con 0x7fed2c0b2990 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0b5400 con 0x7fed2c161430 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.096+0000 7fed32d9f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c161990 con 0x7fed2c0d0590 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c1f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49832/0 (socket says 172.21.15.82:49832) 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 -- 172.21.15.82:0/4135397073 learned_addr learned my addr 172.21.15.82:0/4135397073 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:17.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 -- 172.21.15.82:0/4135397073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c0d0590 msgr2=0x7fed2c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c0d0590 0x7fed2c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 -- 172.21.15.82:0/4135397073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed2c161430 msgr2=0x7fed2c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed2c161430 0x7fed2c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 -- 172.21.15.82:0/4135397073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0cd510 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed31d9d640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c1f7e70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c7f0 tx=0x7fed2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed227fc640 1 -- 172.21.15.82:0/4135397073 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28015070 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed227fc640 1 -- 172.21.15.82:0/4135397073 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28007820 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed227fc640 1 -- 172.21.15.82:0/4135397073 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28007bd0 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c0cd7d0 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.097+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c0cb700 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.098+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2c0cb510 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.099+0000 7fed227fc640 1 -- 172.21.15.82:0/4135397073 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fed28018070 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.099+0000 7fed227fc640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed1006bc00 0x7fed1006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.099+0000 7fed227fc640 1 -- 172.21.15.82:0/4135397073 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed2808aad0 con 0x7fed2c0b2990 2024-03-14T12:16:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.099+0000 7fed3159c640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed1006bc00 0x7fed1006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.099+0000 7fed3159c640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed1006bc00 0x7fed1006e0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed1c036000 tx=0x7fed1c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.103+0000 7fed227fc640 1 -- 172.21.15.82:0/4135397073 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fed28059650 con 0x7fed2c0b2990 2024-03-14T12:16:17.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.272+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed2c075210 con 0x7fed2c0b2990 2024-03-14T12:16:17.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.273+0000 7fed227fc640 1 -- 172.21.15.82:0/4135397073 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed28058f50 con 0x7fed2c0b2990 2024-03-14T12:16:17.272 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:17.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed1006bc00 msgr2=0x7fed1006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed1006bc00 0x7fed1006e0c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed1c036000 tx=0x7fed1c030000 comp rx=0 tx=0).stop 2024-03-14T12:16:17.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 msgr2=0x7fed2c1f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c1f7e70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c7f0 tx=0x7fed2800ccb0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 shutdown_connections 2024-03-14T12:16:17.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed2c161430 0x7fed2c0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fed1006bc00 0x7fed1006e0c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed2c0d0590 0x7fed2c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 --2- 172.21.15.82:0/4135397073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c0b2990 0x7fed2c1f7e70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 >> 172.21.15.82:0/4135397073 conn(0x7fed2c09cf10 msgr2=0x7fed2c06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:17.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 shutdown_connections 2024-03-14T12:16:17.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.275+0000 7fed32d9f640 1 -- 172.21.15.82:0/4135397073 wait complete. 2024-03-14T12:16:17.285 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:16:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.389+0000 7f6d317a2640 1 Processor -- start 2024-03-14T12:16:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.389+0000 7f6d317a2640 1 -- start start 2024-03-14T12:16:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d317a2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6d2c0d1e80 0x7f6d2c0b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d317a2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0af030 0x7f6d2c0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d317a2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2c0b0990 con 0x7f6d2c15bd00 2024-03-14T12:16:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d317a2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2c0af4c0 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d317a2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0b0c70 con 0x7f6d2c0af030 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6d2c15bd00 0x7f6d2c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44654/0 (socket says 172.21.15.82:44654) 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2bfff640 1 -- 172.21.15.82:0/1108979623 learned_addr learned my addr 172.21.15.82:0/1108979623 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2b7fe640 1 --2- 172.21.15.82:0/1108979623 >> v2:172.21.15.164:3300/0 conn(0x7f6d2c0d1e80 0x7f6d2c0b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d30fa1640 1 --2- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0af030 0x7f6d2c0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2affd640 1 -- 172.21.15.82:0/1108979623 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032593437 0 0) 0x7f6d2c0b0990 con 0x7f6d2c15bd00 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2affd640 1 -- 172.21.15.82:0/1108979623 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d14003520 con 0x7f6d2c15bd00 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2affd640 1 -- 172.21.15.82:0/1108979623 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 689840841 0 0) 0x7f6d14003520 con 0x7f6d2c15bd00 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2b7fe640 1 -- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0af030 msgr2=0x7f6d2c0b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.390+0000 7f6d2b7fe640 1 --2- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0af030 0x7f6d2c0b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.391+0000 7f6d2b7fe640 1 -- 172.21.15.82:0/1108979623 >> v1:172.21.15.82:6789/0 conn(0x7f6d2c15bd00 legacy=0x7f6d2c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.391+0000 7f6d2b7fe640 1 -- 172.21.15.82:0/1108979623 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c0b0f50 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.391+0000 7f6d30fa1640 1 --2- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0af030 0x7f6d2c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.391+0000 7f6d2b7fe640 1 --2- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c0b0050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097a0 tx=0x7f6d2002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb7522df20d0208 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d2affd640 1 -- 172.21.15.82:0/1108979623 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20037070 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d2affd640 1 -- 172.21.15.82:0/1108979623 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d20030510 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d2affd640 1 -- 172.21.15.82:0/1108979623 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c001670 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 -- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0d1e80 msgr2=0x7f6d2c0b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 --2- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c0b0050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097a0 tx=0x7f6d2002b790 comp rx=0 tx=0).stop 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 -- 172.21.15.82:0/1108979623 shutdown_connections 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 --2- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0af030 0x7f6d2c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 --2- 172.21.15.82:0/1108979623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c0b0050 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 -- 172.21.15.82:0/1108979623 >> 172.21.15.82:0/1108979623 conn(0x7f6d2c072d10 msgr2=0x7f6d2c09a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 -- 172.21.15.82:0/1108979623 shutdown_connections 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.392+0000 7f6d317a2640 1 -- 172.21.15.82:0/1108979623 wait complete. 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.393+0000 7f6d317a2640 1 Processor -- start 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d317a2640 1 -- start start 2024-03-14T12:16:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d317a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0af030 0x7f6d2c147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d317a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d317a2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c15bd00 0x7f6d2c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d317a2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c15c0e0 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d317a2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0b1510 con 0x7f6d2c0af030 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d317a2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c15c260 con 0x7f6d2c15bd00 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d2b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d2b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c1515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49834/0 (socket says 172.21.15.82:49834) 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.394+0000 7f6d2b7fe640 1 -- 172.21.15.82:0/3364279829 learned_addr learned my addr 172.21.15.82:0/3364279829 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2b7fe640 1 -- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c15bd00 msgr2=0x7f6d2c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2b7fe640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c15bd00 0x7f6d2c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2b7fe640 1 -- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0af030 msgr2=0x7f6d2c147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2bfff640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0af030 0x7f6d2c147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2b7fe640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0af030 0x7f6d2c147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2b7fe640 1 -- 172.21.15.82:0/3364279829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c156e20 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2bfff640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0af030 0x7f6d2c147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d2b7fe640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c1515e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d200096f0 tx=0x7f6d200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d28ff9640 1 -- 172.21.15.82:0/3364279829 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20041020 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d28ff9640 1 -- 172.21.15.82:0/3364279829 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2003c870 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d28ff9640 1 -- 172.21.15.82:0/3364279829 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20037040 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2c157100 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.395+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c150730 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.396+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2c1574f0 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.396+0000 7f6d28ff9640 1 -- 172.21.15.82:0/3364279829 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6d20044020 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.397+0000 7f6d28ff9640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6d1806bbd0 0x7f6d1806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.397+0000 7f6d28ff9640 1 -- 172.21.15.82:0/3364279829 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d200abfa0 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.397+0000 7f6d2bfff640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6d1806bbd0 0x7f6d1806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.397+0000 7f6d2bfff640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6d1806bbd0 0x7f6d1806e090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c001eb0 tx=0x7f6d1c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.401+0000 7f6d28ff9640 1 -- 172.21.15.82:0/3364279829 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6d2004b070 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.571+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d2c075210 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.572+0000 7f6d28ff9640 1 -- 172.21.15.82:0/3364279829 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d2007ab20 con 0x7f6d2c0d1e80 2024-03-14T12:16:17.571 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:16:17.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.573+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6d1806bbd0 msgr2=0x7f6d1806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.573+0000 7f6d317a2640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6d1806bbd0 0x7f6d1806e090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c001eb0 tx=0x7f6d1c009040 comp rx=0 tx=0).stop 2024-03-14T12:16:17.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.573+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d2c0d1e80 msgr2=0x7f6d2c1515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.573+0000 7f6d317a2640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c1515e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d200096f0 tx=0x7f6d200046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.573+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 shutdown_connections 2024-03-14T12:16:17.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.573+0000 7f6d317a2640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6d1806bbd0 0x7f6d1806e090 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.573+0000 7f6d317a2640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c15bd00 0x7f6d2c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.574+0000 7f6d317a2640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d2c0d1e80 0x7f6d2c1515e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.574+0000 7f6d317a2640 1 --2- 172.21.15.82:0/3364279829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0af030 0x7f6d2c147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.574+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 >> 172.21.15.82:0/3364279829 conn(0x7f6d2c072d10 msgr2=0x7f6d2c15ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:17.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.574+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 shutdown_connections 2024-03-14T12:16:17.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.574+0000 7f6d317a2640 1 -- 172.21.15.82:0/3364279829 wait complete. 2024-03-14T12:16:17.584 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:16:17.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.689+0000 7ff58e28c640 1 Processor -- start 2024-03-14T12:16:17.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.689+0000 7ff58e28c640 1 -- start start 2024-03-14T12:16:17.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.689+0000 7ff58e28c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff58815dff0 0x7ff5880d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff58e28c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d12a0 0x7ff5880d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff58e28c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5880d25a0 con 0x7ff5880d0c40 2024-03-14T12:16:17.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff58e28c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5880d10a0 con 0x7ff58815dff0 2024-03-14T12:16:17.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff58e28c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880d2350 con 0x7ff5880d12a0 2024-03-14T12:16:17.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff587fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff58815dff0 0x7ff5880d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff587fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff58815dff0 0x7ff5880d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49846/0 (socket says 172.21.15.82:49846) 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff587fff640 1 -- 172.21.15.82:0/3976450985 learned_addr learned my addr 172.21.15.82:0/3976450985 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff5877fe640 1 --2- 172.21.15.82:0/3976450985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d12a0 0x7ff5880d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff587fff640 1 -- 172.21.15.82:0/3976450985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d12a0 msgr2=0x7ff5880d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff587fff640 1 --2- 172.21.15.82:0/3976450985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d12a0 0x7ff5880d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff587fff640 1 -- 172.21.15.82:0/3976450985 >> v1:172.21.15.164:6789/0 conn(0x7ff5880d0c40 legacy=0x7ff5880ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.690+0000 7ff587fff640 1 -- 172.21.15.82:0/3976450985 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880d2880 con 0x7ff58815dff0 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff587fff640 1 --2- 172.21.15.82:0/3976450985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5880d4480 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff578009870 tx=0x7ff57802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243bfeb42cae647f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff586ffd640 1 -- 172.21.15.82:0/3976450985 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578034070 con 0x7ff58815dff0 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff586ffd640 1 -- 172.21.15.82:0/3976450985 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff578030840 con 0x7ff58815dff0 2024-03-14T12:16:17.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff586ffd640 1 -- 172.21.15.82:0/3976450985 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578030c10 con 0x7ff58815dff0 2024-03-14T12:16:17.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff58e28c640 1 -- 172.21.15.82:0/3976450985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 msgr2=0x7ff5880d4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff58e28c640 1 --2- 172.21.15.82:0/3976450985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5880d4480 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff578009870 tx=0x7ff57802b980 comp rx=0 tx=0).stop 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff58e28c640 1 -- 172.21.15.82:0/3976450985 shutdown_connections 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff58e28c640 1 --2- 172.21.15.82:0/3976450985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d12a0 0x7ff5880d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff58e28c640 1 --2- 172.21.15.82:0/3976450985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5880d4480 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.691+0000 7ff58e28c640 1 -- 172.21.15.82:0/3976450985 >> 172.21.15.82:0/3976450985 conn(0x7ff58809d0d0 msgr2=0x7ff5880d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.692+0000 7ff58e28c640 1 -- 172.21.15.82:0/3976450985 shutdown_connections 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.692+0000 7ff58e28c640 1 -- 172.21.15.82:0/3976450985 wait complete. 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.693+0000 7ff58e28c640 1 Processor -- start 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.693+0000 7ff58e28c640 1 -- start start 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.693+0000 7ff58e28c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d0c40 0x7ff5880b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.693+0000 7ff58e28c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5880d12a0 0x7ff5880cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58e28c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5881f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58e28c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880d4b40 con 0x7ff58815dff0 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58e28c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880d49c0 con 0x7ff5880d12a0 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58e28c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58815e3d0 con 0x7ff5880d0c40 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5881f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5881f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49858/0 (socket says 172.21.15.82:49858) 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 -- 172.21.15.82:0/863339322 learned_addr learned my addr 172.21.15.82:0/863339322 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff587fff640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d0c40 0x7ff5880b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff5877fe640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5880d12a0 0x7ff5880cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 -- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d0c40 msgr2=0x7ff5880b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d0c40 0x7ff5880b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 -- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5880d12a0 msgr2=0x7ff5880cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5880d12a0 0x7ff5880cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 -- 172.21.15.82:0/863339322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5881f4530 con 0x7ff58815dff0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff587fff640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d0c40 0x7ff5880b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58c829640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5881f3a40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c6b0 tx=0x7ff58000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff584ff9640 1 -- 172.21.15.82:0/863339322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580003b70 con 0x7ff58815dff0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff584ff9640 1 -- 172.21.15.82:0/863339322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580003d30 con 0x7ff58815dff0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5881f4810 con 0x7ff58815dff0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff584ff9640 1 -- 172.21.15.82:0/863339322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580005310 con 0x7ff58815dff0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.694+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5880b6e90 con 0x7ff58815dff0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.695+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58804f050 con 0x7ff58815dff0 2024-03-14T12:16:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.696+0000 7ff584ff9640 1 -- 172.21.15.82:0/863339322 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff5800054b0 con 0x7ff58815dff0 2024-03-14T12:16:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.696+0000 7ff584ff9640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff57406bbd0 0x7ff57406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.696+0000 7ff584ff9640 1 -- 172.21.15.82:0/863339322 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff58008a860 con 0x7ff58815dff0 2024-03-14T12:16:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.696+0000 7ff587fff640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff57406bbd0 0x7ff57406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.696+0000 7ff587fff640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff57406bbd0 0x7ff57406e090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff578036000 tx=0x7ff57803b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.700+0000 7ff584ff9640 1 -- 172.21.15.82:0/863339322 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff5800593e0 con 0x7ff58815dff0 2024-03-14T12:16:17.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.874+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5880d0280 con 0x7ff58815dff0 2024-03-14T12:16:17.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.874+0000 7ff584ff9640 1 -- 172.21.15.82:0/863339322 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff58008a0a0 con 0x7ff58815dff0 2024-03-14T12:16:17.874 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff57406bbd0 msgr2=0x7ff57406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff57406bbd0 0x7ff57406e090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff578036000 tx=0x7ff57803b040 comp rx=0 tx=0).stop 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 msgr2=0x7ff5881f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5881f3a40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c6b0 tx=0x7ff58000cb70 comp rx=0 tx=0).stop 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 shutdown_connections 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff58815dff0 0x7ff5881f3a40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5880d12a0 0x7ff5880cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff57406bbd0 0x7ff57406e090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 --2- 172.21.15.82:0/863339322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5880d0c40 0x7ff5880b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.876+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 >> 172.21.15.82:0/863339322 conn(0x7ff58809d0d0 msgr2=0x7ff58815d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:17.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.877+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 shutdown_connections 2024-03-14T12:16:17.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.877+0000 7ff58e28c640 1 -- 172.21.15.82:0/863339322 wait complete. 2024-03-14T12:16:17.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:16:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.988+0000 7fbcf7509640 1 Processor -- start 2024-03-14T12:16:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.989+0000 7fbcf7509640 1 -- start start 2024-03-14T12:16:17.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.989+0000 7fbcf7509640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbcf015b9e0 0x7fbcf00b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf7509640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcf00cd0d0 0x7fbcf015f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf7509640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf015c250 con 0x7fbcf015bdc0 2024-03-14T12:16:17.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf7509640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf00cf730 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf7509640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf00cd810 con 0x7fbcf015b9e0 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf5aa6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcf00cd0d0 0x7fbcf015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf5aa6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcf00cd0d0 0x7fbcf015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49868/0 (socket says 172.21.15.82:49868) 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf5aa6640 1 -- 172.21.15.82:0/549349595 learned_addr learned my addr 172.21.15.82:0/549349595 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf52a5640 1 --2- 172.21.15.82:0/549349595 >> v2:172.21.15.164:3300/0 conn(0x7fbcf015b9e0 0x7fbcf00b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf5aa6640 1 -- 172.21.15.82:0/549349595 >> v2:172.21.15.164:3300/0 conn(0x7fbcf015b9e0 msgr2=0x7fbcf00b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf5aa6640 1 --2- 172.21.15.82:0/549349595 >> v2:172.21.15.164:3300/0 conn(0x7fbcf015b9e0 0x7fbcf00b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf5aa6640 1 -- 172.21.15.82:0/549349595 >> v1:172.21.15.164:6789/0 conn(0x7fbcf015bdc0 legacy=0x7fbcf00d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.990+0000 7fbcf5aa6640 1 -- 172.21.15.82:0/549349595 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00cf980 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.991+0000 7fbcf5aa6640 1 --2- 172.21.15.82:0/549349595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf015f940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0071b50 tx=0x7fbce802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561f69a2f825db0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.991+0000 7fbce7fff640 1 -- 172.21.15.82:0/549349595 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce802ca80 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.991+0000 7fbce7fff640 1 -- 172.21.15.82:0/549349595 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce802cc20 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.991+0000 7fbce7fff640 1 -- 172.21.15.82:0/549349595 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80343f0 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.991+0000 7fbcf7509640 1 -- 172.21.15.82:0/549349595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 msgr2=0x7fbcf015f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.991+0000 7fbcf7509640 1 --2- 172.21.15.82:0/549349595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf015f940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0071b50 tx=0x7fbce802bee0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.992+0000 7fbcf7509640 1 -- 172.21.15.82:0/549349595 shutdown_connections 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.992+0000 7fbcf7509640 1 --2- 172.21.15.82:0/549349595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf015f940 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.992+0000 7fbcf7509640 1 --2- 172.21.15.82:0/549349595 >> v2:172.21.15.164:3300/0 conn(0x7fbcf015b9e0 0x7fbcf00b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.992+0000 7fbcf7509640 1 -- 172.21.15.82:0/549349595 >> 172.21.15.82:0/549349595 conn(0x7fbcf00b3670 msgr2=0x7fbcf00b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.992+0000 7fbcf7509640 1 -- 172.21.15.82:0/549349595 shutdown_connections 2024-03-14T12:16:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.992+0000 7fbcf7509640 1 -- 172.21.15.82:0/549349595 wait complete. 2024-03-14T12:16:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 Processor -- start 2024-03-14T12:16:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 -- start start 2024-03-14T12:16:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf00c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcf015b9e0 0x7fbcf006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcf015bdc0 0x7fbcf0158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00cd520 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00cff40 con 0x7fbcf015b9e0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.993+0000 7fbcf7509640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf015fe80 con 0x7fbcf015bdc0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf00c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf00c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49880/0 (socket says 172.21.15.82:49880) 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 -- 172.21.15.82:0/464877331 learned_addr learned my addr 172.21.15.82:0/464877331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf5aa6640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcf015bdc0 0x7fbcf0158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf4aa4640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcf015b9e0 0x7fbcf006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 -- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcf015bdc0 msgr2=0x7fbcf0158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcf015bdc0 0x7fbcf0158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 -- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcf015b9e0 msgr2=0x7fbcf006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcf015b9e0 0x7fbcf006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 -- 172.21.15.82:0/464877331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf006c900 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf52a5640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf00c4630 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e9b0 tx=0x7fbcec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbce5ffb640 1 -- 172.21.15.82:0/464877331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec017070 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbce5ffb640 1 -- 172.21.15.82:0/464877331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec00c9a0 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbce5ffb640 1 -- 172.21.15.82:0/464877331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00cd90 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf0157c80 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.994+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf0157f60 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.995+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf006d8e0 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.996+0000 7fbce5ffb640 1 -- 172.21.15.82:0/464877331 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbcec01a070 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.996+0000 7fbce5ffb640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbcd406bbd0 0x7fbcd406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.996+0000 7fbce5ffb640 1 -- 172.21.15.82:0/464877331 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcec08aae0 con 0x7fbcf00cd0d0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.996+0000 7fbcf4aa4640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbcd406bbd0 0x7fbcd406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:17.997+0000 7fbcf4aa4640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbcd406bbd0 0x7fbcd406e090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbcf006c830 tx=0x7fbce0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:18.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.001+0000 7fbce5ffb640 1 -- 172.21.15.82:0/464877331 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbcec0595d0 con 0x7fbcf00cd0d0 2024-03-14T12:16:18.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.183+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbcf004f050 con 0x7fbcf00cd0d0 2024-03-14T12:16:18.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.183+0000 7fbce5ffb640 1 -- 172.21.15.82:0/464877331 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbcec058ed0 con 0x7fbcf00cd0d0 2024-03-14T12:16:18.183 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbcd406bbd0 msgr2=0x7fbcd406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbcd406bbd0 0x7fbcd406e090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbcf006c830 tx=0x7fbce0009040 comp rx=0 tx=0).stop 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 msgr2=0x7fbcf00c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf00c4630 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e9b0 tx=0x7fbcec00ee70 comp rx=0 tx=0).stop 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 shutdown_connections 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbcd406bbd0 0x7fbcd406e090 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcf015bdc0 0x7fbcf0158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcf015b9e0 0x7fbcf006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 --2- 172.21.15.82:0/464877331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf00cd0d0 0x7fbcf00c4630 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 >> 172.21.15.82:0/464877331 conn(0x7fbcf00b3670 msgr2=0x7fbcf00b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:18.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 shutdown_connections 2024-03-14T12:16:18.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.185+0000 7fbcf7509640 1 -- 172.21.15.82:0/464877331 wait complete. 2024-03-14T12:16:18.195 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T12:16:18.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:18.195 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:18.230 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:16:18.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:18.231 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T12:16:18.289 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:18.289 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:18.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:16:18.305 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:18.305 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:18.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:16:18.347 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:18.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.448+0000 7f93c5d0a640 1 Processor -- start 2024-03-14T12:16:18.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.448+0000 7f93c5d0a640 1 -- start start 2024-03-14T12:16:18.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93c5d0a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c015e670 0x7f93c00b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93c5d0a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f93c0143f60 0x7f93c015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93c5d0a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c015ebe0 con 0x7f93c0071740 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93c5d0a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c015eec0 con 0x7f93c0143f60 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93c5d0a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c015f1a0 con 0x7f93c015e670 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93bffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f93c0143f60 0x7f93c015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93bf7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f93c0071740 0x7f93c0071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44670/0 (socket says 172.21.15.82:44670) 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93bf7fe640 1 -- 172.21.15.82:0/1844029007 learned_addr learned my addr 172.21.15.82:0/1844029007 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93bffff640 1 -- 172.21.15.82:0/1844029007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c015e670 msgr2=0x7f93c00b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93bffff640 1 --2- 172.21.15.82:0/1844029007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c015e670 0x7f93c00b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93bffff640 1 -- 172.21.15.82:0/1844029007 >> v1:172.21.15.82:6789/0 conn(0x7f93c0071740 legacy=0x7f93c0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93bffff640 1 -- 172.21.15.82:0/1844029007 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c015f480 con 0x7f93c0143f60 2024-03-14T12:16:18.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.449+0000 7f93be7fc640 1 -- 172.21.15.82:0/1844029007 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524582799 0 0) 0x7f93c015ebe0 con 0x7f93c0071740 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.450+0000 7f93bffff640 1 --2- 172.21.15.82:0/1844029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0143f60 0x7f93c015b610 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009d90 tx=0x7f93b802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99438b2cddc0aac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.450+0000 7f93be7fc640 1 -- 172.21.15.82:0/1844029007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8039070 con 0x7f93c0143f60 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.450+0000 7f93be7fc640 1 -- 172.21.15.82:0/1844029007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b802cc50 con 0x7f93c0143f60 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.450+0000 7f93be7fc640 1 -- 172.21.15.82:0/1844029007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8034440 con 0x7f93c0143f60 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.450+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1844029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0143f60 msgr2=0x7f93c015b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.451+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1844029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0143f60 0x7f93c015b610 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009d90 tx=0x7f93b802bea0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.451+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1844029007 shutdown_connections 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.451+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1844029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0143f60 0x7f93c015b610 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.451+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1844029007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c015e670 0x7f93c00b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.451+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1844029007 >> 172.21.15.82:0/1844029007 conn(0x7f93c00b1c70 msgr2=0x7f93c00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:18.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.451+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1844029007 shutdown_connections 2024-03-14T12:16:18.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.451+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1844029007 wait complete. 2024-03-14T12:16:18.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.452+0000 7f93c5d0a640 1 Processor -- start 2024-03-14T12:16:18.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.452+0000 7f93c5d0a640 1 -- start start 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0071740 0x7f93c013c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c0143f60 0x7f93c00b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c015e670 0x7f93c00b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0144670 con 0x7f93c0071740 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c015b080 con 0x7f93c015e670 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01447f0 con 0x7f93c0143f60 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0071740 0x7f93c013c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0071740 0x7f93c013c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49902/0 (socket says 172.21.15.82:49902) 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 -- 172.21.15.82:0/1274279518 learned_addr learned my addr 172.21.15.82:0/1274279518 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 -- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c0143f60 msgr2=0x7f93c00b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93beffd640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c0143f60 0x7f93c00b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bffff640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c015e670 0x7f93c00b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c0143f60 0x7f93c00b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 -- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c015e670 msgr2=0x7f93c00b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c015e670 0x7f93c00b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 -- 172.21.15.82:0/1274279518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c013c940 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93bf7fe640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0071740 0x7f93c013c400 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c460 tx=0x7f93b000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93a7fff640 1 -- 172.21.15.82:0/1274279518 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0018070 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93a7fff640 1 -- 172.21.15.82:0/1274279518 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0007500 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c00af320 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93a7fff640 1 -- 172.21.15.82:0/1274279518 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004a20 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.453+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c00aea80 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.454+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c004f050 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.455+0000 7f93a7fff640 1 -- 172.21.15.82:0/1274279518 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f93b001b070 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.455+0000 7f93a7fff640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93ac06bbd0 0x7f93ac06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.455+0000 7f93a7fff640 1 -- 172.21.15.82:0/1274279518 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b0089cb0 con 0x7f93c0071740 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.455+0000 7f93beffd640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93ac06bbd0 0x7f93ac06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.456+0000 7f93beffd640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93ac06bbd0 0x7f93ac06e090 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f93c015a5b0 tx=0x7f93b4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.460+0000 7f93a7fff640 1 -- 172.21.15.82:0/1274279518 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f93b0058830 con 0x7f93c0071740 2024-03-14T12:16:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.669+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93c00d07b0 con 0x7f93c0071740 2024-03-14T12:16:18.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.670+0000 7f93a7fff640 1 -- 172.21.15.82:0/1274279518 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f93b0058130 con 0x7f93c0071740 2024-03-14T12:16:18.670 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:18.670 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1414844632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":1414844632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1414844632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/456693586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":456693586},{"type":"v1","addr":"172.21.15.164:6834","nonce":456693586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/58260684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":58260684},{"type":"v1","addr":"172.21.15.82:6836","nonce":58260684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:02.017303+0000","modified":"2024-03-14T12:16:04.014283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2708572159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2708572159},{"type":"v1","addr":"172.21.15.82:6837","nonce":2708572159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T12:16:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 21 2024-03-14T12:16:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93ac06bbd0 msgr2=0x7f93ac06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93ac06bbd0 0x7f93ac06e090 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f93c015a5b0 tx=0x7f93b4008040 comp rx=0 tx=0).stop 2024-03-14T12:16:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0071740 msgr2=0x7f93c013c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0071740 0x7f93c013c400 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c460 tx=0x7f93b000c920 comp rx=0 tx=0).stop 2024-03-14T12:16:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 shutdown_connections 2024-03-14T12:16:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93c015e670 0x7f93c00b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f93ac06bbd0 0x7f93ac06e090 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93c0143f60 0x7f93c00b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 --2- 172.21.15.82:0/1274279518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0071740 0x7f93c013c400 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 >> 172.21.15.82:0/1274279518 conn(0x7f93c00b1c70 msgr2=0x7f93c00cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 shutdown_connections 2024-03-14T12:16:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.672+0000 7f93c5d0a640 1 -- 172.21.15.82:0/1274279518 wait complete. 2024-03-14T12:16:18.682 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:18.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.783+0000 7fa996627640 1 Processor -- start 2024-03-14T12:16:18.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.783+0000 7fa996627640 1 -- start start 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa996627640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa99015f7a0 0x7fa99015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa996627640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9900ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa996627640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa990144ac0 0x7fa990144f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa996627640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9900cf150 con 0x7fa99015f7a0 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa996627640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9901600c0 con 0x7fa990144ac0 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa996627640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900ce8b0 con 0x7fa9900cdd60 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa98ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa99015f7a0 0x7fa99015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa98ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa99015f7a0 0x7fa99015fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49916/0 (socket says 172.21.15.82:49916) 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.784+0000 7fa98ffff640 1 -- 172.21.15.82:0/1782635844 learned_addr learned my addr 172.21.15.82:0/1782635844 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa98f7fe640 1 --2- 172.21.15.82:0/1782635844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9900ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa994bc4640 1 --2- 172.21.15.82:0/1782635844 >> v2:172.21.15.164:3300/0 conn(0x7fa990144ac0 0x7fa990144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa98ffff640 1 -- 172.21.15.82:0/1782635844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 msgr2=0x7fa9900ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa98ffff640 1 --2- 172.21.15.82:0/1782635844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9900ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa98ffff640 1 -- 172.21.15.82:0/1782635844 >> v2:172.21.15.164:3300/0 conn(0x7fa990144ac0 msgr2=0x7fa990144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa98ffff640 1 --2- 172.21.15.82:0/1782635844 >> v2:172.21.15.164:3300/0 conn(0x7fa990144ac0 0x7fa990144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa98ffff640 1 -- 172.21.15.82:0/1782635844 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900ceb90 con 0x7fa99015f7a0 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.785+0000 7fa98f7fe640 1 --2- 172.21.15.82:0/1782635844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9900ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa98ffff640 1 --2- 172.21.15.82:0/1782635844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa99015fb80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa9900b22c0 tx=0x7fa98002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb4180d88608c62 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa98effd640 1 -- 172.21.15.82:0/1782635844 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98002e070 con 0x7fa99015f7a0 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa98effd640 1 -- 172.21.15.82:0/1782635844 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa980037070 con 0x7fa99015f7a0 2024-03-14T12:16:18.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa98effd640 1 -- 172.21.15.82:0/1782635844 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980031a00 con 0x7fa99015f7a0 2024-03-14T12:16:18.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 -- 172.21.15.82:0/1782635844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 msgr2=0x7fa99015fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 --2- 172.21.15.82:0/1782635844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa99015fb80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa9900b22c0 tx=0x7fa98002b880 comp rx=0 tx=0).stop 2024-03-14T12:16:18.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 -- 172.21.15.82:0/1782635844 shutdown_connections 2024-03-14T12:16:18.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 --2- 172.21.15.82:0/1782635844 >> v2:172.21.15.164:3300/0 conn(0x7fa990144ac0 0x7fa990144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 --2- 172.21.15.82:0/1782635844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9900ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 --2- 172.21.15.82:0/1782635844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa99015fb80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 -- 172.21.15.82:0/1782635844 >> 172.21.15.82:0/1782635844 conn(0x7fa9900b1c70 msgr2=0x7fa9900cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 -- 172.21.15.82:0/1782635844 shutdown_connections 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.786+0000 7fa996627640 1 -- 172.21.15.82:0/1782635844 wait complete. 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 Processor -- start 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 -- start start 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9901f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990144ac0 0x7fa990142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa9900c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900cc4f0 con 0x7fa99015f7a0 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99006c4e0 con 0x7fa990144ac0 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.788+0000 7fa996627640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900cc670 con 0x7fa9900cdd60 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa9900c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa9900c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49932/0 (socket says 172.21.15.82:49932) 2024-03-14T12:16:18.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 -- 172.21.15.82:0/4284700231 learned_addr learned my addr 172.21.15.82:0/4284700231 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa98f7fe640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990144ac0 0x7fa990142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa98ffff640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9901f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 -- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 msgr2=0x7fa9901f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9901f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 -- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990144ac0 msgr2=0x7fa990142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990144ac0 0x7fa990142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 -- 172.21.15.82:0/4284700231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9901f19e0 con 0x7fa99015f7a0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa98ffff640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9901f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa994bc4640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa9900c5b40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa988007e60 tx=0x7fa98800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa98cff9640 1 -- 172.21.15.82:0/4284700231 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988014420 con 0x7fa99015f7a0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa98cff9640 1 -- 172.21.15.82:0/4284700231 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988019070 con 0x7fa99015f7a0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9901f1ca0 con 0x7fa99015f7a0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa98cff9640 1 -- 172.21.15.82:0/4284700231 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988014820 con 0x7fa99015f7a0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.789+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9901f2540 con 0x7fa99015f7a0 2024-03-14T12:16:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.790+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99004f050 con 0x7fa99015f7a0 2024-03-14T12:16:18.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.790+0000 7fa98cff9640 1 -- 172.21.15.82:0/4284700231 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa9880149c0 con 0x7fa99015f7a0 2024-03-14T12:16:18.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.791+0000 7fa98cff9640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa97c06bbd0 0x7fa97c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:18.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.791+0000 7fa98cff9640 1 -- 172.21.15.82:0/4284700231 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa98808b240 con 0x7fa99015f7a0 2024-03-14T12:16:18.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.791+0000 7fa98ffff640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa97c06bbd0 0x7fa97c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:18.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.791+0000 7fa98ffff640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa97c06bbd0 0x7fa97c06e090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa980036000 tx=0x7fa98003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:18.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.795+0000 7fa98cff9640 1 -- 172.21.15.82:0/4284700231 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa988059d30 con 0x7fa99015f7a0 2024-03-14T12:16:18.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.966+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa990143f60 con 0x7fa99015f7a0 2024-03-14T12:16:18.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.967+0000 7fa98cff9640 1 -- 172.21.15.82:0/4284700231 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fa988059630 con 0x7fa99015f7a0 2024-03-14T12:16:18.967 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:18.967 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:03.671135+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa97c06bbd0 msgr2=0x7fa97c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa97c06bbd0 0x7fa97c06e090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa980036000 tx=0x7fa98003d040 comp rx=0 tx=0).stop 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 msgr2=0x7fa9900c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa9900c5b40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa988007e60 tx=0x7fa98800e4b0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 shutdown_connections 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99015f7a0 0x7fa9900c5b40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990144ac0 0x7fa990142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa97c06bbd0 0x7fa97c06e090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 --2- 172.21.15.82:0/4284700231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9900cdd60 0x7fa9901f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 >> 172.21.15.82:0/4284700231 conn(0x7fa9900b1c70 msgr2=0x7fa99006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 shutdown_connections 2024-03-14T12:16:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:18.969+0000 7fa996627640 1 -- 172.21.15.82:0/4284700231 wait complete. 2024-03-14T12:16:18.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:00.232652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:00.983002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:18.980 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.084+0000 7f25aada9640 1 Processor -- start 2024-03-14T12:16:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25aada9640 1 -- start start 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25aada9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a40d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25aada9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25a40d0e10 0x7f25a40cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25aada9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a40d25a0 con 0x7f25a40d12a0 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25aada9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a40d2350 con 0x7f25a40d0e10 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25aada9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40d2880 con 0x7f25a415dff0 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25a9346640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25a40d0e10 0x7f25a40cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25a3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f25a40d12a0 0x7f25a40d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44684/0 (socket says 172.21.15.82:44684) 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25a9346640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25a40d0e10 0x7f25a40cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49936/0 (socket says 172.21.15.82:49936) 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.085+0000 7f25a3fff640 1 -- 172.21.15.82:0/1471163150 learned_addr learned my addr 172.21.15.82:0/1471163150 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.086+0000 7f25a8b45640 1 --2- 172.21.15.82:0/1471163150 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a40d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.086+0000 7f25a9346640 1 -- 172.21.15.82:0/1471163150 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 msgr2=0x7f25a40d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.086+0000 7f25a9346640 1 --2- 172.21.15.82:0/1471163150 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a40d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.086+0000 7f25a9346640 1 -- 172.21.15.82:0/1471163150 >> v1:172.21.15.82:6789/0 conn(0x7f25a40d12a0 legacy=0x7f25a40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.086+0000 7f25a9346640 1 -- 172.21.15.82:0/1471163150 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40d2b60 con 0x7f25a40d0e10 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.086+0000 7f25a9346640 1 --2- 172.21.15.82:0/1471163150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40cb320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d90 tx=0x7f259c02bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5f0ed68e157f03 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25a37fe640 1 -- 172.21.15.82:0/1471163150 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c039070 con 0x7f25a40d0e10 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25a37fe640 1 -- 172.21.15.82:0/1471163150 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c02ca10 con 0x7f25a40d0e10 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25a37fe640 1 -- 172.21.15.82:0/1471163150 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c02cde0 con 0x7f25a40d0e10 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 -- 172.21.15.82:0/1471163150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 msgr2=0x7f25a40cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 --2- 172.21.15.82:0/1471163150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40cb320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d90 tx=0x7f259c02bce0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 -- 172.21.15.82:0/1471163150 shutdown_connections 2024-03-14T12:16:19.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 --2- 172.21.15.82:0/1471163150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40cb320 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 --2- 172.21.15.82:0/1471163150 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a40d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 -- 172.21.15.82:0/1471163150 >> 172.21.15.82:0/1471163150 conn(0x7f25a409d0d0 msgr2=0x7f25a40d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:19.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 -- 172.21.15.82:0/1471163150 shutdown_connections 2024-03-14T12:16:19.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.087+0000 7f25aada9640 1 -- 172.21.15.82:0/1471163150 wait complete. 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.088+0000 7f25aada9640 1 Processor -- start 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.088+0000 7f25aada9640 1 -- start start 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25aada9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25aada9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a40d12a0 0x7f25a40b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25aada9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a414e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25aada9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a415e550 con 0x7f25a40d0e10 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25aada9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a415e3d0 con 0x7f25a40d12a0 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25aada9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40cb860 con 0x7f25a415dff0 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25a8b45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25a8b45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49950/0 (socket says 172.21.15.82:49950) 2024-03-14T12:16:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.089+0000 7f25a8b45640 1 -- 172.21.15.82:0/1474366183 learned_addr learned my addr 172.21.15.82:0/1474366183 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a3fff640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a40d12a0 0x7f25a40b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a8b45640 1 -- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 msgr2=0x7f25a414e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a9346640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a414e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a8b45640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a414e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a8b45640 1 -- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a40d12a0 msgr2=0x7f25a40b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a8b45640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a40d12a0 0x7f25a40b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a8b45640 1 -- 172.21.15.82:0/1474366183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40b8290 con 0x7f25a40d0e10 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a3fff640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a40d12a0 0x7f25a40b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a8b45640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40c7880 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f259400d5f0 tx=0x7f259400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a17fa640 1 -- 172.21.15.82:0/1474366183 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25940125f0 con 0x7f25a40d0e10 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a17fa640 1 -- 172.21.15.82:0/1474366183 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594019070 con 0x7f25a40d0e10 2024-03-14T12:16:19.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25a17fa640 1 -- 172.21.15.82:0/1474366183 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25940129d0 con 0x7f25a40d0e10 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a40b8570 con 0x7f25a40d0e10 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.090+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a40b8b30 con 0x7f25a40d0e10 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.091+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a4160ea0 con 0x7f25a40d0e10 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.091+0000 7f25a17fa640 1 -- 172.21.15.82:0/1474366183 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2594012b70 con 0x7f25a40d0e10 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.092+0000 7f25a17fa640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f259006bbd0 0x7f259006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.092+0000 7f25a17fa640 1 -- 172.21.15.82:0/1474366183 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f259408ade0 con 0x7f25a40d0e10 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.092+0000 7f25a3fff640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f259006bbd0 0x7f259006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.092+0000 7f25a3fff640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f259006bbd0 0x7f259006e090 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f25a40d4600 tx=0x7f2598009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.096+0000 7f25a17fa640 1 -- 172.21.15.82:0/1474366183 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f25a4160ea0 con 0x7f25a40d0e10 2024-03-14T12:16:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.267+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25a404f050 con 0x7f25a40d0e10 2024-03-14T12:16:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.268+0000 7f25a17fa640 1 -- 172.21.15.82:0/1474366183 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2594059960 con 0x7f25a40d0e10 2024-03-14T12:16:19.267 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f259006bbd0 msgr2=0x7f259006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f259006bbd0 0x7f259006e090 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f25a40d4600 tx=0x7f2598009040 comp rx=0 tx=0).stop 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 msgr2=0x7f25a40c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40c7880 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f259400d5f0 tx=0x7f259400dab0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 shutdown_connections 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f259006bbd0 0x7f259006e090 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a415dff0 0x7f25a414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a40d12a0 0x7f25a40b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 --2- 172.21.15.82:0/1474366183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25a40d0e10 0x7f25a40c7880 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 >> 172.21.15.82:0/1474366183 conn(0x7f25a409d0d0 msgr2=0x7f25a406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 shutdown_connections 2024-03-14T12:16:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.270+0000 7f25aada9640 1 -- 172.21.15.82:0/1474366183 wait complete. 2024-03-14T12:16:19.280 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:16:19.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.384+0000 7f48a8ff1640 1 Processor -- start 2024-03-14T12:16:19.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.384+0000 7f48a8ff1640 1 -- start start 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a8ff1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48a41446f0 0x7f48a415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a8ff1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48a40d0210 0x7f48a415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a8ff1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a40b34e0 con 0x7f48a40cfe30 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a8ff1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a415f1f0 con 0x7f48a41446f0 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a8ff1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a415f4a0 con 0x7f48a40d0210 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48a41446f0 0x7f48a415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f48a40cfe30 0x7f48a4143fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44698/0 (socket says 172.21.15.82:44698) 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a259c640 1 -- 172.21.15.82:0/106708266 learned_addr learned my addr 172.21.15.82:0/106708266 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a2d9d640 1 --2- 172.21.15.82:0/106708266 >> v2:172.21.15.164:3300/0 conn(0x7f48a40d0210 0x7f48a415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a1d9b640 1 -- 172.21.15.82:0/106708266 >> v2:172.21.15.164:3300/0 conn(0x7f48a40d0210 msgr2=0x7f48a415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a1d9b640 1 --2- 172.21.15.82:0/106708266 >> v2:172.21.15.164:3300/0 conn(0x7f48a40d0210 0x7f48a415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.385+0000 7f48a1d9b640 1 -- 172.21.15.82:0/106708266 >> v1:172.21.15.82:6789/0 conn(0x7f48a40cfe30 legacy=0x7f48a4143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.386+0000 7f48a1d9b640 1 -- 172.21.15.82:0/106708266 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40b3780 con 0x7f48a41446f0 2024-03-14T12:16:19.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.386+0000 7f48a2d9d640 1 --2- 172.21.15.82:0/106708266 >> v2:172.21.15.164:3300/0 conn(0x7f48a40d0210 0x7f48a415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.386+0000 7f48a1d9b640 1 --2- 172.21.15.82:0/106708266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41446f0 0x7f48a415e770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f48a415b110 tx=0x7f489802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21035ee9c48b9128 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.386+0000 7f48a159a640 1 -- 172.21.15.82:0/106708266 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898035020 con 0x7f48a41446f0 2024-03-14T12:16:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.386+0000 7f48a159a640 1 -- 172.21.15.82:0/106708266 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898031830 con 0x7f48a41446f0 2024-03-14T12:16:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.386+0000 7f48a159a640 1 -- 172.21.15.82:0/106708266 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898031c00 con 0x7f48a41446f0 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 -- 172.21.15.82:0/106708266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41446f0 msgr2=0x7f48a415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/106708266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41446f0 0x7f48a415e770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f48a415b110 tx=0x7f489802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 -- 172.21.15.82:0/106708266 shutdown_connections 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/106708266 >> v2:172.21.15.164:3300/0 conn(0x7f48a40d0210 0x7f48a415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/106708266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a41446f0 0x7f48a415e770 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 -- 172.21.15.82:0/106708266 >> 172.21.15.82:0/106708266 conn(0x7f48a406aef0 msgr2=0x7f48a406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 -- 172.21.15.82:0/106708266 shutdown_connections 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.387+0000 7f48a8ff1640 1 -- 172.21.15.82:0/106708266 wait complete. 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.388+0000 7f48a8ff1640 1 Processor -- start 2024-03-14T12:16:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.388+0000 7f48a8ff1640 1 -- start start 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a8ff1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a40cfe30 0x7f48a4139f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a8ff1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a40d0210 0x7f48a413e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a8ff1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48a41446f0 0x7f48a4140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a8ff1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40b4260 con 0x7f48a40d0210 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a8ff1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40b3d40 con 0x7f48a40cfe30 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a8ff1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40b43e0 con 0x7f48a41446f0 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a40d0210 0x7f48a413e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a40d0210 0x7f48a413e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49972/0 (socket says 172.21.15.82:49972) 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 -- 172.21.15.82:0/4200903196 learned_addr learned my addr 172.21.15.82:0/4200903196 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a259c640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a40cfe30 0x7f48a4139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a2d9d640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48a41446f0 0x7f48a4140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 -- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48a41446f0 msgr2=0x7f48a4140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48a41446f0 0x7f48a4140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 -- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a40cfe30 msgr2=0x7f48a4139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a40cfe30 0x7f48a4139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a1d9b640 1 -- 172.21.15.82:0/4200903196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a413a7d0 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.389+0000 7f48a2d9d640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48a41446f0 0x7f48a4140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.390+0000 7f48a1d9b640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a40d0210 0x7f48a413e8e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4898000c00 tx=0x7f48980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.390+0000 7f4892ffd640 1 -- 172.21.15.82:0/4200903196 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898040070 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.390+0000 7f4892ffd640 1 -- 172.21.15.82:0/4200903196 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898039940 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.390+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a41f0f30 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.390+0000 7f4892ffd640 1 -- 172.21.15.82:0/4200903196 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898035040 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.390+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a41f17d0 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.391+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a404f050 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.391+0000 7f4892ffd640 1 -- 172.21.15.82:0/4200903196 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f489804b060 con 0x7f48a40d0210 2024-03-14T12:16:19.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.392+0000 7f4892ffd640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f488406bbd0 0x7f488406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.392+0000 7f4892ffd640 1 -- 172.21.15.82:0/4200903196 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48980ac3f0 con 0x7f48a40d0210 2024-03-14T12:16:19.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.392+0000 7f48a259c640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f488406bbd0 0x7f488406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.392+0000 7f48a259c640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f488406bbd0 0x7f488406e090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f48940012e0 tx=0x7f4894009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.396+0000 7f4892ffd640 1 -- 172.21.15.82:0/4200903196 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f489807af70 con 0x7f48a40d0210 2024-03-14T12:16:19.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.566+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48a40b2ce0 con 0x7f48a40d0210 2024-03-14T12:16:19.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.566+0000 7f4892ffd640 1 -- 172.21.15.82:0/4200903196 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f489807a870 con 0x7f48a40d0210 2024-03-14T12:16:19.566 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f488406bbd0 msgr2=0x7f488406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f488406bbd0 0x7f488406e090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f48940012e0 tx=0x7f4894009040 comp rx=0 tx=0).stop 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a40d0210 msgr2=0x7f48a413e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a40d0210 0x7f48a413e8e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4898000c00 tx=0x7f48980046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 shutdown_connections 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f488406bbd0 0x7f488406e090 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48a41446f0 0x7f48a4140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48a40d0210 0x7f48a413e8e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 --2- 172.21.15.82:0/4200903196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48a40cfe30 0x7f48a4139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 >> 172.21.15.82:0/4200903196 conn(0x7f48a406aef0 msgr2=0x7f48a415dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 shutdown_connections 2024-03-14T12:16:19.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.568+0000 7f48a8ff1640 1 -- 172.21.15.82:0/4200903196 wait complete. 2024-03-14T12:16:19.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:16:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.681+0000 7f0a20e8e640 1 Processor -- start 2024-03-14T12:16:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.681+0000 7f0a20e8e640 1 -- start start 2024-03-14T12:16:19.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a20e8e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 0x7f0a1c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a20e8e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c0670c0 con 0x7f0a1c0b1e00 2024-03-14T12:16:19.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a20e8e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c06c130 con 0x7f0a1c15c2f0 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a20e8e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c0b62e0 con 0x7f0a1c15bf10 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a1ad9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0a1c0b1e00 0x7f0a1c142e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44702/0 (socket says 172.21.15.82:44702) 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a1ad9d640 1 -- 172.21.15.82:0/2256802013 learned_addr learned my addr 172.21.15.82:0/2256802013 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 0x7f0a1c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953901133 0 0) 0x7f0a1c0670c0 con 0x7f0a1c0b1e00 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.682+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a04003520 con 0x7f0a1c0b1e00 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 731027850 0 0) 0x7f0a04003520 con 0x7f0a1c0b1e00 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 msgr2=0x7f0a1c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 0x7f0a1c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 >> v1:172.21.15.164:6789/0 conn(0x7f0a1c15c2f0 legacy=0x7f0a1c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c066de0 con 0x7f0a1c0b1e00 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f0a140019f0 con 0x7f0a1c0b1e00 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 >> v1:172.21.15.82:6789/0 conn(0x7f0a1c0b1e00 legacy=0x7f0a1c142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a04003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a04004390 0x7f0a04004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 0x7f0a04005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c0b62e0 con 0x7f0a040039e0 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0400ba70 con 0x7f0a04004390 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0400bc70 con 0x7f0a1c15bf10 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a04003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a19d9b640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a04004390 0x7f0a04004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1ad9d640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 0x7f0a04005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1a59c640 1 -- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 msgr2=0x7f0a04005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15bf10 0x7f0a04005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1a59c640 1 -- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a04004390 msgr2=0x7f0a04004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a04004390 0x7f0a04004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.683+0000 7f0a1a59c640 1 -- 172.21.15.82:0/2256802013 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04003520 con 0x7f0a040039e0 2024-03-14T12:16:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.684+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a04003dc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ac0 tx=0x7f0a04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac5b37b68007e80 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.684+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10039070 con 0x7f0a040039e0 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.684+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1002c6b0 con 0x7f0a040039e0 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.684+0000 7f0a1959a640 1 -- 172.21.15.82:0/2256802013 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10009690 con 0x7f0a040039e0 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.684+0000 7f0a20e8e640 1 -- 172.21.15.82:0/2256802013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 msgr2=0x7f0a04003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.684+0000 7f0a20e8e640 1 --2- 172.21.15.82:0/2256802013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a04003dc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ac0 tx=0x7f0a04005830 comp rx=0 tx=0).stop 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.684+0000 7f0a1ad9d640 1 -- 172.21.15.82:0/2256802013 reap_dead start 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.685+0000 7f0a20e8e640 1 -- 172.21.15.82:0/2256802013 shutdown_connections 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.685+0000 7f0a20e8e640 1 -- 172.21.15.82:0/2256802013 >> 172.21.15.82:0/2256802013 conn(0x7f0a1c0b4fd0 msgr2=0x7f0a1c0b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.685+0000 7f0a20e8e640 1 -- 172.21.15.82:0/2256802013 shutdown_connections 2024-03-14T12:16:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.685+0000 7f0a20e8e640 1 -- 172.21.15.82:0/2256802013 wait complete. 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.686+0000 7f0a20e8e640 1 Processor -- start 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.686+0000 7f0a20e8e640 1 -- start start 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a1c0994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15ea30 0x7f0a1c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a1c154b80 0x7f0a1c155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c145b60 con 0x7f0a040039e0 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c06c740 con 0x7f0a1c154b80 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c066a20 con 0x7f0a1c15ea30 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a1c0994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a1c0994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49984/0 (socket says 172.21.15.82:49984) 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 -- 172.21.15.82:0/3499609432 learned_addr learned my addr 172.21.15.82:0/3499609432 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1ad9d640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a1c154b80 0x7f0a1c155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a19d9b640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15ea30 0x7f0a1c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 -- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15ea30 msgr2=0x7f0a1c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15ea30 0x7f0a1c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 -- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a1c154b80 msgr2=0x7f0a1c155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a1c154b80 0x7f0a1c155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 -- 172.21.15.82:0/3499609432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c099a20 con 0x7f0a040039e0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a19d9b640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15ea30 0x7f0a1c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a1a59c640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a1c0994e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0a10037040 tx=0x7f0a10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a0affd640 1 -- 172.21.15.82:0/3499609432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10042020 con 0x7f0a040039e0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a0affd640 1 -- 172.21.15.82:0/3499609432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1003a040 con 0x7f0a040039e0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c15d7b0 con 0x7f0a040039e0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a0affd640 1 -- 172.21.15.82:0/3499609432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10039070 con 0x7f0a040039e0 2024-03-14T12:16:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.687+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c15dd70 con 0x7f0a040039e0 2024-03-14T12:16:19.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.689+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c0687d0 con 0x7f0a040039e0 2024-03-14T12:16:19.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.689+0000 7f0a0affd640 1 -- 172.21.15.82:0/3499609432 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0a10045020 con 0x7f0a040039e0 2024-03-14T12:16:19.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.689+0000 7f0a0affd640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f09fc06bc00 0x7f09fc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.689+0000 7f0a19d9b640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f09fc06bc00 0x7f09fc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.689+0000 7f0a0affd640 1 -- 172.21.15.82:0/3499609432 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a100abdb0 con 0x7f0a040039e0 2024-03-14T12:16:19.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.690+0000 7f0a19d9b640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f09fc06bc00 0x7f09fc06e0c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c073e30 tx=0x7f0a0c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.693+0000 7f0a0affd640 1 -- 172.21.15.82:0/3499609432 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0a1c0687d0 con 0x7f0a040039e0 2024-03-14T12:16:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.866+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a1c04f050 con 0x7f0a040039e0 2024-03-14T12:16:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.866+0000 7f0a0affd640 1 -- 172.21.15.82:0/3499609432 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a1004c070 con 0x7f0a040039e0 2024-03-14T12:16:19.866 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f09fc06bc00 msgr2=0x7f09fc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f09fc06bc00 0x7f09fc06e0c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c073e30 tx=0x7f0a0c009040 comp rx=0 tx=0).stop 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 msgr2=0x7f0a1c0994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a1c0994e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0a10037040 tx=0x7f0a10004730 comp rx=0 tx=0).stop 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 shutdown_connections 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a1c154b80 0x7f0a1c155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f09fc06bc00 0x7f09fc06e0c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a1c15ea30 0x7f0a1c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 --2- 172.21.15.82:0/3499609432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a040039e0 0x7f0a1c0994e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 >> 172.21.15.82:0/3499609432 conn(0x7f0a1c0b4fd0 msgr2=0x7f0a1c0af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 shutdown_connections 2024-03-14T12:16:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.868+0000 7f0a20e8e640 1 -- 172.21.15.82:0/3499609432 wait complete. 2024-03-14T12:16:19.878 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:19.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.978+0000 7f2e82a38640 1 Processor -- start 2024-03-14T12:16:19.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.978+0000 7f2e82a38640 1 -- start start 2024-03-14T12:16:19.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e82a38640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 0x7f2e7c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e82a38640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7c15ebe0 con 0x7f2e7c143f60 2024-03-14T12:16:19.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e82a38640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7c15eec0 con 0x7f2e7c071740 2024-03-14T12:16:19.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e82a38640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c15f1a0 con 0x7f2e7c15e670 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e80fd5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2e7c143f60 0x7f2e7c15b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44712/0 (socket says 172.21.15.82:44712) 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e80fd5640 1 -- 172.21.15.82:0/955627390 learned_addr learned my addr 172.21.15.82:0/955627390 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e7b7fe640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 0x7f2e7c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488982302 0 0) 0x7f2e7c15ebe0 con 0x7f2e7c143f60 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.979+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e64003520 con 0x7f2e7c143f60 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014909228 0 0) 0x7f2e64003520 con 0x7f2e7c143f60 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 msgr2=0x7f2e7c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 0x7f2e7c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 >> v1:172.21.15.164:6789/0 conn(0x7f2e7c071740 legacy=0x7f2e7c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c15f480 con 0x7f2e7c143f60 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f2e740019f0 con 0x7f2e7c143f60 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 >> v1:172.21.15.82:6789/0 conn(0x7f2e7c143f60 legacy=0x7f2e7c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e640039e0 0x7f2e64003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 0x7f2e64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e64004d40 0x7f2e640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c15f1a0 con 0x7f2e64004d40 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6400b9f0 con 0x7f2e640039e0 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6400bbf0 con 0x7f2e7c15e670 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e80fd5640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e64004d40 0x7f2e640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7b7fe640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 0x7f2e64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e7bfff640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e640039e0 0x7f2e64003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e80fd5640 1 -- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 msgr2=0x7f2e64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e80fd5640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c15e670 0x7f2e64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e80fd5640 1 -- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e640039e0 msgr2=0x7f2e64003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e80fd5640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e640039e0 0x7f2e64003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.980+0000 7f2e80fd5640 1 -- 172.21.15.82:0/955627390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64003520 con 0x7f2e64004d40 2024-03-14T12:16:19.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.981+0000 7f2e80fd5640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e64004d40 0x7f2e640051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400a7c0 tx=0x7f2e64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688eec11dc1ee50d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.981+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400a410 con 0x7f2e64004d40 2024-03-14T12:16:19.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.981+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e74034630 con 0x7f2e64004d40 2024-03-14T12:16:19.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.981+0000 7f2e7affd640 1 -- 172.21.15.82:0/955627390 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e74034a20 con 0x7f2e64004d40 2024-03-14T12:16:19.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.982+0000 7f2e82a38640 1 -- 172.21.15.82:0/955627390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e64004d40 msgr2=0x7f2e640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.982+0000 7f2e82a38640 1 --2- 172.21.15.82:0/955627390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e64004d40 0x7f2e640051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400a7c0 tx=0x7f2e64005830 comp rx=0 tx=0).stop 2024-03-14T12:16:19.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.982+0000 7f2e80fd5640 1 -- 172.21.15.82:0/955627390 reap_dead start 2024-03-14T12:16:19.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.982+0000 7f2e82a38640 1 -- 172.21.15.82:0/955627390 shutdown_connections 2024-03-14T12:16:19.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.982+0000 7f2e82a38640 1 -- 172.21.15.82:0/955627390 >> 172.21.15.82:0/955627390 conn(0x7f2e7c0b1c70 msgr2=0x7f2e7c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:19.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.982+0000 7f2e82a38640 1 -- 172.21.15.82:0/955627390 shutdown_connections 2024-03-14T12:16:19.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.982+0000 7f2e82a38640 1 -- 172.21.15.82:0/955627390 wait complete. 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.983+0000 7f2e82a38640 1 Processor -- start 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.983+0000 7f2e82a38640 1 -- start start 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e7c0caf30 0x7f2e7c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7c15d090 0x7f2e7c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c0b4460 0x7f2e7c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c144670 con 0x7f2e7c15d090 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c15b080 con 0x7f2e7c0caf30 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c1447f0 con 0x7f2e7c0b4460 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7c15d090 0x7f2e7c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7c15d090 0x7f2e7c0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50000/0 (socket says 172.21.15.82:50000) 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 -- 172.21.15.82:0/1305004627 learned_addr learned my addr 172.21.15.82:0/1305004627 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7bfff640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e7c0caf30 0x7f2e7c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e80fd5640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c0b4460 0x7f2e7c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 -- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c0b4460 msgr2=0x7f2e7c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c0b4460 0x7f2e7c070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 -- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e7c0caf30 msgr2=0x7f2e7c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e7c0caf30 0x7f2e7c0c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 -- 172.21.15.82:0/1305004627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c15dba0 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7bfff640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e7c0caf30 0x7f2e7c0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e80fd5640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c0b4460 0x7f2e7c070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e7b7fe640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7c15d090 0x7f2e7c0b3f20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2e700119a0 tx=0x7f2e70011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e78ff9640 1 -- 172.21.15.82:0/1305004627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7000faf0 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e78ff9640 1 -- 172.21.15.82:0/1305004627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7000fc90 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7c0c9430 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e78ff9640 1 -- 172.21.15.82:0/1305004627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70014440 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.984+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7c0c99f0 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.985+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7c04f050 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.986+0000 7f2e78ff9640 1 -- 172.21.15.82:0/1305004627 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2e70012040 con 0x7f2e7c15d090 2024-03-14T12:16:19.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.986+0000 7f2e78ff9640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e6806bc00 0x7f2e6806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:19.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.986+0000 7f2e78ff9640 1 -- 172.21.15.82:0/1305004627 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e7008b0a0 con 0x7f2e7c15d090 2024-03-14T12:16:19.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.986+0000 7f2e7bfff640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e6806bc00 0x7f2e6806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:19.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.987+0000 7f2e7bfff640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e6806bc00 0x7f2e6806e0c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c000c00 tx=0x7f2e6c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:19.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:19.990+0000 7f2e78ff9640 1 -- 172.21.15.82:0/1305004627 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2e7c15dba0 con 0x7f2e7c15d090 2024-03-14T12:16:20.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.197+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e7c0d07b0 con 0x7f2e7c15d090 2024-03-14T12:16:20.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.198+0000 7f2e78ff9640 1 -- 172.21.15.82:0/1305004627 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f2e70019030 con 0x7f2e7c15d090 2024-03-14T12:16:20.197 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:20.197 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1414844632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":1414844632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1414844632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/456693586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":456693586},{"type":"v1","addr":"172.21.15.164:6834","nonce":456693586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/58260684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":58260684},{"type":"v1","addr":"172.21.15.82:6836","nonce":58260684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:02.017303+0000","modified":"2024-03-14T12:16:04.014283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2708572159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2708572159},{"type":"v1","addr":"172.21.15.82:6837","nonce":2708572159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T12:16:20.198 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 21 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e6806bc00 msgr2=0x7f2e6806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e6806bc00 0x7f2e6806e0c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c000c00 tx=0x7f2e6c009040 comp rx=0 tx=0).stop 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7c15d090 msgr2=0x7f2e7c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7c15d090 0x7f2e7c0b3f20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2e700119a0 tx=0x7f2e70011e60 comp rx=0 tx=0).stop 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 shutdown_connections 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2e6806bc00 0x7f2e6806e0c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e7c0b4460 0x7f2e7c070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e7c15d090 0x7f2e7c0b3f20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 --2- 172.21.15.82:0/1305004627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e7c0caf30 0x7f2e7c0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 >> 172.21.15.82:0/1305004627 conn(0x7f2e7c0b1c70 msgr2=0x7f2e7c143310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 shutdown_connections 2024-03-14T12:16:20.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.200+0000 7f2e82a38640 1 -- 172.21.15.82:0/1305004627 wait complete. 2024-03-14T12:16:20.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:20.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.311+0000 7fd4d617e640 1 Processor -- start 2024-03-14T12:16:20.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.311+0000 7fd4d617e640 1 -- start start 2024-03-14T12:16:20.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.311+0000 7fd4d617e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4d015e670 0x7fd4d00b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.311+0000 7fd4d617e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd4d0143f60 0x7fd4d015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.311+0000 7fd4d617e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d015ebe0 con 0x7fd4d0071740 2024-03-14T12:16:20.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.311+0000 7fd4d617e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d015eec0 con 0x7fd4d0143f60 2024-03-14T12:16:20.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.311+0000 7fd4d617e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d015f1a0 con 0x7fd4d015e670 2024-03-14T12:16:20.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd4d0143f60 0x7fd4d015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd4d0143f60 0x7fd4d015b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50014/0 (socket says 172.21.15.82:50014) 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cf7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd4d0071740 0x7fd4d0071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44728/0 (socket says 172.21.15.82:44728) 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 -- 172.21.15.82:0/3053255513 learned_addr learned my addr 172.21.15.82:0/3053255513 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 -- 172.21.15.82:0/3053255513 >> v2:172.21.15.164:3300/0 conn(0x7fd4d015e670 msgr2=0x7fd4d00b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 --2- 172.21.15.82:0/3053255513 >> v2:172.21.15.164:3300/0 conn(0x7fd4d015e670 0x7fd4d00b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 -- 172.21.15.82:0/3053255513 >> v1:172.21.15.82:6789/0 conn(0x7fd4d0071740 legacy=0x7fd4d0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 -- 172.21.15.82:0/3053255513 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d015f480 con 0x7fd4d0143f60 2024-03-14T12:16:20.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4ceffd640 1 -- 172.21.15.82:0/3053255513 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078833432 0 0) 0x7fd4d015ebe0 con 0x7fd4d0071740 2024-03-14T12:16:20.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.312+0000 7fd4cffff640 1 --2- 172.21.15.82:0/3053255513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d015b610 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e50 tx=0x7fd4c802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc86a3c964be4351 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4ceffd640 1 -- 172.21.15.82:0/3053255513 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c802cb20 con 0x7fd4d0143f60 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4ceffd640 1 -- 172.21.15.82:0/3053255513 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c802ccc0 con 0x7fd4d0143f60 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4ceffd640 1 -- 172.21.15.82:0/3053255513 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8034610 con 0x7fd4d0143f60 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 -- 172.21.15.82:0/3053255513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 msgr2=0x7fd4d015b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 --2- 172.21.15.82:0/3053255513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d015b610 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009e50 tx=0x7fd4c802bf80 comp rx=0 tx=0).stop 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 -- 172.21.15.82:0/3053255513 shutdown_connections 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 --2- 172.21.15.82:0/3053255513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d015b610 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 --2- 172.21.15.82:0/3053255513 >> v2:172.21.15.164:3300/0 conn(0x7fd4d015e670 0x7fd4d00b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 -- 172.21.15.82:0/3053255513 >> 172.21.15.82:0/3053255513 conn(0x7fd4d00b1c70 msgr2=0x7fd4d00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 -- 172.21.15.82:0/3053255513 shutdown_connections 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.313+0000 7fd4d617e640 1 -- 172.21.15.82:0/3053255513 wait complete. 2024-03-14T12:16:20.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.314+0000 7fd4d617e640 1 Processor -- start 2024-03-14T12:16:20.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4d617e640 1 -- start start 2024-03-14T12:16:20.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4d617e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4d0071740 0x7fd4d0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4d617e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d0158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4d617e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d015e670 0x7fd4d0166130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4d617e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0144670 con 0x7fd4d0143f60 2024-03-14T12:16:20.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4d617e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d015b080 con 0x7fd4d015e670 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4d617e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01447f0 con 0x7fd4d0071740 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d0158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d0158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50022/0 (socket says 172.21.15.82:50022) 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.315+0000 7fd4c6ffd640 1 -- 172.21.15.82:0/1681430242 learned_addr learned my addr 172.21.15.82:0/1681430242 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4cffff640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d015e670 0x7fd4d0166130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4c6ffd640 1 -- 172.21.15.82:0/1681430242 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4d0071740 msgr2=0x7fd4d0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4c6ffd640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4d0071740 0x7fd4d0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4c6ffd640 1 -- 172.21.15.82:0/1681430242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d015e670 msgr2=0x7fd4d0166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4c6ffd640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d015e670 0x7fd4d0166130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4c6ffd640 1 -- 172.21.15.82:0/1681430242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d015a710 con 0x7fd4d0143f60 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4c6ffd640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d0158420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800b4a0 tx=0x7fd4b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4ccff9640 1 -- 172.21.15.82:0/1681430242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8014070 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4ccff9640 1 -- 172.21.15.82:0/1681430242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8010610 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4ccff9640 1 -- 172.21.15.82:0/1681430242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8010a00 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d015a9f0 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.316+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d0165c10 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.317+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d0165a00 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.317+0000 7fd4ccff9640 1 -- 172.21.15.82:0/1681430242 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd4b8017070 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.318+0000 7fd4ccff9640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd4b406bbd0 0x7fd4b406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.318+0000 7fd4ccff9640 1 -- 172.21.15.82:0/1681430242 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4b808a4c0 con 0x7fd4d0143f60 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.318+0000 7fd4cf7fe640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd4b406bbd0 0x7fd4b406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.318+0000 7fd4cf7fe640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd4b406bbd0 0x7fd4b406e090 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0007fd0 tx=0x7fd4c0007960 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.322+0000 7fd4ccff9640 1 -- 172.21.15.82:0/1681430242 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd4b8058fb0 con 0x7fd4d0143f60 2024-03-14T12:16:20.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.496+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4d004f050 con 0x7fd4d0143f60 2024-03-14T12:16:20.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.497+0000 7fd4ccff9640 1 -- 172.21.15.82:0/1681430242 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fd4b80588b0 con 0x7fd4d0143f60 2024-03-14T12:16:20.497 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:20.497 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:03.671135+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd4b406bbd0 msgr2=0x7fd4b406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd4b406bbd0 0x7fd4b406e090 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0007fd0 tx=0x7fd4c0007960 comp rx=0 tx=0).stop 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 msgr2=0x7fd4d0158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d0158420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800b4a0 tx=0x7fd4b800b960 comp rx=0 tx=0).stop 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 shutdown_connections 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d015e670 0x7fd4d0166130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4d0143f60 0x7fd4d0158420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd4b406bbd0 0x7fd4b406e090 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 --2- 172.21.15.82:0/1681430242 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4d0071740 0x7fd4d0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 >> 172.21.15.82:0/1681430242 conn(0x7fd4d00b1c70 msgr2=0x7fd4d00afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:20.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 shutdown_connections 2024-03-14T12:16:20.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.499+0000 7fd4d617e640 1 -- 172.21.15.82:0/1681430242 wait complete. 2024-03-14T12:16:20.510 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T12:16:20.510 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:20.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.610+0000 7f82d5cfc640 1 Processor -- start 2024-03-14T12:16:20.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.610+0000 7f82d5cfc640 1 -- start start 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82d5cfc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f82d00b0f60 0x7f82d00cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82d5cfc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00afff0 0x7f82d00d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82d5cfc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d00b1340 con 0x7f82d006ad10 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82d5cfc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d00d0ee0 con 0x7f82d00b0f60 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82d5cfc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00b4ee0 con 0x7f82d00afff0 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82cf7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f82d00b0f60 0x7f82d00cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82cf7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f82d00b0f60 0x7f82d00cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50034/0 (socket says 172.21.15.82:50034) 2024-03-14T12:16:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82cf7fe640 1 -- 172.21.15.82:0/457089815 learned_addr learned my addr 172.21.15.82:0/457089815 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82cf7fe640 1 -- 172.21.15.82:0/457089815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00afff0 msgr2=0x7f82d00d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82cf7fe640 1 --2- 172.21.15.82:0/457089815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00afff0 0x7f82d00d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82cf7fe640 1 -- 172.21.15.82:0/457089815 >> v1:172.21.15.164:6789/0 conn(0x7f82d006ad10 legacy=0x7f82d006b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.611+0000 7f82cf7fe640 1 -- 172.21.15.82:0/457089815 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d00cd030 con 0x7f82d00b0f60 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.612+0000 7f82cf7fe640 1 --2- 172.21.15.82:0/457089815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d00b0f60 0x7f82d00cc9a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009870 tx=0x7f82c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac10d4da59f3d3e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.612+0000 7f82ce7fc640 1 -- 172.21.15.82:0/457089815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0035020 con 0x7f82d00b0f60 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.612+0000 7f82ce7fc640 1 -- 172.21.15.82:0/457089815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c0031640 con 0x7f82d00b0f60 2024-03-14T12:16:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.612+0000 7f82ce7fc640 1 -- 172.21.15.82:0/457089815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c002d040 con 0x7f82d00b0f60 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 -- 172.21.15.82:0/457089815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d00b0f60 msgr2=0x7f82d00cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/457089815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d00b0f60 0x7f82d00cc9a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009870 tx=0x7f82c002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 -- 172.21.15.82:0/457089815 shutdown_connections 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/457089815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00afff0 0x7f82d00d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/457089815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d00b0f60 0x7f82d00cc9a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 -- 172.21.15.82:0/457089815 >> 172.21.15.82:0/457089815 conn(0x7f82d0072310 msgr2=0x7f82d0072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 -- 172.21.15.82:0/457089815 shutdown_connections 2024-03-14T12:16:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.613+0000 7f82d5cfc640 1 -- 172.21.15.82:0/457089815 wait complete. 2024-03-14T12:16:20.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.614+0000 7f82d5cfc640 1 Processor -- start 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.614+0000 7f82d5cfc640 1 -- start start 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82d5cfc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d006ad10 0x7f82d01557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82d5cfc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82d00afff0 0x7f82d0151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82d5cfc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00b0f60 0x7f82d0099490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82d5cfc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00cd8d0 con 0x7f82d006ad10 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82d5cfc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00cd5f0 con 0x7f82d00afff0 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82d5cfc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00b0450 con 0x7f82d00b0f60 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d006ad10 0x7f82d01557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d006ad10 0x7f82d01557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50046/0 (socket says 172.21.15.82:50046) 2024-03-14T12:16:20.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 -- 172.21.15.82:0/814089383 learned_addr learned my addr 172.21.15.82:0/814089383 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cffff640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00b0f60 0x7f82d0099490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82ceffd640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82d00afff0 0x7f82d0151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 -- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00b0f60 msgr2=0x7f82d0099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00b0f60 0x7f82d0099490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 -- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82d00afff0 msgr2=0x7f82d0151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82d00afff0 0x7f82d0151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 -- 172.21.15.82:0/814089383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d00ca020 con 0x7f82d006ad10 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cffff640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00b0f60 0x7f82d0099490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:20.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.615+0000 7f82cf7fe640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d006ad10 0x7f82d01557f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f82c002b930 tx=0x7f82c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.616+0000 7f82b7fff640 1 -- 172.21.15.82:0/814089383 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c003ac60 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.616+0000 7f82b7fff640 1 -- 172.21.15.82:0/814089383 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c003ae20 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.616+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d009ba40 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.616+0000 7f82b7fff640 1 -- 172.21.15.82:0/814089383 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0044630 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.616+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d00c9d40 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.617+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d004f050 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.617+0000 7f82b7fff640 1 -- 172.21.15.82:0/814089383 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f82c004b060 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.617+0000 7f82b7fff640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82bc06bbd0 0x7f82bc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.618+0000 7f82b7fff640 1 -- 172.21.15.82:0/814089383 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82c00ac630 con 0x7f82d006ad10 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.618+0000 7f82ceffd640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82bc06bbd0 0x7f82bc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.618+0000 7f82ceffd640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82bc06bbd0 0x7f82bc06e090 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f82d00d0090 tx=0x7f82c4009110 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.622+0000 7f82b7fff640 1 -- 172.21.15.82:0/814089383 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f82d009ba40 con 0x7f82d006ad10 2024-03-14T12:16:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.830+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82d00b2770 con 0x7f82d006ad10 2024-03-14T12:16:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.830+0000 7f82b7fff640 1 -- 172.21.15.82:0/814089383 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f82c003f070 con 0x7f82d006ad10 2024-03-14T12:16:20.830 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:20.830 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1414844632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":1414844632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1414844632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/456693586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":456693586},{"type":"v1","addr":"172.21.15.164:6834","nonce":456693586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/58260684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":58260684},{"type":"v1","addr":"172.21.15.82:6836","nonce":58260684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:02.017303+0000","modified":"2024-03-14T12:16:04.014283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2708572159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2708572159},{"type":"v1","addr":"172.21.15.82:6837","nonce":2708572159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T12:16:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 21 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82bc06bbd0 msgr2=0x7f82bc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82bc06bbd0 0x7f82bc06e090 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f82d00d0090 tx=0x7f82c4009110 comp rx=0 tx=0).stop 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d006ad10 msgr2=0x7f82d01557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d006ad10 0x7f82d01557f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f82c002b930 tx=0x7f82c00046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 shutdown_connections 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82bc06bbd0 0x7f82bc06e090 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82d00b0f60 0x7f82d0099490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82d00afff0 0x7f82d0151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 --2- 172.21.15.82:0/814089383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82d006ad10 0x7f82d01557f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 >> 172.21.15.82:0/814089383 conn(0x7f82d0072310 msgr2=0x7f82d00cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.832+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 shutdown_connections 2024-03-14T12:16:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.833+0000 7f82d5cfc640 1 -- 172.21.15.82:0/814089383 wait complete. 2024-03-14T12:16:20.843 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:20.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.942+0000 7fdbc66fc640 1 Processor -- start 2024-03-14T12:16:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.942+0000 7fdbc66fc640 1 -- start start 2024-03-14T12:16:20.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc66fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0071f90 0x7fdbc015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc66fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbc00b3fe0 0x7fdbc00b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc66fc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc0147c60 con 0x7fdbc0147880 2024-03-14T12:16:20.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc66fc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc00adcf0 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc66fc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc00b4950 con 0x7fdbc0071f90 2024-03-14T12:16:20.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc4c99640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbc00b3fe0 0x7fdbc00b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc4c99640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbc00b3fe0 0x7fdbc00b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50056/0 (socket says 172.21.15.82:50056) 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.943+0000 7fdbc4c99640 1 -- 172.21.15.82:0/2539279274 learned_addr learned my addr 172.21.15.82:0/2539279274 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbc4c99640 1 -- 172.21.15.82:0/2539279274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0071f90 msgr2=0x7fdbc015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbbf7fe640 1 --2- 172.21.15.82:0/2539279274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0071f90 0x7fdbc015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbc4c99640 1 --2- 172.21.15.82:0/2539279274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0071f90 0x7fdbc015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbc4c99640 1 -- 172.21.15.82:0/2539279274 >> v1:172.21.15.164:6789/0 conn(0x7fdbc0147880 legacy=0x7fdbc006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbc4c99640 1 -- 172.21.15.82:0/2539279274 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc0072600 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbbf7fe640 1 --2- 172.21.15.82:0/2539279274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0071f90 0x7fdbc015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbc4c99640 1 --2- 172.21.15.82:0/2539279274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc00b4410 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009e50 tx=0x7fdbb802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fa471c2abd338c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbbeffd640 1 -- 172.21.15.82:0/2539279274 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb802c9e0 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbbeffd640 1 -- 172.21.15.82:0/2539279274 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb802cb80 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.944+0000 7fdbbeffd640 1 -- 172.21.15.82:0/2539279274 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80343f0 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 -- 172.21.15.82:0/2539279274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 msgr2=0x7fdbc00b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/2539279274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc00b4410 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009e50 tx=0x7fdbb802be40 comp rx=0 tx=0).stop 2024-03-14T12:16:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 -- 172.21.15.82:0/2539279274 shutdown_connections 2024-03-14T12:16:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/2539279274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc00b4410 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/2539279274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0071f90 0x7fdbc015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 -- 172.21.15.82:0/2539279274 >> 172.21.15.82:0/2539279274 conn(0x7fdbc00b1bd0 msgr2=0x7fdbc00b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 -- 172.21.15.82:0/2539279274 shutdown_connections 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.945+0000 7fdbc66fc640 1 -- 172.21.15.82:0/2539279274 wait complete. 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.946+0000 7fdbc66fc640 1 Processor -- start 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.946+0000 7fdbc66fc640 1 -- start start 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbc66fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc0071f90 0x7fdbc00ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbc66fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc0138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbc66fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0147880 0x7fdbc00cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbc66fc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc0072bc0 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbc66fc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc00ad980 con 0x7fdbc0071f90 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbc66fc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc015eb10 con 0x7fdbc0147880 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc0138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc0138600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50072/0 (socket says 172.21.15.82:50072) 2024-03-14T12:16:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 -- 172.21.15.82:0/945554516 learned_addr learned my addr 172.21.15.82:0/945554516 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbffff640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc0071f90 0x7fdbc00ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbc4c99640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0147880 0x7fdbc00cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 -- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0147880 msgr2=0x7fdbc00cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0147880 0x7fdbc00cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 -- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc0071f90 msgr2=0x7fdbc00ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc0071f90 0x7fdbc00ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.947+0000 7fdbbf7fe640 1 -- 172.21.15.82:0/945554516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc0138b40 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.948+0000 7fdbbffff640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc0071f90 0x7fdbc00ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.948+0000 7fdbbf7fe640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc0138600 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c7f0 tx=0x7fdbb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.948+0000 7fdbbcff9640 1 -- 172.21.15.82:0/945554516 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4015070 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.948+0000 7fdbbcff9640 1 -- 172.21.15.82:0/945554516 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb4007a30 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.948+0000 7fdbbcff9640 1 -- 172.21.15.82:0/945554516 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4007e00 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.948+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc00ce9a0 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.948+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc00cf240 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.949+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc0074690 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.949+0000 7fdbbcff9640 1 -- 172.21.15.82:0/945554516 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdbb4018070 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.950+0000 7fdbbcff9640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdbac06bbd0 0x7fdbac06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.950+0000 7fdbbcff9640 1 -- 172.21.15.82:0/945554516 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbb408aab0 con 0x7fdbc00b3fe0 2024-03-14T12:16:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.950+0000 7fdbbffff640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdbac06bbd0 0x7fdbac06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.950+0000 7fdbbffff640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdbac06bbd0 0x7fdbac06e090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0006fd0 tx=0x7fdbb0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:20.954+0000 7fdbbcff9640 1 -- 172.21.15.82:0/945554516 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdbc0138b40 con 0x7fdbc00b3fe0 2024-03-14T12:16:21.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.160+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbc0146f30 con 0x7fdbc00b3fe0 2024-03-14T12:16:21.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.161+0000 7fdbbcff9640 1 -- 172.21.15.82:0/945554516 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fdbb4059510 con 0x7fdbc00b3fe0 2024-03-14T12:16:21.161 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:21.161 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1414844632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":1414844632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1414844632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/456693586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":456693586},{"type":"v1","addr":"172.21.15.164:6834","nonce":456693586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/58260684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":58260684},{"type":"v1","addr":"172.21.15.82:6836","nonce":58260684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:02.017303+0000","modified":"2024-03-14T12:16:04.014283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2708572159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2708572159},{"type":"v1","addr":"172.21.15.82:6837","nonce":2708572159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T12:16:21.162 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 21 2024-03-14T12:16:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdbac06bbd0 msgr2=0x7fdbac06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdbac06bbd0 0x7fdbac06e090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0006fd0 tx=0x7fdbb0008040 comp rx=0 tx=0).stop 2024-03-14T12:16:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 msgr2=0x7fdbc0138600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc0138600 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c7f0 tx=0x7fdbb400cbf0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 shutdown_connections 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdbac06bbd0 0x7fdbac06e090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc0147880 0x7fdbc00cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00b3fe0 0x7fdbc0138600 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 --2- 172.21.15.82:0/945554516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc0071f90 0x7fdbc00ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 >> 172.21.15.82:0/945554516 conn(0x7fdbc00b1bd0 msgr2=0x7fdbc015ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 shutdown_connections 2024-03-14T12:16:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.163+0000 7fdbc66fc640 1 -- 172.21.15.82:0/945554516 wait complete. 2024-03-14T12:16:21.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.274+0000 7f65f60e9640 1 Processor -- start 2024-03-14T12:16:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.274+0000 7f65f60e9640 1 -- start start 2024-03-14T12:16:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65f60e9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f65f0071740 0x7f65f0071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65f60e9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65f015e670 0x7f65f00b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65f60e9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f0072060 con 0x7f65f0143f60 2024-03-14T12:16:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65f60e9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f015ebe0 con 0x7f65f015e670 2024-03-14T12:16:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65f60e9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f015eec0 con 0x7f65f0071740 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65eeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65f015e670 0x7f65f00b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65eeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65f015e670 0x7f65f00b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50082/0 (socket says 172.21.15.82:50082) 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65eeffd640 1 -- 172.21.15.82:0/1928930205 learned_addr learned my addr 172.21.15.82:0/1928930205 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65ef7fe640 1 --2- 172.21.15.82:0/1928930205 >> v2:172.21.15.164:3300/0 conn(0x7f65f0071740 0x7f65f0071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65eeffd640 1 -- 172.21.15.82:0/1928930205 >> v2:172.21.15.164:3300/0 conn(0x7f65f0071740 msgr2=0x7f65f0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65eeffd640 1 --2- 172.21.15.82:0/1928930205 >> v2:172.21.15.164:3300/0 conn(0x7f65f0071740 0x7f65f0071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65eeffd640 1 -- 172.21.15.82:0/1928930205 >> v1:172.21.15.164:6789/0 conn(0x7f65f0143f60 legacy=0x7f65f015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.275+0000 7f65eeffd640 1 -- 172.21.15.82:0/1928930205 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f015f1a0 con 0x7f65f015e670 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.276+0000 7f65eeffd640 1 --2- 172.21.15.82:0/1928930205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015e670 0x7f65f00b2940 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f65f00d04e0 tx=0x7f65e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768e9244ba1158cd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.276+0000 7f65ee7fc640 1 -- 172.21.15.82:0/1928930205 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4034070 con 0x7f65f015e670 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.276+0000 7f65ee7fc640 1 -- 172.21.15.82:0/1928930205 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4030860 con 0x7f65f015e670 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.276+0000 7f65ee7fc640 1 -- 172.21.15.82:0/1928930205 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4030c30 con 0x7f65f015e670 2024-03-14T12:16:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.276+0000 7f65f60e9640 1 -- 172.21.15.82:0/1928930205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015e670 msgr2=0x7f65f00b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.276+0000 7f65f60e9640 1 --2- 172.21.15.82:0/1928930205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015e670 0x7f65f00b2940 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f65f00d04e0 tx=0x7f65e402b9a0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.276+0000 7f65f60e9640 1 -- 172.21.15.82:0/1928930205 shutdown_connections 2024-03-14T12:16:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.277+0000 7f65f60e9640 1 --2- 172.21.15.82:0/1928930205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015e670 0x7f65f00b2940 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.277+0000 7f65f60e9640 1 --2- 172.21.15.82:0/1928930205 >> v2:172.21.15.164:3300/0 conn(0x7f65f0071740 0x7f65f0071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.277+0000 7f65f60e9640 1 -- 172.21.15.82:0/1928930205 >> 172.21.15.82:0/1928930205 conn(0x7f65f00b1c70 msgr2=0x7f65f00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.277+0000 7f65f60e9640 1 -- 172.21.15.82:0/1928930205 shutdown_connections 2024-03-14T12:16:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.277+0000 7f65f60e9640 1 -- 172.21.15.82:0/1928930205 wait complete. 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.278+0000 7f65f60e9640 1 Processor -- start 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.278+0000 7f65f60e9640 1 -- start start 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.278+0000 7f65f60e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65f0071740 0x7f65f01388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65f60e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f0143f60 0x7f65f013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65f60e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65f015e670 0x7f65f015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65f60e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015b080 con 0x7f65f0143f60 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65f60e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0143ce0 con 0x7f65f015e670 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65f60e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0144670 con 0x7f65f0071740 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f0143f60 0x7f65f013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f0143f60 0x7f65f013c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50092/0 (socket says 172.21.15.82:50092) 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 -- 172.21.15.82:0/3284674713 learned_addr learned my addr 172.21.15.82:0/3284674713 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 -- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65f0071740 msgr2=0x7f65f01388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65f0071740 0x7f65f01388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 -- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65f015e670 msgr2=0x7f65f015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65effff640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65f015e670 0x7f65f015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65f015e670 0x7f65f015a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 -- 172.21.15.82:0/3284674713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f013f2d0 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65effff640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65f015e670 0x7f65f015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65eeffd640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f0143f60 0x7f65f013c6b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65d7fff640 1 -- 172.21.15.82:0/3284674713 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4034040 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65d7fff640 1 -- 172.21.15.82:0/3284674713 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4030660 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f013f5b0 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65d7fff640 1 -- 172.21.15.82:0/3284674713 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e40423f0 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.279+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f013fb70 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.280+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f00ccff0 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.281+0000 7f65d7fff640 1 -- 172.21.15.82:0/3284674713 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f65e4037070 con 0x7f65f0143f60 2024-03-14T12:16:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.281+0000 7f65d7fff640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f65dc06bbd0 0x7f65dc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.281+0000 7f65ef7fe640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f65dc06bbd0 0x7f65dc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.281+0000 7f65d7fff640 1 -- 172.21.15.82:0/3284674713 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65e40ace00 con 0x7f65f0143f60 2024-03-14T12:16:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.282+0000 7f65ef7fe640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f65dc06bbd0 0x7f65dc06e090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.286+0000 7f65d7fff640 1 -- 172.21.15.82:0/3284674713 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f65f00ccff0 con 0x7f65f0143f60 2024-03-14T12:16:21.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.458+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65f004f050 con 0x7f65f0143f60 2024-03-14T12:16:21.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.459+0000 7f65d7fff640 1 -- 172.21.15.82:0/3284674713 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f65e407ba10 con 0x7f65f0143f60 2024-03-14T12:16:21.459 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:21.459 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:03.671135+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:00.232652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:00.983002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f65dc06bbd0 msgr2=0x7f65dc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f65dc06bbd0 0x7f65dc06e090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0008040 comp rx=0 tx=0).stop 2024-03-14T12:16:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f0143f60 msgr2=0x7f65f013c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f0143f60 0x7f65f013c6b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e40046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 shutdown_connections 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65f015e670 0x7f65f015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f0143f60 0x7f65f013c6b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f65dc06bbd0 0x7f65dc06e090 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 --2- 172.21.15.82:0/3284674713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f65f0071740 0x7f65f01388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 >> 172.21.15.82:0/3284674713 conn(0x7f65f00b1c70 msgr2=0x7f65f00cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 shutdown_connections 2024-03-14T12:16:21.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.461+0000 7f65f60e9640 1 -- 172.21.15.82:0/3284674713 wait complete. 2024-03-14T12:16:21.472 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T12:16:21.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.573+0000 7f22b122b640 1 Processor -- start 2024-03-14T12:16:21.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.573+0000 7f22b122b640 1 -- start start 2024-03-14T12:16:21.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.574+0000 7f22b122b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac06b420 0x7f22ac0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.574+0000 7f22b122b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac0b1340 con 0x7f22ac0afff0 2024-03-14T12:16:21.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.574+0000 7f22b122b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac0b0480 con 0x7f22ac0b0f60 2024-03-14T12:16:21.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.574+0000 7f22b122b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0b4ee0 con 0x7f22ac06b420 2024-03-14T12:16:21.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.574+0000 7f22ab59e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f22ac0afff0 0x7f22ac0b5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33472/0 (socket says 172.21.15.82:33472) 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.574+0000 7f22ab59e640 1 -- 172.21.15.82:0/545946 learned_addr learned my addr 172.21.15.82:0/545946 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.574+0000 7f22a3fff640 1 --2- 172.21.15.82:0/545946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac06b420 0x7f22ac0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281273105 0 0) 0x7f22ac0b1340 con 0x7f22ac0afff0 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2298003520 con 0x7f22ac0afff0 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402242448 0 0) 0x7f22ac0b0480 con 0x7f22ac0b0f60 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22ac0b1340 con 0x7f22ac0b0f60 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22a3fff640 1 -- 172.21.15.82:0/545946 >> v1:172.21.15.164:6789/0 conn(0x7f22ac0b0f60 legacy=0x7f22ac06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1727985405 0 0) 0x7f2298003520 con 0x7f22ac0afff0 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22a3fff640 1 -- 172.21.15.82:0/545946 >> v1:172.21.15.82:6789/0 conn(0x7f22ac0afff0 legacy=0x7f22ac0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.575+0000 7f22a3fff640 1 -- 172.21.15.82:0/545946 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac0cd030 con 0x7f22ac06b420 2024-03-14T12:16:21.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22a3fff640 1 --2- 172.21.15.82:0/545946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac06b420 0x7f22ac0cc9a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22940097d0 tx=0x7f229402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af5077dadac0eea server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294037070 con 0x7f22ac06b420 2024-03-14T12:16:21.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2294030510 con 0x7f22ac06b420 2024-03-14T12:16:21.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22aa59c640 1 -- 172.21.15.82:0/545946 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a40019f0 con 0x7f22ac06b420 2024-03-14T12:16:21.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22b122b640 1 -- 172.21.15.82:0/545946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac06b420 msgr2=0x7f22ac0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22b122b640 1 --2- 172.21.15.82:0/545946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac06b420 0x7f22ac0cc9a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22940097d0 tx=0x7f229402b800 comp rx=0 tx=0).stop 2024-03-14T12:16:21.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22b122b640 1 -- 172.21.15.82:0/545946 shutdown_connections 2024-03-14T12:16:21.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22b122b640 1 --2- 172.21.15.82:0/545946 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac06b420 0x7f22ac0cc9a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22b122b640 1 -- 172.21.15.82:0/545946 >> 172.21.15.82:0/545946 conn(0x7f22ac072310 msgr2=0x7f22ac072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:21.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22b122b640 1 -- 172.21.15.82:0/545946 shutdown_connections 2024-03-14T12:16:21.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.576+0000 7f22b122b640 1 -- 172.21.15.82:0/545946 wait complete. 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 Processor -- start 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 -- start start 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac06b420 0x7f22ac0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22ac0afff0 0x7f22ac0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac0b0f60 0x7f22ac154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0cd8d0 con 0x7f22ac06b420 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0cd5f0 con 0x7f22ac0afff0 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22b122b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0d09a0 con 0x7f22ac0b0f60 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.578+0000 7f22aad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac06b420 0x7f22ac0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac06b420 0x7f22ac0c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38648/0 (socket says 172.21.15.82:38648) 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 -- 172.21.15.82:0/4134345142 learned_addr learned my addr 172.21.15.82:0/4134345142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22a3fff640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22ac0afff0 0x7f22ac0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22ab59e640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac0b0f60 0x7f22ac154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 -- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac0b0f60 msgr2=0x7f22ac154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac0b0f60 0x7f22ac154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 -- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22ac0afff0 msgr2=0x7f22ac0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22ac0afff0 0x7f22ac0c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 -- 172.21.15.82:0/4134345142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac0c4d60 con 0x7f22ac06b420 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22a3fff640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22ac0afff0 0x7f22ac0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22aad9d640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac06b420 0x7f22ac0c9bf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c430 tx=0x7f229c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22a37fe640 1 -- 172.21.15.82:0/4134345142 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c017070 con 0x7f22ac06b420 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22a37fe640 1 -- 172.21.15.82:0/4134345142 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c0052f0 con 0x7f22ac06b420 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac0c5040 con 0x7f22ac06b420 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22a37fe640 1 -- 172.21.15.82:0/4134345142 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c003830 con 0x7f22ac06b420 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.579+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac154750 con 0x7f22ac06b420 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.580+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac04f050 con 0x7f22ac06b420 2024-03-14T12:16:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.581+0000 7f22a37fe640 1 -- 172.21.15.82:0/4134345142 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f229c01a070 con 0x7f22ac06b420 2024-03-14T12:16:21.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.581+0000 7f22a37fe640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f228c06bbd0 0x7f228c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.581+0000 7f22a37fe640 1 -- 172.21.15.82:0/4134345142 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f229c089f20 con 0x7f22ac06b420 2024-03-14T12:16:21.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.581+0000 7f22a3fff640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f228c06bbd0 0x7f228c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.582+0000 7f22a3fff640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f228c06bbd0 0x7f228c06e090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2294034000 tx=0x7f2294008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.585+0000 7f22a37fe640 1 -- 172.21.15.82:0/4134345142 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f229c058990 con 0x7f22ac06b420 2024-03-14T12:16:21.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.799+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f22ac06cfc0 con 0x7f22ac06b420 2024-03-14T12:16:21.800 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-14T12:16:21.800+0000 7f8354c85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T12:16:21.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.801+0000 7f22a37fe640 1 -- 172.21.15.82:0/4134345142 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f229c058290 con 0x7f22ac06b420 2024-03-14T12:16:21.801 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f228c06bbd0 msgr2=0x7f228c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f228c06bbd0 0x7f228c06e090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2294034000 tx=0x7f2294008030 comp rx=0 tx=0).stop 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac06b420 msgr2=0x7f22ac0c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac06b420 0x7f22ac0c9bf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c430 tx=0x7f229c00c8f0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 shutdown_connections 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f228c06bbd0 0x7f228c06e090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ac0b0f60 0x7f22ac154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22ac0afff0 0x7f22ac0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 --2- 172.21.15.82:0/4134345142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac06b420 0x7f22ac0c9bf0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 >> 172.21.15.82:0/4134345142 conn(0x7f22ac072310 msgr2=0x7f22ac06d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:21.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 shutdown_connections 2024-03-14T12:16:21.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.804+0000 7f22b122b640 1 -- 172.21.15.82:0/4134345142 wait complete. 2024-03-14T12:16:21.806 INFO:tasks.ceph.mds.c.smithi082.stderr: -2449> 2024-03-14T12:16:07.925+0000 7fc3372cd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:16:21.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T12:16:21.833 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:16:21.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb52710a640 1 Processor -- start 2024-03-14T12:16:21.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb52710a640 1 -- start start 2024-03-14T12:16:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb52710a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb52015dff0 0x7fb5200d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb52710a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb5200d12a0 0x7fb5200d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb52710a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5200d25a0 con 0x7fb5200d0c40 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb52710a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5200d2350 con 0x7fb52015dff0 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb52710a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5200d2880 con 0x7fb5200d12a0 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb524ea6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb52015dff0 0x7fb5200d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb524ea6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb52015dff0 0x7fb5200d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38684/0 (socket says 172.21.15.82:38684) 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.920+0000 7fb524ea6640 1 -- 172.21.15.82:0/4244401085 learned_addr learned my addr 172.21.15.82:0/4244401085 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.921+0000 7fb51ffff640 1 --2- 172.21.15.82:0/4244401085 >> v2:172.21.15.164:3300/0 conn(0x7fb5200d12a0 0x7fb5200d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.921+0000 7fb524ea6640 1 -- 172.21.15.82:0/4244401085 >> v2:172.21.15.164:3300/0 conn(0x7fb5200d12a0 msgr2=0x7fb5200d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.921+0000 7fb524ea6640 1 --2- 172.21.15.82:0/4244401085 >> v2:172.21.15.164:3300/0 conn(0x7fb5200d12a0 0x7fb5200d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.921+0000 7fb524ea6640 1 -- 172.21.15.82:0/4244401085 >> v1:172.21.15.164:6789/0 conn(0x7fb5200d0c40 legacy=0x7fb5200ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.921+0000 7fb524ea6640 1 -- 172.21.15.82:0/4244401085 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200d2b60 con 0x7fb52015dff0 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.921+0000 7fb51ffff640 1 --2- 172.21.15.82:0/4244401085 >> v2:172.21.15.164:3300/0 conn(0x7fb5200d12a0 0x7fb5200d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.921+0000 7fb524ea6640 1 --2- 172.21.15.82:0/4244401085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb52015dff0 0x7fb5200d4480 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb5200d3390 tx=0x7fb51402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657cbe9c2b9b627 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb51f7fe640 1 -- 172.21.15.82:0/4244401085 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514035020 con 0x7fb52015dff0 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb51f7fe640 1 -- 172.21.15.82:0/4244401085 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514031870 con 0x7fb52015dff0 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb51f7fe640 1 -- 172.21.15.82:0/4244401085 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514031c40 con 0x7fb52015dff0 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 -- 172.21.15.82:0/4244401085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb52015dff0 msgr2=0x7fb5200d4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 --2- 172.21.15.82:0/4244401085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb52015dff0 0x7fb5200d4480 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb5200d3390 tx=0x7fb51402b910 comp rx=0 tx=0).stop 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 -- 172.21.15.82:0/4244401085 shutdown_connections 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 --2- 172.21.15.82:0/4244401085 >> v2:172.21.15.164:3300/0 conn(0x7fb5200d12a0 0x7fb5200d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 --2- 172.21.15.82:0/4244401085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb52015dff0 0x7fb5200d4480 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 -- 172.21.15.82:0/4244401085 >> 172.21.15.82:0/4244401085 conn(0x7fb52009d0d0 msgr2=0x7fb5200d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 -- 172.21.15.82:0/4244401085 shutdown_connections 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.922+0000 7fb52710a640 1 -- 172.21.15.82:0/4244401085 wait complete. 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.923+0000 7fb52710a640 1 Processor -- start 2024-03-14T12:16:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.923+0000 7fb52710a640 1 -- start start 2024-03-14T12:16:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb52710a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5200d0c40 0x7fb5201f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb52710a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5200d12a0 0x7fb520153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb52710a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb52015dff0 0x7fb5200c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb52710a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015e550 con 0x7fb5200d12a0 2024-03-14T12:16:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb52710a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015e3d0 con 0x7fb52015dff0 2024-03-14T12:16:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb52710a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200cc330 con 0x7fb5200d0c40 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb51ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5200d12a0 0x7fb520153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb51ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5200d12a0 0x7fb520153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38696/0 (socket says 172.21.15.82:38696) 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb51ffff640 1 -- 172.21.15.82:0/717157152 learned_addr learned my addr 172.21.15.82:0/717157152 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb5256a7640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb52015dff0 0x7fb5200c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.924+0000 7fb524ea6640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5200d0c40 0x7fb5201f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51ffff640 1 -- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5200d0c40 msgr2=0x7fb5201f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51ffff640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5200d0c40 0x7fb5201f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51ffff640 1 -- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb52015dff0 msgr2=0x7fb5200c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51ffff640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb52015dff0 0x7fb5200c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51ffff640 1 -- 172.21.15.82:0/717157152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200ca4d0 con 0x7fb5200d12a0 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51ffff640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5200d12a0 0x7fb520153200 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c7f0 tx=0x7fb50c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51d7fa640 1 -- 172.21.15.82:0/717157152 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c015070 con 0x7fb5200d12a0 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51d7fa640 1 -- 172.21.15.82:0/717157152 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c007a30 con 0x7fb5200d12a0 2024-03-14T12:16:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb51d7fa640 1 -- 172.21.15.82:0/717157152 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c007e00 con 0x7fb5200d12a0 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5200ca7b0 con 0x7fb5200d12a0 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.925+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5201f7420 con 0x7fb5200d12a0 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.926+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52004f050 con 0x7fb5200d12a0 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.926+0000 7fb51d7fa640 1 -- 172.21.15.82:0/717157152 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb50c018070 con 0x7fb5200d12a0 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.927+0000 7fb51d7fa640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb50406bbd0 0x7fb50406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.927+0000 7fb51d7fa640 1 -- 172.21.15.82:0/717157152 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb50c08a940 con 0x7fb5200d12a0 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.927+0000 7fb524ea6640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb50406bbd0 0x7fb50406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.927+0000 7fb524ea6640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb50406bbd0 0x7fb50406e090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb514036000 tx=0x7fb51403c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:21.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:21.931+0000 7fb51d7fa640 1 -- 172.21.15.82:0/717157152 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb50c058e10 con 0x7fb5200d12a0 2024-03-14T12:16:21.982 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:16:21.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:16:22.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.144+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb52015d580 con 0x7fb5200d12a0 2024-03-14T12:16:22.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.802+0000 7fb51d7fa640 1 -- 172.21.15.82:0/717157152 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb50c05ceb0 con 0x7fb5200d12a0 2024-03-14T12:16:22.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb50406bbd0 msgr2=0x7fb50406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb50406bbd0 0x7fb50406e090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb514036000 tx=0x7fb51403c040 comp rx=0 tx=0).stop 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5200d12a0 msgr2=0x7fb520153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5200d12a0 0x7fb520153200 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00c7f0 tx=0x7fb50c00ccb0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 shutdown_connections 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb52015dff0 0x7fb5200c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5200d12a0 0x7fb520153200 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb50406bbd0 0x7fb50406e090 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 --2- 172.21.15.82:0/717157152 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5200d0c40 0x7fb5201f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 >> 172.21.15.82:0/717157152 conn(0x7fb52009d0d0 msgr2=0x7fb52006b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 shutdown_connections 2024-03-14T12:16:22.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.806+0000 7fb52710a640 1 -- 172.21.15.82:0/717157152 wait complete. 2024-03-14T12:16:22.820 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T12:16:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.929+0000 7f0f16c81640 1 Processor -- start 2024-03-14T12:16:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.929+0000 7f0f16c81640 1 -- start start 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.929+0000 7f0f16c81640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0f1015df30 0x7f0f1006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.929+0000 7f0f16c81640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f10072560 con 0x7f0f1006f7f0 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.929+0000 7f0f16c81640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f100d1590 con 0x7f0f100b39c0 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.929+0000 7f0f16c81640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f10161b70 con 0x7f0f1015df30 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f14a1d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0f1015df30 0x7f0f1006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f0ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0f1006f7f0 0x7f0f10071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33490/0 (socket says 172.21.15.82:33490) 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f14a1d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0f1015df30 0x7f0f1006ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38710/0 (socket says 172.21.15.82:38710) 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f0ffff640 1 -- 172.21.15.82:0/141306330 learned_addr learned my addr 172.21.15.82:0/141306330 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f14a1d640 1 -- 172.21.15.82:0/141306330 >> v1:172.21.15.164:6789/0 conn(0x7f0f100b39c0 legacy=0x7f0f10071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f14a1d640 1 -- 172.21.15.82:0/141306330 >> v1:172.21.15.82:6789/0 conn(0x7f0f1006f7f0 legacy=0x7f0f10071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f0f7fe640 1 -- 172.21.15.82:0/141306330 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057100286 0 0) 0x7f0f10072560 con 0x7f0f1006f7f0 2024-03-14T12:16:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.930+0000 7f0f14a1d640 1 -- 172.21.15.82:0/141306330 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f10161e50 con 0x7f0f1015df30 2024-03-14T12:16:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f14a1d640 1 --2- 172.21.15.82:0/141306330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1015df30 0x7f0f1006ad10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0f000097a0 tx=0x7f0f0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2060b394d7d98aef server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f0f7fe640 1 -- 172.21.15.82:0/141306330 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00035020 con 0x7f0f1015df30 2024-03-14T12:16:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f0f7fe640 1 -- 172.21.15.82:0/141306330 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00031830 con 0x7f0f1015df30 2024-03-14T12:16:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f0f7fe640 1 -- 172.21.15.82:0/141306330 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00031c00 con 0x7f0f1015df30 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f16c81640 1 -- 172.21.15.82:0/141306330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1015df30 msgr2=0x7f0f1006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f16c81640 1 --2- 172.21.15.82:0/141306330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1015df30 0x7f0f1006ad10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0f000097a0 tx=0x7f0f0002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f16c81640 1 -- 172.21.15.82:0/141306330 shutdown_connections 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f16c81640 1 --2- 172.21.15.82:0/141306330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1015df30 0x7f0f1006ad10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f16c81640 1 -- 172.21.15.82:0/141306330 >> 172.21.15.82:0/141306330 conn(0x7f0f100d09d0 msgr2=0x7f0f100d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f16c81640 1 -- 172.21.15.82:0/141306330 shutdown_connections 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.931+0000 7f0f16c81640 1 -- 172.21.15.82:0/141306330 wait complete. 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.932+0000 7f0f16c81640 1 Processor -- start 2024-03-14T12:16:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f16c81640 1 -- start start 2024-03-14T12:16:22.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f16c81640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f1006f7f0 0x7f0f101592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:22.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f16c81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f100b39c0 0x7f0f1015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f16c81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f1015df30 0x7f0f1015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f16c81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f101627d0 con 0x7f0f100b39c0 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f16c81640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f10162650 con 0x7f0f1006f7f0 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f16c81640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f10162950 con 0x7f0f1015df30 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f0ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f100b39c0 0x7f0f1015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f0ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f100b39c0 0x7f0f1015cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38712/0 (socket says 172.21.15.82:38712) 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.933+0000 7f0f0ffff640 1 -- 172.21.15.82:0/2689170016 learned_addr learned my addr 172.21.15.82:0/2689170016 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0ffff640 1 -- 172.21.15.82:0/2689170016 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f1015df30 msgr2=0x7f0f1015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0ffff640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f1015df30 0x7f0f1015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f14a1d640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f1006f7f0 0x7f0f101592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0ffff640 1 -- 172.21.15.82:0/2689170016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f1006f7f0 msgr2=0x7f0f101592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0ffff640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f1006f7f0 0x7f0f101592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0ffff640 1 -- 172.21.15.82:0/2689170016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1015b0e0 con 0x7f0f100b39c0 2024-03-14T12:16:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0ffff640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f100b39c0 0x7f0f1015cd10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c4a0 tx=0x7f0f0400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:22.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0d7fa640 1 -- 172.21.15.82:0/2689170016 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04017070 con 0x7f0f100b39c0 2024-03-14T12:16:22.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0d7fa640 1 -- 172.21.15.82:0/2689170016 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f04005ce0 con 0x7f0f100b39c0 2024-03-14T12:16:22.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f0d7fa640 1 -- 172.21.15.82:0/2689170016 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f040072b0 con 0x7f0f100b39c0 2024-03-14T12:16:22.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1015b3c0 con 0x7f0f100b39c0 2024-03-14T12:16:22.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.934+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1015b980 con 0x7f0f100b39c0 2024-03-14T12:16:22.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.935+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1004f050 con 0x7f0f100b39c0 2024-03-14T12:16:22.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.935+0000 7f0f0d7fa640 1 -- 172.21.15.82:0/2689170016 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0f0401a070 con 0x7f0f100b39c0 2024-03-14T12:16:22.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.936+0000 7f0f0d7fa640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0efc06bbd0 0x7f0efc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:22.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.936+0000 7f0f0d7fa640 1 -- 172.21.15.82:0/2689170016 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0f04089e20 con 0x7f0f100b39c0 2024-03-14T12:16:22.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.936+0000 7f0f14a1d640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0efc06bbd0 0x7f0efc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:22.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.936+0000 7f0f14a1d640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0efc06bbd0 0x7f0efc06e090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0f00036000 tx=0x7f0f0003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:22.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:22.940+0000 7f0f0d7fa640 1 -- 172.21.15.82:0/2689170016 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0f04058940 con 0x7f0f100b39c0 2024-03-14T12:16:23.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.111+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f100cfad0 con 0x7f0f100b39c0 2024-03-14T12:16:23.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.810+0000 7f0f0d7fa640 1 -- 172.21.15.82:0/2689170016 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0f04058240 con 0x7f0f100b39c0 2024-03-14T12:16:23.810 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0efc06bbd0 msgr2=0x7f0efc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0efc06bbd0 0x7f0efc06e090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0f00036000 tx=0x7f0f0003c040 comp rx=0 tx=0).stop 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f100b39c0 msgr2=0x7f0f1015cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f100b39c0 0x7f0f1015cd10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c4a0 tx=0x7f0f0400c960 comp rx=0 tx=0).stop 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 shutdown_connections 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0efc06bbd0 0x7f0efc06e090 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f1015df30 0x7f0f1015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f100b39c0 0x7f0f1015cd10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 --2- 172.21.15.82:0/2689170016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f1006f7f0 0x7f0f101592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 >> 172.21.15.82:0/2689170016 conn(0x7f0f100d09d0 msgr2=0x7f0f100ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 shutdown_connections 2024-03-14T12:16:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.813+0000 7f0f16c81640 1 -- 172.21.15.82:0/2689170016 wait complete. 2024-03-14T12:16:23.827 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T12:16:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.936+0000 7f6e65fb3640 1 Processor -- start 2024-03-14T12:16:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.936+0000 7f6e65fb3640 1 -- start start 2024-03-14T12:16:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.937+0000 7f6e65fb3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 0x7f6e6006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.937+0000 7f6e65fb3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e600d08a0 con 0x7f6e60161a90 2024-03-14T12:16:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.937+0000 7f6e65fb3640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e600d0b80 con 0x7f6e600b0bb0 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.937+0000 7f6e65fb3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600d0e60 con 0x7f6e60162580 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.937+0000 7f6e5f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6e60161a90 0x7f6e60161e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33492/0 (socket says 172.21.15.82:33492) 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.937+0000 7f6e5f7fe640 1 -- 172.21.15.82:0/2803588563 learned_addr learned my addr 172.21.15.82:0/2803588563 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.937+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141584449 0 0) 0x7f6e600d08a0 con 0x7f6e60161a90 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e48003520 con 0x7f6e60161a90 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5ffff640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 0x7f6e6006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3549424796 0 0) 0x7f6e48003520 con 0x7f6e60161a90 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 msgr2=0x7f6e6006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 0x7f6e6006cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 >> v1:172.21.15.164:6789/0 conn(0x7f6e600b0bb0 legacy=0x7f6e6006c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6006bc00 con 0x7f6e60161a90 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6e50001670 con 0x7f6e60161a90 2024-03-14T12:16:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 >> v1:172.21.15.82:6789/0 conn(0x7f6e60161a90 legacy=0x7f6e60161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 0x7f6e480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e48004010 0x7f6e48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004d40 0x7f6e480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600d0e60 con 0x7f6e48004010 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4800bb80 con 0x7f6e48004d40 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4800bd80 con 0x7f6e60162580 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5effd640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e48004010 0x7f6e48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5f7fe640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 0x7f6e480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5ffff640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004d40 0x7f6e480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5effd640 1 -- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 msgr2=0x7f6e480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5effd640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e60162580 0x7f6e480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5effd640 1 -- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004d40 msgr2=0x7f6e480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5effd640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004d40 0x7f6e480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.938+0000 7f6e5effd640 1 -- 172.21.15.82:0/2803588563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48003520 con 0x7f6e48004010 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.939+0000 7f6e5effd640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e48004010 0x7f6e48004800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6e54001750 tx=0x7f6e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8501abfecd81470a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.939+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54001370 con 0x7f6e48004010 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.939+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e540052d0 con 0x7f6e48004010 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.939+0000 7f6e5e7fc640 1 -- 172.21.15.82:0/2803588563 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54003830 con 0x7f6e48004010 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.940+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2803588563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e48004010 msgr2=0x7f6e48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.940+0000 7f6e65fb3640 1 --2- 172.21.15.82:0/2803588563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e48004010 0x7f6e48004800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6e54001750 tx=0x7f6e48005830 comp rx=0 tx=0).stop 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.940+0000 7f6e5ffff640 1 -- 172.21.15.82:0/2803588563 reap_dead start 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.940+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2803588563 shutdown_connections 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.940+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2803588563 >> 172.21.15.82:0/2803588563 conn(0x7f6e600b36a0 msgr2=0x7f6e600b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.940+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2803588563 shutdown_connections 2024-03-14T12:16:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.940+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2803588563 wait complete. 2024-03-14T12:16:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.941+0000 7f6e65fb3640 1 Processor -- start 2024-03-14T12:16:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.941+0000 7f6e65fb3640 1 -- start start 2024-03-14T12:16:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e65fb3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004010 0x7f6e60157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e65fb3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e600b6cb0 0x7f6e6015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e65fb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e600afc70 0x7f6e6015bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e65fb3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6006cb10 con 0x7f6e600afc70 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e65fb3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600cfb40 con 0x7f6e48004010 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e65fb3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6006cc90 con 0x7f6e600b6cb0 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e600afc70 0x7f6e6015bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e600afc70 0x7f6e6015bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38728/0 (socket says 172.21.15.82:38728) 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 -- 172.21.15.82:0/2488263440 learned_addr learned my addr 172.21.15.82:0/2488263440 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 -- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e600b6cb0 msgr2=0x7f6e6015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5effd640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e600b6cb0 0x7f6e6015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5f7fe640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004010 0x7f6e60157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e600b6cb0 0x7f6e6015d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 -- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004010 msgr2=0x7f6e60157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004010 0x7f6e60157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 -- 172.21.15.82:0/2488263440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e600b7440 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e5ffff640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e600afc70 0x7f6e6015bad0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6e600cc8d0 tx=0x7f6e58007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.942+0000 7f6e47fff640 1 -- 172.21.15.82:0/2488263440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e580046a0 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.943+0000 7f6e47fff640 1 -- 172.21.15.82:0/2488263440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5801a070 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.943+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e600b7160 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.943+0000 7f6e47fff640 1 -- 172.21.15.82:0/2488263440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58015640 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.943+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6015ca10 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.944+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6004f050 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.944+0000 7f6e47fff640 1 -- 172.21.15.82:0/2488263440 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6e580157e0 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.944+0000 7f6e47fff640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e4c06bc00 0x7f6e4c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.944+0000 7f6e47fff640 1 -- 172.21.15.82:0/2488263440 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f6e5808ae90 con 0x7f6e600afc70 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.944+0000 7f6e5f7fe640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e4c06bc00 0x7f6e4c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.945+0000 7f6e5f7fe640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e4c06bc00 0x7f6e4c06e0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6e60162330 tx=0x7f6e50009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:23.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:23.949+0000 7f6e47fff640 1 -- 172.21.15.82:0/2488263440 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6e58059b70 con 0x7f6e600afc70 2024-03-14T12:16:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.124+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e60072b90 con 0x7f6e600afc70 2024-03-14T12:16:24.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.811+0000 7f6e47fff640 1 -- 172.21.15.82:0/2488263440 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6e58059470 con 0x7f6e600afc70 2024-03-14T12:16:24.812 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.814+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e4c06bc00 msgr2=0x7f6e4c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.814+0000 7f6e65fb3640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e4c06bc00 0x7f6e4c06e0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6e60162330 tx=0x7f6e50009040 comp rx=0 tx=0).stop 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.814+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e600afc70 msgr2=0x7f6e6015bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.814+0000 7f6e65fb3640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e600afc70 0x7f6e6015bad0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6e600cc8d0 tx=0x7f6e58007700 comp rx=0 tx=0).stop 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 shutdown_connections 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e600afc70 0x7f6e6015bad0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6e4c06bc00 0x7f6e4c06e0c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e600b6cb0 0x7f6e6015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 --2- 172.21.15.82:0/2488263440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e48004010 0x7f6e60157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 >> 172.21.15.82:0/2488263440 conn(0x7f6e600b36a0 msgr2=0x7f6e6015b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:24.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 shutdown_connections 2024-03-14T12:16:24.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.815+0000 7f6e65fb3640 1 -- 172.21.15.82:0/2488263440 wait complete. 2024-03-14T12:16:24.829 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-14T12:16:24.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.937+0000 7fe2a8812640 1 Processor -- start 2024-03-14T12:16:24.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.937+0000 7fe2a8812640 1 -- start start 2024-03-14T12:16:24.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a8812640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2a40cf180 0x7fe2a415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:24.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a8812640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2a415e3a0 0x7fe2a40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:24.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a8812640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a40b2f70 con 0x7fe2a41549d0 2024-03-14T12:16:24.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a8812640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a4154db0 con 0x7fe2a40cf180 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a8812640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a406d610 con 0x7fe2a415e3a0 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2a40cf180 0x7fe2a415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe2a41549d0 0x7fe2a406daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33502/0 (socket says 172.21.15.82:33502) 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a259c640 1 -- 172.21.15.82:0/2867474913 learned_addr learned my addr 172.21.15.82:0/2867474913 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a2d9d640 1 --2- 172.21.15.82:0/2867474913 >> v2:172.21.15.164:3300/0 conn(0x7fe2a415e3a0 0x7fe2a40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a1d9b640 1 -- 172.21.15.82:0/2867474913 >> v2:172.21.15.164:3300/0 conn(0x7fe2a415e3a0 msgr2=0x7fe2a40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a1d9b640 1 --2- 172.21.15.82:0/2867474913 >> v2:172.21.15.164:3300/0 conn(0x7fe2a415e3a0 0x7fe2a40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a1d9b640 1 -- 172.21.15.82:0/2867474913 >> v1:172.21.15.82:6789/0 conn(0x7fe2a41549d0 legacy=0x7fe2a406daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a1d9b640 1 -- 172.21.15.82:0/2867474913 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a40b29b0 con 0x7fe2a40cf180 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a159a640 1 -- 172.21.15.82:0/2867474913 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803684577 0 0) 0x7fe2a40b2f70 con 0x7fe2a41549d0 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.938+0000 7fe2a2d9d640 1 --2- 172.21.15.82:0/2867474913 >> v2:172.21.15.164:3300/0 conn(0x7fe2a415e3a0 0x7fe2a40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.939+0000 7fe2a1d9b640 1 --2- 172.21.15.82:0/2867474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a40cf180 0x7fe2a415de60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40d0650 tx=0x7fe29802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658097fa24f28ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.939+0000 7fe2a159a640 1 -- 172.21.15.82:0/2867474913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298034070 con 0x7fe2a40cf180 2024-03-14T12:16:24.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.939+0000 7fe2a159a640 1 -- 172.21.15.82:0/2867474913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe298030610 con 0x7fe2a40cf180 2024-03-14T12:16:24.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.939+0000 7fe2a159a640 1 -- 172.21.15.82:0/2867474913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2980309e0 con 0x7fe2a40cf180 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 -- 172.21.15.82:0/2867474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a40cf180 msgr2=0x7fe2a415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 --2- 172.21.15.82:0/2867474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a40cf180 0x7fe2a415de60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40d0650 tx=0x7fe29802b960 comp rx=0 tx=0).stop 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 -- 172.21.15.82:0/2867474913 shutdown_connections 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 --2- 172.21.15.82:0/2867474913 >> v2:172.21.15.164:3300/0 conn(0x7fe2a415e3a0 0x7fe2a40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 --2- 172.21.15.82:0/2867474913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a40cf180 0x7fe2a415de60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 -- 172.21.15.82:0/2867474913 >> 172.21.15.82:0/2867474913 conn(0x7fe2a406aef0 msgr2=0x7fe2a406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 -- 172.21.15.82:0/2867474913 shutdown_connections 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.940+0000 7fe2a8812640 1 -- 172.21.15.82:0/2867474913 wait complete. 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.941+0000 7fe2a8812640 1 Processor -- start 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.941+0000 7fe2a8812640 1 -- start start 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a8812640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a40cf180 0x7fe2a4144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a8812640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a41549d0 0x7fe2a4159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a8812640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a415e3a0 0x7fe2a4159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a8812640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a406cfc0 con 0x7fe2a41549d0 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a8812640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a415e7b0 con 0x7fe2a40cf180 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a8812640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a406d2a0 con 0x7fe2a415e3a0 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a41549d0 0x7fe2a4159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a41549d0 0x7fe2a4159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38748/0 (socket says 172.21.15.82:38748) 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 -- 172.21.15.82:0/1183492964 learned_addr learned my addr 172.21.15.82:0/1183492964 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a259c640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a40cf180 0x7fe2a4144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a2d9d640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a415e3a0 0x7fe2a4159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 -- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a415e3a0 msgr2=0x7fe2a4159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a415e3a0 0x7fe2a4159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 -- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a40cf180 msgr2=0x7fe2a4144440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a40cf180 0x7fe2a4144440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 -- 172.21.15.82:0/1183492964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4147ef0 con 0x7fe2a41549d0 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a2d9d640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a415e3a0 0x7fe2a4159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe2a1d9b640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a41549d0 0x7fe2a4159340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe29802bf70 tx=0x7fe2980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.942+0000 7fe292ffd640 1 -- 172.21.15.82:0/1183492964 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298047070 con 0x7fe2a41549d0 2024-03-14T12:16:24.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.943+0000 7fe292ffd640 1 -- 172.21.15.82:0/1183492964 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe298039a10 con 0x7fe2a41549d0 2024-03-14T12:16:24.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.943+0000 7fe292ffd640 1 -- 172.21.15.82:0/1183492964 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298034040 con 0x7fe2a41549d0 2024-03-14T12:16:24.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.943+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a4146e50 con 0x7fe2a41549d0 2024-03-14T12:16:24.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.943+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a4147410 con 0x7fe2a41549d0 2024-03-14T12:16:24.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.944+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a404f050 con 0x7fe2a41549d0 2024-03-14T12:16:24.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.944+0000 7fe292ffd640 1 -- 172.21.15.82:0/1183492964 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe29804a070 con 0x7fe2a41549d0 2024-03-14T12:16:24.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.944+0000 7fe292ffd640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe28406bc00 0x7fe28406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:24.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.945+0000 7fe292ffd640 1 -- 172.21.15.82:0/1183492964 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe2980ac1b0 con 0x7fe2a41549d0 2024-03-14T12:16:24.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.945+0000 7fe2a259c640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe28406bc00 0x7fe28406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:24.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.945+0000 7fe2a259c640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe28406bc00 0x7fe28406e0c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe294000f60 tx=0x7fe294007b70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:24.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:24.949+0000 7fe292ffd640 1 -- 172.21.15.82:0/1183492964 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe29807b150 con 0x7fe2a41549d0 2024-03-14T12:16:25.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.121+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe2a406cad0 con 0x7fe2a41549d0 2024-03-14T12:16:25.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.124+0000 7fe292ffd640 1 -- 172.21.15.82:0/1183492964 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe29807aa50 con 0x7fe2a41549d0 2024-03-14T12:16:25.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe28406bc00 msgr2=0x7fe28406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe28406bc00 0x7fe28406e0c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe294000f60 tx=0x7fe294007b70 comp rx=0 tx=0).stop 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a41549d0 msgr2=0x7fe2a4159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a41549d0 0x7fe2a4159340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe29802bf70 tx=0x7fe2980046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 shutdown_connections 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe28406bc00 0x7fe28406e0c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a415e3a0 0x7fe2a4159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a41549d0 0x7fe2a4159340 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 --2- 172.21.15.82:0/1183492964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a40cf180 0x7fe2a4144440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 >> 172.21.15.82:0/1183492964 conn(0x7fe2a406aef0 msgr2=0x7fe2a4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 shutdown_connections 2024-03-14T12:16:25.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.126+0000 7fe2a8812640 1 -- 172.21.15.82:0/1183492964 wait complete. 2024-03-14T12:16:25.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_connect' 2024-03-14T12:16:25.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.236+0000 7fd985832640 1 Processor -- start 2024-03-14T12:16:25.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.236+0000 7fd985832640 1 -- start start 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd985832640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd980071dc0 0x7fd9800b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd985832640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9800b53f0 con 0x7fd980158750 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd985832640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd980158b30 con 0x7fd98006efe0 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd985832640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9800b4e30 con 0x7fd980071dc0 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd97effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd980158750 0x7fd98015bba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33516/0 (socket says 172.21.15.82:33516) 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd97effd640 1 -- 172.21.15.82:0/1669446746 learned_addr learned my addr 172.21.15.82:0/1669446746 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd97f7fe640 1 --2- 172.21.15.82:0/1669446746 >> v2:172.21.15.82:3300/0 conn(0x7fd980071dc0 0x7fd9800b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd97f7fe640 1 -- 172.21.15.82:0/1669446746 >> v1:172.21.15.164:6789/0 conn(0x7fd98006efe0 legacy=0x7fd9800716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd97f7fe640 1 -- 172.21.15.82:0/1669446746 >> v1:172.21.15.82:6789/0 conn(0x7fd980158750 legacy=0x7fd98015bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.237+0000 7fd97f7fe640 1 -- 172.21.15.82:0/1669446746 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9800b5110 con 0x7fd980071dc0 2024-03-14T12:16:25.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.238+0000 7fd97f7fe640 1 --2- 172.21.15.82:0/1669446746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd980071dc0 0x7fd9800b47e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd9780097a0 tx=0x7fd97802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4752aeefed96bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.238+0000 7fd97dffb640 1 -- 172.21.15.82:0/1669446746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978034070 con 0x7fd980071dc0 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.238+0000 7fd97dffb640 1 -- 172.21.15.82:0/1669446746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd978030610 con 0x7fd980071dc0 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.238+0000 7fd97dffb640 1 -- 172.21.15.82:0/1669446746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780309e0 con 0x7fd980071dc0 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.239+0000 7fd985832640 1 -- 172.21.15.82:0/1669446746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd980071dc0 msgr2=0x7fd9800b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.239+0000 7fd985832640 1 --2- 172.21.15.82:0/1669446746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd980071dc0 0x7fd9800b47e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd9780097a0 tx=0x7fd97802b960 comp rx=0 tx=0).stop 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.239+0000 7fd985832640 1 -- 172.21.15.82:0/1669446746 shutdown_connections 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.239+0000 7fd985832640 1 --2- 172.21.15.82:0/1669446746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd980071dc0 0x7fd9800b47e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.239+0000 7fd985832640 1 -- 172.21.15.82:0/1669446746 >> 172.21.15.82:0/1669446746 conn(0x7fd9800d0a20 msgr2=0x7fd9800d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.239+0000 7fd985832640 1 -- 172.21.15.82:0/1669446746 shutdown_connections 2024-03-14T12:16:25.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.239+0000 7fd985832640 1 -- 172.21.15.82:0/1669446746 wait complete. 2024-03-14T12:16:25.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.240+0000 7fd985832640 1 Processor -- start 2024-03-14T12:16:25.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.240+0000 7fd985832640 1 -- start start 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd985832640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd98006efe0 0x7fd9800cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd985832640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd980071dc0 0x7fd980141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd985832640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd980158750 0x7fd980158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd985832640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980165750 con 0x7fd98006efe0 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd985832640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98006dab0 con 0x7fd980071dc0 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd985832640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9801658d0 con 0x7fd980158750 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd98006efe0 0x7fd9800cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd98006efe0 0x7fd9800cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38766/0 (socket says 172.21.15.82:38766) 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 -- 172.21.15.82:0/1761592202 learned_addr learned my addr 172.21.15.82:0/1761592202 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97e7fc640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd980071dc0 0x7fd980141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97f7fe640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd980158750 0x7fd980158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 -- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd980158750 msgr2=0x7fd980158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd980158750 0x7fd980158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 -- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd980071dc0 msgr2=0x7fd980141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd980071dc0 0x7fd980141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 -- 172.21.15.82:0/1761592202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980159260 con 0x7fd98006efe0 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.241+0000 7fd97effd640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd98006efe0 0x7fd9800cc400 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd970002230 tx=0x7fd97000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.242+0000 7fd967fff640 1 -- 172.21.15.82:0/1761592202 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970017070 con 0x7fd98006efe0 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.242+0000 7fd967fff640 1 -- 172.21.15.82:0/1761592202 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97000ad40 con 0x7fd98006efe0 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.242+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd980159500 con 0x7fd98006efe0 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.242+0000 7fd967fff640 1 -- 172.21.15.82:0/1761592202 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970012620 con 0x7fd98006efe0 2024-03-14T12:16:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.242+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd980149ab0 con 0x7fd98006efe0 2024-03-14T12:16:25.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.243+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98004f050 con 0x7fd98006efe0 2024-03-14T12:16:25.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.243+0000 7fd967fff640 1 -- 172.21.15.82:0/1761592202 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd97001a070 con 0x7fd98006efe0 2024-03-14T12:16:25.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.243+0000 7fd967fff640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd96c06bbd0 0x7fd96c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.244+0000 7fd967fff640 1 -- 172.21.15.82:0/1761592202 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd97008a4f0 con 0x7fd98006efe0 2024-03-14T12:16:25.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.244+0000 7fd97e7fc640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd96c06bbd0 0x7fd96c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.244+0000 7fd97e7fc640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd96c06bbd0 0x7fd96c06e090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9801488a0 tx=0x7fd974009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:25.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.248+0000 7fd967fff640 1 -- 172.21.15.82:0/1761592202 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd970058c90 con 0x7fd98006efe0 2024-03-14T12:16:25.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.485+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_connect"]} v 0) v1 -- 0x7fd9800b3b50 con 0x7fd98006efe0 2024-03-14T12:16:25.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.820+0000 7fd967fff640 1 -- 172.21.15.82:0/1761592202 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_connect"]}]=0 v147) v1 ==== 182+0+0 (secure 0 0 0) 0x7fd97005cd30 con 0x7fd98006efe0 2024-03-14T12:16:25.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.822+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd96c06bbd0 msgr2=0x7fd96c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.822+0000 7fd985832640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd96c06bbd0 0x7fd96c06e090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9801488a0 tx=0x7fd974009040 comp rx=0 tx=0).stop 2024-03-14T12:16:25.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.822+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd98006efe0 msgr2=0x7fd9800cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.822+0000 7fd985832640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd98006efe0 0x7fd9800cc400 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd970002230 tx=0x7fd97000a430 comp rx=0 tx=0).stop 2024-03-14T12:16:25.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.822+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 shutdown_connections 2024-03-14T12:16:25.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.823+0000 7fd985832640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd96c06bbd0 0x7fd96c06e090 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.823+0000 7fd985832640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd980158750 0x7fd980158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.823+0000 7fd985832640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd980071dc0 0x7fd980141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.823+0000 7fd985832640 1 --2- 172.21.15.82:0/1761592202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd98006efe0 0x7fd9800cc400 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.823+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 >> 172.21.15.82:0/1761592202 conn(0x7fd9800d0a20 msgr2=0x7fd9800cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:25.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.823+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 shutdown_connections 2024-03-14T12:16:25.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.823+0000 7fd985832640 1 -- 172.21.15.82:0/1761592202 wait complete. 2024-03-14T12:16:25.834 INFO:tasks.cephfs_test_runner:That the per-client labelled metrics are generated during client connect ... ok 2024-03-14T12:16:25.835 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters_on_client_disconnect (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled metrics are unavailable during client disconnect 2024-03-14T12:16:25.835 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters_on_client_disconnect (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-14T12:16:25.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.935+0000 7f6b38b59640 1 Processor -- start 2024-03-14T12:16:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.935+0000 7f6b38b59640 1 -- start start 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b38b59640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6b341551d0 0x7f6b340b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b38b59640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6b340b5bb0 0x7f6b34161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b38b59640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340b60f0 con 0x7f6b34161fc0 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b38b59640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b34070340 con 0x7f6b341551d0 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b38b59640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b340b6d00 con 0x7f6b340b5bb0 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b3259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6b341551d0 0x7f6b340b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b3259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6b341551d0 0x7f6b340b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38768/0 (socket says 172.21.15.82:38768) 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b3259c640 1 -- 172.21.15.82:0/1271817892 learned_addr learned my addr 172.21.15.82:0/1271817892 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.936+0000 7f6b31d9b640 1 --2- 172.21.15.82:0/1271817892 >> v2:172.21.15.164:3300/0 conn(0x7f6b340b5bb0 0x7f6b34161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3259c640 1 -- 172.21.15.82:0/1271817892 >> v2:172.21.15.164:3300/0 conn(0x7f6b340b5bb0 msgr2=0x7f6b34161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3259c640 1 --2- 172.21.15.82:0/1271817892 >> v2:172.21.15.164:3300/0 conn(0x7f6b340b5bb0 0x7f6b34161a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3259c640 1 -- 172.21.15.82:0/1271817892 >> v1:172.21.15.164:6789/0 conn(0x7f6b34161fc0 legacy=0x7f6b3415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3259c640 1 -- 172.21.15.82:0/1271817892 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340b63d0 con 0x7f6b341551d0 2024-03-14T12:16:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b31d9b640 1 --2- 172.21.15.82:0/1271817892 >> v2:172.21.15.164:3300/0 conn(0x7f6b340b5bb0 0x7f6b34161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3259c640 1 --2- 172.21.15.82:0/1271817892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b341551d0 0x7f6b340b5670 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6b34068a10 tx=0x7f6b2402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfde79a38e918b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3159a640 1 -- 172.21.15.82:0/1271817892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2402cb20 con 0x7f6b341551d0 2024-03-14T12:16:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3159a640 1 -- 172.21.15.82:0/1271817892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2402ccc0 con 0x7f6b341551d0 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.937+0000 7f6b3159a640 1 -- 172.21.15.82:0/1271817892 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24034610 con 0x7f6b341551d0 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 -- 172.21.15.82:0/1271817892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b341551d0 msgr2=0x7f6b340b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1271817892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b341551d0 0x7f6b340b5670 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6b34068a10 tx=0x7f6b2402bf80 comp rx=0 tx=0).stop 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 -- 172.21.15.82:0/1271817892 shutdown_connections 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1271817892 >> v2:172.21.15.164:3300/0 conn(0x7f6b340b5bb0 0x7f6b34161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1271817892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b341551d0 0x7f6b340b5670 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 -- 172.21.15.82:0/1271817892 >> 172.21.15.82:0/1271817892 conn(0x7f6b34072ee0 msgr2=0x7f6b3409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 -- 172.21.15.82:0/1271817892 shutdown_connections 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.938+0000 7f6b38b59640 1 -- 172.21.15.82:0/1271817892 wait complete. 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.939+0000 7f6b38b59640 1 Processor -- start 2024-03-14T12:16:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.939+0000 7f6b38b59640 1 -- start start 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b38b59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b340b5bb0 0x7f6b3414a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b38b59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b341551d0 0x7f6b3414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b38b59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b34161fc0 0x7f6b34150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b38b59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3415eba0 con 0x7f6b34161fc0 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b38b59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340b6990 con 0x7f6b341551d0 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b38b59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3415ed20 con 0x7f6b340b5bb0 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b34161fc0 0x7f6b34150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b34161fc0 0x7f6b34150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38780/0 (socket says 172.21.15.82:38780) 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 -- 172.21.15.82:0/1029919120 learned_addr learned my addr 172.21.15.82:0/1029919120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b31d9b640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b341551d0 0x7f6b3414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b3259c640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b340b5bb0 0x7f6b3414a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 -- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b340b5bb0 msgr2=0x7f6b3414a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b340b5bb0 0x7f6b3414a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 -- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b341551d0 msgr2=0x7f6b3414fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b341551d0 0x7f6b3414fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b32d9d640 1 -- 172.21.15.82:0/1029919120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3414ef40 con 0x7f6b34161fc0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.940+0000 7f6b3259c640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b340b5bb0 0x7f6b3414a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.941+0000 7f6b32d9d640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b34161fc0 0x7f6b34150200 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c390 tx=0x7f6b2c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.941+0000 7f6b22ffd640 1 -- 172.21.15.82:0/1029919120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c017070 con 0x7f6b34161fc0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.941+0000 7f6b22ffd640 1 -- 172.21.15.82:0/1029919120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c0052d0 con 0x7f6b34161fc0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.941+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3414aa00 con 0x7f6b34161fc0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.941+0000 7f6b22ffd640 1 -- 172.21.15.82:0/1029919120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c003810 con 0x7f6b34161fc0 2024-03-14T12:16:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.941+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3414afc0 con 0x7f6b34161fc0 2024-03-14T12:16:25.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.942+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3415f0b0 con 0x7f6b34161fc0 2024-03-14T12:16:25.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.942+0000 7f6b22ffd640 1 -- 172.21.15.82:0/1029919120 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6b2c01a070 con 0x7f6b34161fc0 2024-03-14T12:16:25.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.942+0000 7f6b22ffd640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b1406bbd0 0x7f6b1406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:25.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.943+0000 7f6b22ffd640 1 -- 172.21.15.82:0/1029919120 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6b2c089ad0 con 0x7f6b34161fc0 2024-03-14T12:16:25.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.943+0000 7f6b3259c640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b1406bbd0 0x7f6b1406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:25.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.943+0000 7f6b3259c640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b1406bbd0 0x7f6b1406e090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b24035000 tx=0x7f6b24030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:25.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:25.947+0000 7f6b22ffd640 1 -- 172.21.15.82:0/1029919120 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6b3415f0b0 con 0x7f6b34161fc0 2024-03-14T12:16:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.120+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b3404f050 con 0x7f6b34161fc0 2024-03-14T12:16:26.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.120+0000 7f6b22ffd640 1 -- 172.21.15.82:0/1029919120 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+13008 (secure 0 0 0) 0x7f6b3414ef40 con 0x7f6b34161fc0 2024-03-14T12:16:26.121 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:26.121 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:24.811595+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/2708572159":"2024-03-15T12:16:21.801410+0000","172.21.15.82:6835/2708572159":"2024-03-15T12:16:21.801410+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:26.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.122+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b1406bbd0 msgr2=0x7f6b1406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.122+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b1406bbd0 0x7f6b1406e090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b24035000 tx=0x7f6b24030000 comp rx=0 tx=0).stop 2024-03-14T12:16:26.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.122+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b34161fc0 msgr2=0x7f6b34150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.122+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b34161fc0 0x7f6b34150200 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c390 tx=0x7f6b2c00c850 comp rx=0 tx=0).stop 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 shutdown_connections 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b34161fc0 0x7f6b34150200 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b341551d0 0x7f6b3414fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b1406bbd0 0x7f6b1406e090 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 --2- 172.21.15.82:0/1029919120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b340b5bb0 0x7f6b3414a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 >> 172.21.15.82:0/1029919120 conn(0x7f6b34072ee0 msgr2=0x7f6b340aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 shutdown_connections 2024-03-14T12:16:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.123+0000 7f6b38b59640 1 -- 172.21.15.82:0/1029919120 wait complete. 2024-03-14T12:16:26.135 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:26.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:26.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.237+0000 7f82ab659640 1 Processor -- start 2024-03-14T12:16:26.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.237+0000 7f82ab659640 1 -- start start 2024-03-14T12:16:26.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f82ab659640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f82a415c2f0 0x7f82a40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f82ab659640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a40670c0 con 0x7f82a40b1e00 2024-03-14T12:16:26.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f82ab659640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a406c130 con 0x7f82a415bf10 2024-03-14T12:16:26.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f82ab659640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a40b62e0 con 0x7f82a415c2f0 2024-03-14T12:16:26.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f82a9bf6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f82a40b1e00 0x7f82a4142e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33526/0 (socket says 172.21.15.82:33526) 2024-03-14T12:16:26.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f82a9bf6640 1 -- 172.21.15.82:0/2952929637 learned_addr learned my addr 172.21.15.82:0/2952929637 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/2952929637 >> v2:172.21.15.164:3300/0 conn(0x7f82a415c2f0 0x7f82a40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044581398 0 0) 0x7f82a40670c0 con 0x7f82a40b1e00 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.238+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8290003520 con 0x7f82a40b1e00 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756766988 0 0) 0x7f8290003520 con 0x7f82a40b1e00 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 >> v2:172.21.15.164:3300/0 conn(0x7f82a415c2f0 msgr2=0x7f82a40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 --2- 172.21.15.82:0/2952929637 >> v2:172.21.15.164:3300/0 conn(0x7f82a415c2f0 0x7f82a40b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 >> v1:172.21.15.164:6789/0 conn(0x7f82a415bf10 legacy=0x7f82a40af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4066de0 con 0x7f82a40b1e00 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f829c0019f0 con 0x7f82a40b1e00 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 >> v1:172.21.15.82:6789/0 conn(0x7f82a40b1e00 legacy=0x7f82a4142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82900039b0 0x7f8290003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8290004800 0x7f8290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82900051c0 0x7f829000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40b62e0 con 0x7f8290004800 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829000c160 con 0x7f82900051c0 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829000c360 con 0x7f82900039b0 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8290004800 0x7f8290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a9bf6640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82900051c0 0x7f829000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a93f5640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82900039b0 0x7f8290003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a8bf4640 1 -- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82900039b0 msgr2=0x7f8290003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82900039b0 0x7f8290003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a8bf4640 1 -- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82900051c0 msgr2=0x7f829000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82900051c0 0x7f829000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a9bf6640 1 -- 172.21.15.82:0/2952929637 reap_dead start 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.239+0000 7f82a8bf4640 1 -- 172.21.15.82:0/2952929637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290003520 con 0x7f8290004800 2024-03-14T12:16:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.240+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8290004800 0x7f8290004c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8294009a90 tx=0x7f8290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad57072fa3fc49b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.240+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294039070 con 0x7f8290004800 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.240+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829402c680 con 0x7f8290004800 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.240+0000 7f829bfff640 1 -- 172.21.15.82:0/2952929637 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294009690 con 0x7f8290004800 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.241+0000 7f82ab659640 1 -- 172.21.15.82:0/2952929637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8290004800 msgr2=0x7f8290004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.241+0000 7f82ab659640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8290004800 0x7f8290004c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8294009a90 tx=0x7f8290005830 comp rx=0 tx=0).stop 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.241+0000 7f82ab659640 1 -- 172.21.15.82:0/2952929637 shutdown_connections 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.241+0000 7f82ab659640 1 --2- 172.21.15.82:0/2952929637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8290004800 0x7f8290004c80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.241+0000 7f82ab659640 1 -- 172.21.15.82:0/2952929637 >> 172.21.15.82:0/2952929637 conn(0x7f82a40b4fd0 msgr2=0x7f82a40b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.241+0000 7f82ab659640 1 -- 172.21.15.82:0/2952929637 shutdown_connections 2024-03-14T12:16:26.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.241+0000 7f82ab659640 1 -- 172.21.15.82:0/2952929637 wait complete. 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.242+0000 7f82ab659640 1 Processor -- start 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.242+0000 7f82ab659640 1 -- start start 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82ab659640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8290004800 0x7f82a40cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82ab659640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a40cbc00 0x7f82a406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82ab659640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a406ede0 0x7f82a41e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82ab659640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a4145b60 con 0x7f82a40cbc00 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82ab659640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a406c740 con 0x7f82a406ede0 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82ab659640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a4066a20 con 0x7f8290004800 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a40cbc00 0x7f82a406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a40cbc00 0x7f82a406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38804/0 (socket says 172.21.15.82:38804) 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 -- 172.21.15.82:0/1063885428 learned_addr learned my addr 172.21.15.82:0/1063885428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a9bf6640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a406ede0 0x7f82a41e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a93f5640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8290004800 0x7f82a40cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 -- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8290004800 msgr2=0x7f82a40cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8290004800 0x7f82a40cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 -- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a406ede0 msgr2=0x7f82a41e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a406ede0 0x7f82a41e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 -- 172.21.15.82:0/1063885428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a41f7090 con 0x7f82a40cbc00 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82a8bf4640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a40cbc00 0x7f82a406e8a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8294037040 tx=0x7f8294004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f8299ffb640 1 -- 172.21.15.82:0/1063885428 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294042020 con 0x7f82a40cbc00 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f8299ffb640 1 -- 172.21.15.82:0/1063885428 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829403a040 con 0x7f82a40cbc00 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f8299ffb640 1 -- 172.21.15.82:0/1063885428 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294039070 con 0x7f82a40cbc00 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.243+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a41f7370 con 0x7f82a40cbc00 2024-03-14T12:16:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.244+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a41f7930 con 0x7f82a40cbc00 2024-03-14T12:16:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.245+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a40af460 con 0x7f82a40cbc00 2024-03-14T12:16:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.245+0000 7f8299ffb640 1 -- 172.21.15.82:0/1063885428 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8294045020 con 0x7f82a40cbc00 2024-03-14T12:16:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.245+0000 7f8299ffb640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f828806bbd0 0x7f828806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.246+0000 7f82a93f5640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f828806bbd0 0x7f828806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.246+0000 7f8299ffb640 1 -- 172.21.15.82:0/1063885428 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f82940abb70 con 0x7f82a40cbc00 2024-03-14T12:16:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.246+0000 7f82a93f5640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f828806bbd0 0x7f828806e090 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c00 tx=0x7f82a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.250+0000 7f8299ffb640 1 -- 172.21.15.82:0/1063885428 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f82a40af460 con 0x7f82a40cbc00 2024-03-14T12:16:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.421+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82a404f050 con 0x7f82a40cbc00 2024-03-14T12:16:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.422+0000 7f8299ffb640 1 -- 172.21.15.82:0/1063885428 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f829404c070 con 0x7f82a40cbc00 2024-03-14T12:16:26.422 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.423+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f828806bbd0 msgr2=0x7f828806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.423+0000 7f82ab659640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f828806bbd0 0x7f828806e090 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c00 tx=0x7f82a0009040 comp rx=0 tx=0).stop 2024-03-14T12:16:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.423+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a40cbc00 msgr2=0x7f82a406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.423+0000 7f82ab659640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a40cbc00 0x7f82a406e8a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8294037040 tx=0x7f8294004730 comp rx=0 tx=0).stop 2024-03-14T12:16:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.423+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 shutdown_connections 2024-03-14T12:16:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.423+0000 7f82ab659640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a406ede0 0x7f82a41e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.424+0000 7f82ab659640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a40cbc00 0x7f82a406e8a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.424+0000 7f82ab659640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f828806bbd0 0x7f828806e090 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.424+0000 7f82ab659640 1 --2- 172.21.15.82:0/1063885428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8290004800 0x7f82a40cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.424+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 >> 172.21.15.82:0/1063885428 conn(0x7f82a40b4fd0 msgr2=0x7f82a415a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.424+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 shutdown_connections 2024-03-14T12:16:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.424+0000 7f82ab659640 1 -- 172.21.15.82:0/1063885428 wait complete. 2024-03-14T12:16:26.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_disconnect' 2024-03-14T12:16:26.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.535+0000 7f2b3b2dc640 1 Processor -- start 2024-03-14T12:16:26.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b3b2dc640 1 -- start start 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b3b2dc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b340d0760 0x7f2b340d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b3b2dc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340cf370 0x7f2b340d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b3b2dc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b340d1da0 0x7f2b340d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b3b2dc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b340d1080 con 0x7f2b340d0760 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b3b2dc640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b340cf8c0 con 0x7f2b340d1da0 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b3b2dc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340cfba0 con 0x7f2b340cf370 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b39078640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b340d0760 0x7f2b340d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b39078640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b340d0760 0x7f2b340d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38816/0 (socket says 172.21.15.82:38816) 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.536+0000 7f2b39078640 1 -- 172.21.15.82:0/1236747823 learned_addr learned my addr 172.21.15.82:0/1236747823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39879640 1 --2- 172.21.15.82:0/1236747823 >> v2:172.21.15.164:3300/0 conn(0x7f2b340d1da0 0x7f2b340d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b38877640 1 --2- 172.21.15.82:0/1236747823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340cf370 0x7f2b340d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39078640 1 -- 172.21.15.82:0/1236747823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340cf370 msgr2=0x7f2b340d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39078640 1 --2- 172.21.15.82:0/1236747823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340cf370 0x7f2b340d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39078640 1 -- 172.21.15.82:0/1236747823 >> v2:172.21.15.164:3300/0 conn(0x7f2b340d1da0 msgr2=0x7f2b340d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39078640 1 --2- 172.21.15.82:0/1236747823 >> v2:172.21.15.164:3300/0 conn(0x7f2b340d1da0 0x7f2b340d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39078640 1 -- 172.21.15.82:0/1236747823 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340b6c40 con 0x7f2b340d0760 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b38877640 1 --2- 172.21.15.82:0/1236747823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340cf370 0x7f2b340d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39879640 1 --2- 172.21.15.82:0/1236747823 >> v2:172.21.15.164:3300/0 conn(0x7f2b340d1da0 0x7f2b340d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.537+0000 7f2b39078640 1 --2- 172.21.15.82:0/1236747823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340d0b40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2b3408cbb0 tx=0x7f2b3002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14c2e571bbfa362 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b2bfff640 1 -- 172.21.15.82:0/1236747823 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3002cb00 con 0x7f2b340d0760 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b2bfff640 1 -- 172.21.15.82:0/1236747823 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3002cca0 con 0x7f2b340d0760 2024-03-14T12:16:26.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b2bfff640 1 -- 172.21.15.82:0/1236747823 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30034610 con 0x7f2b340d0760 2024-03-14T12:16:26.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/1236747823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 msgr2=0x7f2b340d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/1236747823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340d0b40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2b3408cbb0 tx=0x7f2b3002bf60 comp rx=0 tx=0).stop 2024-03-14T12:16:26.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/1236747823 shutdown_connections 2024-03-14T12:16:26.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/1236747823 >> v2:172.21.15.164:3300/0 conn(0x7f2b340d1da0 0x7f2b340d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/1236747823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340cf370 0x7f2b340d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/1236747823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340d0b40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/1236747823 >> 172.21.15.82:0/1236747823 conn(0x7f2b3409d030 msgr2=0x7f2b340d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/1236747823 shutdown_connections 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.538+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/1236747823 wait complete. 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.539+0000 7f2b3b2dc640 1 Processor -- start 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.539+0000 7f2b3b2dc640 1 -- start start 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b3b2dc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b340cf370 0x7f2b3415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b3b2dc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b3b2dc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340d1da0 0x7f2b3414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b3b2dc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340d4ba0 con 0x7f2b340d0760 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b3b2dc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b7200 con 0x7f2b340cf370 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b3b2dc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b341617a0 con 0x7f2b340d1da0 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b38877640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b38877640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38830/0 (socket says 172.21.15.82:38830) 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.540+0000 7f2b38877640 1 -- 172.21.15.82:0/3684797836 learned_addr learned my addr 172.21.15.82:0/3684797836 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b39078640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b340cf370 0x7f2b3415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b38877640 1 -- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340d1da0 msgr2=0x7f2b3414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b39879640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340d1da0 0x7f2b3414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b38877640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340d1da0 0x7f2b3414cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b38877640 1 -- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b340cf370 msgr2=0x7f2b3415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b38877640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b340cf370 0x7f2b3415d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b38877640 1 -- 172.21.15.82:0/3684797836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340cd210 con 0x7f2b340d0760 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b38877640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340cc750 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c820 tx=0x7f2b2400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b29ffb640 1 -- 172.21.15.82:0/3684797836 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24015070 con 0x7f2b340d0760 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b29ffb640 1 -- 172.21.15.82:0/3684797836 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b24007a30 con 0x7f2b340d0760 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b340cd4f0 con 0x7f2b340d0760 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b29ffb640 1 -- 172.21.15.82:0/3684797836 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24007e20 con 0x7f2b340d0760 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.541+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b341509b0 con 0x7f2b340d0760 2024-03-14T12:16:26.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.542+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3404f050 con 0x7f2b340d0760 2024-03-14T12:16:26.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.542+0000 7f2b29ffb640 1 -- 172.21.15.82:0/3684797836 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2b24018070 con 0x7f2b340d0760 2024-03-14T12:16:26.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.543+0000 7f2b29ffb640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b1806bbd0 0x7f2b1806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.543+0000 7f2b29ffb640 1 -- 172.21.15.82:0/3684797836 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2b2408a680 con 0x7f2b340d0760 2024-03-14T12:16:26.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.543+0000 7f2b39078640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b1806bbd0 0x7f2b1806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.543+0000 7f2b39078640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b1806bbd0 0x7f2b1806e090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2b30035000 tx=0x7f2b30030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.547+0000 7f2b29ffb640 1 -- 172.21.15.82:0/3684797836 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2b24059480 con 0x7f2b340d0760 2024-03-14T12:16:26.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.786+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_disconnect"]} v 0) v1 -- 0x7f2b340d0540 con 0x7f2b340d0760 2024-03-14T12:16:26.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.821+0000 7f2b29ffb640 1 -- 172.21.15.82:0/3684797836 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_disconnect"]}]=0 v148) v1 ==== 188+0+0 (secure 0 0 0) 0x7f2b24058d80 con 0x7f2b340d0760 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b1806bbd0 msgr2=0x7f2b1806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b1806bbd0 0x7f2b1806e090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2b30035000 tx=0x7f2b30030000 comp rx=0 tx=0).stop 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 msgr2=0x7f2b340cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340cc750 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c820 tx=0x7f2b2400cce0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 shutdown_connections 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b1806bbd0 0x7f2b1806e090 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b340d1da0 0x7f2b3414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b340d0760 0x7f2b340cc750 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 --2- 172.21.15.82:0/3684797836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b340cf370 0x7f2b3415d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 >> 172.21.15.82:0/3684797836 conn(0x7f2b3409d030 msgr2=0x7f2b340cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:26.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 shutdown_connections 2024-03-14T12:16:26.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.823+0000 7f2b3b2dc640 1 -- 172.21.15.82:0/3684797836 wait complete. 2024-03-14T12:16:26.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-14T12:16:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.938+0000 7f5247297640 1 Processor -- start 2024-03-14T12:16:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.938+0000 7f5247297640 1 -- start start 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5247297640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52400d02d0 0x7f524015e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5247297640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52400b41d0 0x7f524015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5247297640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52400b3450 con 0x7f52400cf3f0 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5247297640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52400cf7d0 con 0x7f52400b41d0 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5247297640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524015f200 con 0x7f52400d02d0 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5245834640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52400b41d0 0x7f524015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5245834640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52400b41d0 0x7f524015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38842/0 (socket says 172.21.15.82:38842) 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5245834640 1 -- 172.21.15.82:0/1959821749 learned_addr learned my addr 172.21.15.82:0/1959821749 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5245834640 1 -- 172.21.15.82:0/1959821749 >> v2:172.21.15.164:3300/0 conn(0x7f52400d02d0 msgr2=0x7f524015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5245834640 1 --2- 172.21.15.82:0/1959821749 >> v2:172.21.15.164:3300/0 conn(0x7f52400d02d0 0x7f524015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5245834640 1 -- 172.21.15.82:0/1959821749 >> v1:172.21.15.164:6789/0 conn(0x7f52400cf3f0 legacy=0x7f52400cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.939+0000 7f5245834640 1 -- 172.21.15.82:0/1959821749 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524015f4b0 con 0x7f52400b41d0 2024-03-14T12:16:26.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.940+0000 7f5245834640 1 --2- 172.21.15.82:0/1959821749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f524015ecc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f524006d2d0 tx=0x7f523802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa45f58324df155e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.940+0000 7f5237fff640 1 -- 172.21.15.82:0/1959821749 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238038070 con 0x7f52400b41d0 2024-03-14T12:16:26.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.940+0000 7f5237fff640 1 -- 172.21.15.82:0/1959821749 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523802f050 con 0x7f52400b41d0 2024-03-14T12:16:26.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.940+0000 7f5237fff640 1 -- 172.21.15.82:0/1959821749 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238031970 con 0x7f52400b41d0 2024-03-14T12:16:26.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 -- 172.21.15.82:0/1959821749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 msgr2=0x7f524015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 --2- 172.21.15.82:0/1959821749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f524015ecc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f524006d2d0 tx=0x7f523802b7f0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 -- 172.21.15.82:0/1959821749 shutdown_connections 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 --2- 172.21.15.82:0/1959821749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f524015ecc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 --2- 172.21.15.82:0/1959821749 >> v2:172.21.15.164:3300/0 conn(0x7f52400d02d0 0x7f524015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 -- 172.21.15.82:0/1959821749 >> 172.21.15.82:0/1959821749 conn(0x7f524006aef0 msgr2=0x7f524006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 -- 172.21.15.82:0/1959821749 shutdown_connections 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.941+0000 7f5247297640 1 -- 172.21.15.82:0/1959821749 wait complete. 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.942+0000 7f5247297640 1 Processor -- start 2024-03-14T12:16:26.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.942+0000 7f5247297640 1 -- start start 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5247297640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f5240138410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5247297640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52400cf3f0 0x7f52401514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5247297640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52400d02d0 0x7f52400b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5247297640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52400b3cb0 con 0x7f52400b41d0 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5247297640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52400b39d0 con 0x7f52400cf3f0 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5247297640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524006d890 con 0x7f52400d02d0 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f5240138410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f5240138410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38850/0 (socket says 172.21.15.82:38850) 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 -- 172.21.15.82:0/1784908041 learned_addr learned my addr 172.21.15.82:0/1784908041 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:26.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5244832640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52400cf3f0 0x7f52401514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245834640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52400d02d0 0x7f52400b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 -- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52400d02d0 msgr2=0x7f52400b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52400d02d0 0x7f52400b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 -- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52400cf3f0 msgr2=0x7f52401514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52400cf3f0 0x7f52401514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 -- 172.21.15.82:0/1784908041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240150420 con 0x7f52400b41d0 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5244832640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52400cf3f0 0x7f52401514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5245033640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f5240138410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c7a0 tx=0x7f523c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5235ffb640 1 -- 172.21.15.82:0/1784908041 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c017070 con 0x7f52400b41d0 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5235ffb640 1 -- 172.21.15.82:0/1784908041 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c003da0 con 0x7f52400b41d0 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5235ffb640 1 -- 172.21.15.82:0/1784908041 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c0052b0 con 0x7f52400b41d0 2024-03-14T12:16:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.943+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5240150700 con 0x7f52400b41d0 2024-03-14T12:16:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.944+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5240150cc0 con 0x7f52400b41d0 2024-03-14T12:16:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.945+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52401510f0 con 0x7f52400b41d0 2024-03-14T12:16:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.945+0000 7f5235ffb640 1 -- 172.21.15.82:0/1784908041 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f523c01a070 con 0x7f52400b41d0 2024-03-14T12:16:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.945+0000 7f5235ffb640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f522406bbd0 0x7f522406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.945+0000 7f5235ffb640 1 -- 172.21.15.82:0/1784908041 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f523c089de0 con 0x7f52400b41d0 2024-03-14T12:16:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.945+0000 7f5244832640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f522406bbd0 0x7f522406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.946+0000 7f5244832640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f522406bbd0 0x7f522406e090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f524014afc0 tx=0x7f5230008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:26.950+0000 7f5235ffb640 1 -- 172.21.15.82:0/1784908041 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f523c058c60 con 0x7f52400b41d0 2024-03-14T12:16:27.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.122+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f524004f050 con 0x7f52400b41d0 2024-03-14T12:16:27.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.125+0000 7f5235ffb640 1 -- 172.21.15.82:0/1784908041 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f523c058560 con 0x7f52400b41d0 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f522406bbd0 msgr2=0x7f522406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f522406bbd0 0x7f522406e090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f524014afc0 tx=0x7f5230008040 comp rx=0 tx=0).stop 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 msgr2=0x7f5240138410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f5240138410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c7a0 tx=0x7f523c00cc60 comp rx=0 tx=0).stop 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 shutdown_connections 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f522406bbd0 0x7f522406e090 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52400d02d0 0x7f52400b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52400cf3f0 0x7f52401514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 --2- 172.21.15.82:0/1784908041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52400b41d0 0x7f5240138410 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 >> 172.21.15.82:0/1784908041 conn(0x7f524006aef0 msgr2=0x7f524015e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:27.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 shutdown_connections 2024-03-14T12:16:27.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.127+0000 7f5247297640 1 -- 172.21.15.82:0/1784908041 wait complete. 2024-03-14T12:16:27.139 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:16:27.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:27.175 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:27.223 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:16:27.223 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:27.223 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:27.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:16:27.240 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:27.240 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:27.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:16:27.257 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:27.257 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:27.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:16:27.294 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:27.295 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:27.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:16:27.314 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-14T12:16:27.314 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:27.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.435+0000 7f2b417e1640 1 Processor -- start 2024-03-14T12:16:27.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.435+0000 7f2b417e1640 1 -- start start 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.435+0000 7f2b417e1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b3c0d0760 0x7f2b3c0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.435+0000 7f2b417e1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0cf370 0x7f2b3c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.435+0000 7f2b417e1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c0d1080 con 0x7f2b3c0d1da0 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.435+0000 7f2b417e1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3c0cf8c0 con 0x7f2b3c0d0760 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.435+0000 7f2b417e1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0cfba0 con 0x7f2b3c0cf370 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b3c0d0760 0x7f2b3c0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b3c0d0760 0x7f2b3c0d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38852/0 (socket says 172.21.15.82:38852) 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3affd640 1 -- 172.21.15.82:0/224810799 learned_addr learned my addr 172.21.15.82:0/224810799 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3a7fc640 1 --2- 172.21.15.82:0/224810799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0cf370 0x7f2b3c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3affd640 1 -- 172.21.15.82:0/224810799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0cf370 msgr2=0x7f2b3c0d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3affd640 1 --2- 172.21.15.82:0/224810799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0cf370 0x7f2b3c0d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3affd640 1 -- 172.21.15.82:0/224810799 >> v1:172.21.15.164:6789/0 conn(0x7f2b3c0d1da0 legacy=0x7f2b3c0d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3affd640 1 -- 172.21.15.82:0/224810799 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c0b6c40 con 0x7f2b3c0d0760 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.436+0000 7f2b3a7fc640 1 --2- 172.21.15.82:0/224810799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0cf370 0x7f2b3c0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b3affd640 1 --2- 172.21.15.82:0/224810799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0d0760 0x7f2b3c0d0b40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009870 tx=0x7f2b2c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c366141bb7da424 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b39ffb640 1 -- 172.21.15.82:0/224810799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c02f840 con 0x7f2b3c0d0760 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b39ffb640 1 -- 172.21.15.82:0/224810799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c02f9e0 con 0x7f2b3c0d0760 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b39ffb640 1 -- 172.21.15.82:0/224810799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c02fdb0 con 0x7f2b3c0d0760 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 -- 172.21.15.82:0/224810799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0d0760 msgr2=0x7f2b3c0d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 --2- 172.21.15.82:0/224810799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0d0760 0x7f2b3c0d0b40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009870 tx=0x7f2b2c02ba20 comp rx=0 tx=0).stop 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 -- 172.21.15.82:0/224810799 shutdown_connections 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 --2- 172.21.15.82:0/224810799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0cf370 0x7f2b3c0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 --2- 172.21.15.82:0/224810799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0d0760 0x7f2b3c0d0b40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 -- 172.21.15.82:0/224810799 >> 172.21.15.82:0/224810799 conn(0x7f2b3c09d030 msgr2=0x7f2b3c0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:27.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 -- 172.21.15.82:0/224810799 shutdown_connections 2024-03-14T12:16:27.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.437+0000 7f2b417e1640 1 -- 172.21.15.82:0/224810799 wait complete. 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 Processor -- start 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 -- start start 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0cf370 0x7f2b3c158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0d0760 0x7f2b3c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3c0d1da0 0x7f2b3c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0d4ba0 con 0x7f2b3c0cf370 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0b7200 con 0x7f2b3c0d1da0 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b417e1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1617a0 con 0x7f2b3c0d0760 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b3affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0cf370 0x7f2b3c158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b3affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0cf370 0x7f2b3c158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38864/0 (socket says 172.21.15.82:38864) 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.439+0000 7f2b3affd640 1 -- 172.21.15.82:0/1359020735 learned_addr learned my addr 172.21.15.82:0/1359020735 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3b7fe640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3c0d1da0 0x7f2b3c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3a7fc640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0d0760 0x7f2b3c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3affd640 1 -- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0d0760 msgr2=0x7f2b3c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3affd640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0d0760 0x7f2b3c153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3affd640 1 -- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3c0d1da0 msgr2=0x7f2b3c15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3affd640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3c0d1da0 0x7f2b3c15b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3affd640 1 -- 172.21.15.82:0/1359020735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c0c7920 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b3affd640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0cf370 0x7f2b3c158bc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c02ba60 tx=0x7f2b2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b237fe640 1 -- 172.21.15.82:0/1359020735 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c047070 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b237fe640 1 -- 172.21.15.82:0/1359020735 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c042480 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c0c7c00 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b237fe640 1 -- 172.21.15.82:0/1359020735 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c042870 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.440+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c0c81c0 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.441+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c04f050 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.441+0000 7f2b237fe640 1 -- 172.21.15.82:0/1359020735 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2b2c04a070 con 0x7f2b3c0cf370 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.442+0000 7f2b237fe640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b2806bbd0 0x7f2b2806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.442+0000 7f2b3a7fc640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b2806bbd0 0x7f2b2806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.442+0000 7f2b237fe640 1 -- 172.21.15.82:0/1359020735 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2b2c0acae0 con 0x7f2b3c0cf370 2024-03-14T12:16:27.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.442+0000 7f2b3a7fc640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b2806bbd0 0x7f2b2806e090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0b9a10 tx=0x7f2b30008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:27.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.446+0000 7f2b237fe640 1 -- 172.21.15.82:0/1359020735 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2b2c033030 con 0x7f2b3c0cf370 2024-03-14T12:16:27.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.653+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b3c0d0540 con 0x7f2b3c0cf370 2024-03-14T12:16:27.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.654+0000 7f2b237fe640 1 -- 172.21.15.82:0/1359020735 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f2b2c07b960 con 0x7f2b3c0cf370 2024-03-14T12:16:27.654 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:27.654 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1414844632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":1414844632},{"type":"v1","addr":"172.21.15.164:6835","nonce":1414844632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/456693586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":456693586},{"type":"v1","addr":"172.21.15.164:6834","nonce":456693586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/58260684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":58260684},{"type":"v1","addr":"172.21.15.82:6836","nonce":58260684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/140256028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":140256028},{"type":"v1","addr":"172.21.15.82:6837","nonce":140256028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[]} 2024-03-14T12:16:27.655 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 23 2024-03-14T12:16:27.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b2806bbd0 msgr2=0x7f2b2806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b2806bbd0 0x7f2b2806e090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0b9a10 tx=0x7f2b30008040 comp rx=0 tx=0).stop 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0cf370 msgr2=0x7f2b3c158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0cf370 0x7f2b3c158bc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c02ba60 tx=0x7f2b2c004730 comp rx=0 tx=0).stop 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 shutdown_connections 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b3c0d1da0 0x7f2b3c15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2b2806bbd0 0x7f2b2806e090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b3c0d0760 0x7f2b3c153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 --2- 172.21.15.82:0/1359020735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c0cf370 0x7f2b3c158bc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 >> 172.21.15.82:0/1359020735 conn(0x7f2b3c09d030 msgr2=0x7f2b3c15f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 shutdown_connections 2024-03-14T12:16:27.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.656+0000 7f2b417e1640 1 -- 172.21.15.82:0/1359020735 wait complete. 2024-03-14T12:16:27.667 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T12:16:27.667 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-14T12:16:27.668 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T12:16:27.668 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:27.668 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T12:16:27.668 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-14T12:16:27.668 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T12:16:27.668 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:27.668 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T12:16:27.669 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-14T12:16:27.669 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T12:16:27.669 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:27.669 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T12:16:27.669 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-14T12:16:27.669 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T12:16:27.669 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:27.678 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T12:16:27.678 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T12:16:27.679 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T12:16:27.679 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T12:16:27.681 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T12:16:27.681 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T12:16:27.682 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T12:16:27.682 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T12:16:27.721 INFO:tasks.ceph.mds.a:Started 2024-03-14T12:16:27.721 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:27.721 INFO:tasks.ceph.mds.c:Started 2024-03-14T12:16:27.722 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:27.724 INFO:tasks.ceph.mds.b:Started 2024-03-14T12:16:27.724 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:27.725 INFO:tasks.ceph.mds.d:Started 2024-03-14T12:16:27.725 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:27.725 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-14T12:16:27.796 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-14T12:16:27.797 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:16:27.810 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-03-14T12:16:27.810 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-03-14T12:16:27.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.833+0000 7f21b222b640 1 Processor -- start 2024-03-14T12:16:27.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.833+0000 7f21b222b640 1 -- start start 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21b222b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21b222b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21ac15bdc0 0x7f21ac0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21b222b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac15c220 con 0x7f21ac0d0440 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21b222b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac0cf730 con 0x7f21ac15bdc0 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21b222b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0cd810 con 0x7f21ac15b9e0 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21a2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21ac15bdc0 0x7f21ac0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21abfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f21ac0d0440 0x7f21ac15f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33564/0 (socket says 172.21.15.82:33564) 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21a2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21ac15bdc0 0x7f21ac0cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38898/0 (socket says 172.21.15.82:38898) 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21abfff640 1 -- 172.21.15.82:0/40939302 learned_addr learned my addr 172.21.15.82:0/40939302 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21ab7fe640 1 --2- 172.21.15.82:0/40939302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21a2ffd640 1 -- 172.21.15.82:0/40939302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 msgr2=0x7f21ac0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:27.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21a2ffd640 1 --2- 172.21.15.82:0/40939302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac0b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21a2ffd640 1 -- 172.21.15.82:0/40939302 >> v1:172.21.15.82:6789/0 conn(0x7f21ac0d0440 legacy=0x7f21ac15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.834+0000 7f21a2ffd640 1 -- 172.21.15.82:0/40939302 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0cf980 con 0x7f21ac15bdc0 2024-03-14T12:16:27.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.835+0000 7f21a2ffd640 1 --2- 172.21.15.82:0/40939302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0cd0d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6fe66bf9441d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:27.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.836+0000 7f21aaffd640 1 -- 172.21.15.82:0/40939302 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940315a0 con 0x7f21ac15bdc0 2024-03-14T12:16:27.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.836+0000 7f21aaffd640 1 -- 172.21.15.82:0/40939302 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194037070 con 0x7f21ac15bdc0 2024-03-14T12:16:27.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.836+0000 7f21aaffd640 1 -- 172.21.15.82:0/40939302 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031960 con 0x7f21ac15bdc0 2024-03-14T12:16:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 -- 172.21.15.82:0/40939302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 msgr2=0x7f21ac0cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 --2- 172.21.15.82:0/40939302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0cd0d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402b840 comp rx=0 tx=0).stop 2024-03-14T12:16:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 -- 172.21.15.82:0/40939302 shutdown_connections 2024-03-14T12:16:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 --2- 172.21.15.82:0/40939302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0cd0d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 --2- 172.21.15.82:0/40939302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac0b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 -- 172.21.15.82:0/40939302 >> 172.21.15.82:0/40939302 conn(0x7f21ac0b3670 msgr2=0x7f21ac0b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:27.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 -- 172.21.15.82:0/40939302 shutdown_connections 2024-03-14T12:16:27.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.837+0000 7f21b222b640 1 -- 172.21.15.82:0/40939302 wait complete. 2024-03-14T12:16:27.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 Processor -- start 2024-03-14T12:16:27.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 -- start start 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21ac0d0440 0x7f21ac06d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac06c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac160050 con 0x7f21ac15bdc0 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0cff40 con 0x7f21ac0d0440 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.839+0000 7f21b222b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac1601d0 con 0x7f21ac15b9e0 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38902/0 (socket says 172.21.15.82:38902) 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21a2ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 -- 172.21.15.82:0/1932037910 learned_addr learned my addr 172.21.15.82:0/1932037910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21ab7fe640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21ac0d0440 0x7f21ac06d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 -- 172.21.15.82:0/1932037910 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 msgr2=0x7f21ac06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 -- 172.21.15.82:0/1932037910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21ac0d0440 msgr2=0x7f21ac06d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:27.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21ac0d0440 0x7f21ac06d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:27.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 -- 172.21.15.82:0/1932037910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac06cfa0 con 0x7f21ac15bdc0 2024-03-14T12:16:27.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.840+0000 7f21abfff640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0c47e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cab0 tx=0x7f21a400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.842+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1932037910 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4017070 con 0x7f21ac15bdc0 2024-03-14T12:16:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.842+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1932037910 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a400ac30 con 0x7f21ac15bdc0 2024-03-14T12:16:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.842+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1932037910 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4012400 con 0x7f21ac15bdc0 2024-03-14T12:16:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.842+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac15c690 con 0x7f21ac15bdc0 2024-03-14T12:16:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.843+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac15cc50 con 0x7f21ac15bdc0 2024-03-14T12:16:27.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.843+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac04f050 con 0x7f21ac15bdc0 2024-03-14T12:16:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.844+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1932037910 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f21a401a070 con 0x7f21ac15bdc0 2024-03-14T12:16:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.844+0000 7f21a8ff9640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f218c06bbd0 0x7f218c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.844+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1932037910 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f21a408a2f0 con 0x7f21ac15bdc0 2024-03-14T12:16:27.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.844+0000 7f21ab7fe640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f218c06bbd0 0x7f218c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:27.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.845+0000 7f21ab7fe640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f218c06bbd0 0x7f218c06e090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f219c009b40 tx=0x7f219c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:27.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:27.849+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1932037910 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f21a4058b20 con 0x7f21ac15bdc0 2024-03-14T12:16:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.034+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f21ac0b7ae0 con 0x7f21ac15bdc0 2024-03-14T12:16:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.850+0000 7f21a8ff9640 1 -- 172.21.15.82:0/1932037910 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7f21a405cbc0 con 0x7f21ac15bdc0 2024-03-14T12:16:28.851 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f218c06bbd0 msgr2=0x7f218c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f218c06bbd0 0x7f218c06e090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f219c009b40 tx=0x7f219c009190 comp rx=0 tx=0).stop 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 msgr2=0x7f21ac0c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0c47e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cab0 tx=0x7f21a400cf70 comp rx=0 tx=0).stop 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 shutdown_connections 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac15bdc0 0x7f21ac0c47e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f218c06bbd0 0x7f218c06e090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21ac15b9e0 0x7f21ac06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 --2- 172.21.15.82:0/1932037910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21ac0d0440 0x7f21ac06d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 >> 172.21.15.82:0/1932037910 conn(0x7f21ac0b3670 msgr2=0x7f21ac159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 shutdown_connections 2024-03-14T12:16:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.853+0000 7f21b222b640 1 -- 172.21.15.82:0/1932037910 wait complete. 2024-03-14T12:16:28.867 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-14T12:16:28.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.972+0000 7f881df9d640 1 Processor -- start 2024-03-14T12:16:28.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.972+0000 7f881df9d640 1 -- start start 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f881df9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88180cdae0 0x7f881806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f881df9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f881806cf80 0x7f8818161c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f881df9d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88180d1000 con 0x7f8818162180 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f881df9d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88180d0b70 con 0x7f88180cdae0 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f881df9d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88180cdec0 con 0x7f881806cf80 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f88177fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88180cdae0 0x7f881806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f8816ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f881806cf80 0x7f8818161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f8817fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8818162180 0x7f88180ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:51184/0 (socket says 172.21.15.82:51184) 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.973+0000 7f8817fff640 1 -- 172.21.15.82:0/2322017966 learned_addr learned my addr 172.21.15.82:0/2322017966 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f88177fe640 1 -- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f881806cf80 msgr2=0x7f8818161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:28.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f88177fe640 1 --2- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f881806cf80 0x7f8818161c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f88177fe640 1 -- 172.21.15.82:0/2322017966 >> v1:172.21.15.164:6789/0 conn(0x7f8818162180 legacy=0x7f88180ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:28.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f88177fe640 1 -- 172.21.15.82:0/2322017966 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180cff60 con 0x7f88180cdae0 2024-03-14T12:16:28.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f88167fc640 1 -- 172.21.15.82:0/2322017966 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636405168 0 0) 0x7f8810001490 con 0x7f8818162180 2024-03-14T12:16:28.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f8816ffd640 1 --2- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f881806cf80 0x7f8818161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:28.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f88177fe640 1 --2- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 0x7f881806ca40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f88080097a0 tx=0x7f880802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77789042baadbf29 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:28.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.974+0000 7f88167fc640 1 -- 172.21.15.82:0/2322017966 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808034070 con 0x7f88180cdae0 2024-03-14T12:16:28.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f88167fc640 1 -- 172.21.15.82:0/2322017966 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8808030800 con 0x7f88180cdae0 2024-03-14T12:16:28.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f88167fc640 1 -- 172.21.15.82:0/2322017966 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808030bd0 con 0x7f88180cdae0 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 -- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 msgr2=0x7f881806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 --2- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 0x7f881806ca40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f88080097a0 tx=0x7f880802b940 comp rx=0 tx=0).stop 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 -- 172.21.15.82:0/2322017966 shutdown_connections 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 --2- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f881806cf80 0x7f8818161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 --2- 172.21.15.82:0/2322017966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 0x7f881806ca40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 -- 172.21.15.82:0/2322017966 >> 172.21.15.82:0/2322017966 conn(0x7f88180b3610 msgr2=0x7f88180b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 -- 172.21.15.82:0/2322017966 shutdown_connections 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.975+0000 7f881df9d640 1 -- 172.21.15.82:0/2322017966 wait complete. 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.976+0000 7f881df9d640 1 Processor -- start 2024-03-14T12:16:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.976+0000 7f881df9d640 1 -- start start 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f881df9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f881806cf80 0x7f8818155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f881df9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 0x7f881815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f881df9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8818162180 0x7f881815f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f881df9d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88180d0800 con 0x7f881806cf80 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f881df9d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88180d0520 con 0x7f88180cdae0 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f881df9d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88180b4360 con 0x7f8818162180 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f88177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f881806cf80 0x7f8818155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f88177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f881806cf80 0x7f8818155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38916/0 (socket says 172.21.15.82:38916) 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f88177fe640 1 -- 172.21.15.82:0/1784290629 learned_addr learned my addr 172.21.15.82:0/1784290629 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f8816ffd640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 0x7f881815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.977+0000 7f8817fff640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8818162180 0x7f881815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f88177fe640 1 -- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8818162180 msgr2=0x7f881815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f88177fe640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8818162180 0x7f881815f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f88177fe640 1 -- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 msgr2=0x7f881815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f88177fe640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 0x7f881815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f88177fe640 1 -- 172.21.15.82:0/1784290629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88181f6e70 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f88177fe640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f881806cf80 0x7f8818155ff0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f88080097a0 tx=0x7f88080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f87fffff640 1 -- 172.21.15.82:0/1784290629 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808034040 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f87fffff640 1 -- 172.21.15.82:0/1784290629 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8808030600 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88181f7150 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f87fffff640 1 -- 172.21.15.82:0/1784290629 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808006e10 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.978+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881815a760 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.979+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881804f050 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.979+0000 7f87fffff640 1 -- 172.21.15.82:0/1784290629 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8808037070 con 0x7f881806cf80 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.980+0000 7f87fffff640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f880406bbd0 0x7f880406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.980+0000 7f87fffff640 1 -- 172.21.15.82:0/1784290629 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88080ac340 con 0x7f881806cf80 2024-03-14T12:16:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.980+0000 7f8816ffd640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f880406bbd0 0x7f880406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.980+0000 7f8816ffd640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f880406bbd0 0x7f880406e090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f880c009560 tx=0x7f880c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:28.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:28.984+0000 7f87fffff640 1 -- 172.21.15.82:0/1784290629 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f880807abd0 con 0x7f881806cf80 2024-03-14T12:16:29.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.222+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f881806c940 con 0x7f881806cf80 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.222+0000 7f87fffff640 1 -- 172.21.15.82:0/1784290629 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f880807ec70 con 0x7f881806cf80 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlAQmRIBAA45IMoIGH+bLaSIro/QUEmQ==", 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:29.223 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJljqfOLBAAulbwLxzEv+lHJOPUy3p4BQ==", 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.224 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlt7bdJxAAH34bGUjVrlElC4nA11eNIQ==", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlmeIuNBAAzNKyG5CdCVCURsWZywpTnw==", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:29.225 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDJ6fJlKwo/HBAA/Ma3FBjIdO1LF3RySRD0DQ==", 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-03-14T12:16:29.226 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDM6fJlot+1GhAAEQqFqku2rdbqZ3rEBfVO2g==", 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDP6fJlPRIYGRAAUxqEbSYn6AhIE+3lXEqEOg==", 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDS6fJlEYhJGBAApIl3qciLHwIJzDq5J2NkqA==", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDY6fJlL2cQHxAAeHY/pdO0TvpUciB22xvOUg==", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.228 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDb6fJlrNxAHRAALwT6cIpCh9fE4NMoqV6PWA==", 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-03-14T12:16:29.229 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDe6fJlPLi2GxAAeZ/Nk/340P6Y0ZJYojfNMw==", 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDh6fJlLhW0GhAAtvxrvm93pbGKTJbzzpIRiQ==", 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg==", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlUlkBBhAA9QGPbDVg8uWX5WTR2W936g==", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:16:29.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlDQeZAhAAJW1K+HBJuwgzu4nVr8Mq8g==", 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-03-14T12:16:29.232 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJldFHSCxAAci5DVZh9406uu1BZIAZXiw==", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlhoD/AxAA/LRLwtPkAs14z0fLh+eGjA==", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:29.233 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJl5qEWHRAAI8LqGdvUwjHG/5pNgcAB1A==", 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlxtwQGRAArcSgLFwgKjmSqjq0sst8Gw==", 2024-03-14T12:16:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:29.235 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:16:29.235 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:16:29.235 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:29.235 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.235 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:29.235 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:16:29.235 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:29.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f880406bbd0 msgr2=0x7f880406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f880406bbd0 0x7f880406e090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f880c009560 tx=0x7f880c009190 comp rx=0 tx=0).stop 2024-03-14T12:16:29.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f881806cf80 msgr2=0x7f8818155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f881806cf80 0x7f8818155ff0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f88080097a0 tx=0x7f88080046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 shutdown_connections 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f880406bbd0 0x7f880406e090 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8818162180 0x7f881815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88180cdae0 0x7f881815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 --2- 172.21.15.82:0/1784290629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f881806cf80 0x7f8818155ff0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 >> 172.21.15.82:0/1784290629 conn(0x7f88180b3610 msgr2=0x7f88180b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 shutdown_connections 2024-03-14T12:16:29.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.224+0000 7f881df9d640 1 -- 172.21.15.82:0/1784290629 wait complete. 2024-03-14T12:16:29.238 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.337+0000 7f1e1fa4a640 1 Processor -- start 2024-03-14T12:16:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.337+0000 7f1e1fa4a640 1 -- start start 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.338+0000 7f1e1fa4a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e1815e180 0x7f1e180b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.338+0000 7f1e1fa4a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e180b5080 con 0x7f1e180d07b0 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.338+0000 7f1e1fa4a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e180b5360 con 0x7f1e18066a20 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.338+0000 7f1e1fa4a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e180b5640 con 0x7f1e1815e180 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.338+0000 7f1e1d7e6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e1815e180 0x7f1e180b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.338+0000 7f1e1d7e6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e1815e180 0x7f1e180b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38930/0 (socket says 172.21.15.82:38930) 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.338+0000 7f1e1d7e6640 1 -- 172.21.15.82:0/2245665320 learned_addr learned my addr 172.21.15.82:0/2245665320 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.339+0000 7f1e1d7e6640 1 -- 172.21.15.82:0/2245665320 >> v1:172.21.15.164:6789/0 conn(0x7f1e18066a20 legacy=0x7f1e180d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.339+0000 7f1e1d7e6640 1 -- 172.21.15.82:0/2245665320 >> v1:172.21.15.82:6789/0 conn(0x7f1e180d07b0 legacy=0x7f1e180b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.339+0000 7f1e1d7e6640 1 -- 172.21.15.82:0/2245665320 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180b5920 con 0x7f1e1815e180 2024-03-14T12:16:29.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.339+0000 7f1e0ffff640 1 -- 172.21.15.82:0/2245665320 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459513665 0 0) 0x7f1e180b5080 con 0x7f1e180d07b0 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.339+0000 7f1e1d7e6640 1 --2- 172.21.15.82:0/2245665320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815e180 0x7f1e180b5e40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009e80 tx=0x7f1e1402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96a85921b275f3b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e0ffff640 1 -- 172.21.15.82:0/2245665320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1402cb20 con 0x7f1e1815e180 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e0ffff640 1 -- 172.21.15.82:0/2245665320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1402ccc0 con 0x7f1e1815e180 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e0ffff640 1 -- 172.21.15.82:0/2245665320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14034610 con 0x7f1e1815e180 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/2245665320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815e180 msgr2=0x7f1e180b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/2245665320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815e180 0x7f1e180b5e40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009e80 tx=0x7f1e1402bf80 comp rx=0 tx=0).stop 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/2245665320 shutdown_connections 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/2245665320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815e180 0x7f1e180b5e40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/2245665320 >> 172.21.15.82:0/2245665320 conn(0x7f1e180721a0 msgr2=0x7f1e180725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:29.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/2245665320 shutdown_connections 2024-03-14T12:16:29.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.340+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/2245665320 wait complete. 2024-03-14T12:16:29.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.341+0000 7f1e1fa4a640 1 Processor -- start 2024-03-14T12:16:29.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1fa4a640 1 -- start start 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1fa4a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e18066a20 0x7f1e180c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1fa4a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d07b0 0x7f1e180999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1fa4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1815e180 0x7f1e180cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1fa4a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180afa50 con 0x7f1e180d07b0 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1fa4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e18072cc0 con 0x7f1e1815e180 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1fa4a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e181f48b0 con 0x7f1e18066a20 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1cfe5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d07b0 0x7f1e180999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1cfe5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d07b0 0x7f1e180999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38936/0 (socket says 172.21.15.82:38936) 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1cfe5640 1 -- 172.21.15.82:0/224968623 learned_addr learned my addr 172.21.15.82:0/224968623 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1d7e6640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e18066a20 0x7f1e180c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.342+0000 7f1e1dfe7640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1815e180 0x7f1e180cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1cfe5640 1 -- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e18066a20 msgr2=0x7f1e180c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1cfe5640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e18066a20 0x7f1e180c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1cfe5640 1 -- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1815e180 msgr2=0x7f1e180cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1cfe5640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1815e180 0x7f1e180cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1cfe5640 1 -- 172.21.15.82:0/224968623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180d0220 con 0x7f1e180d07b0 2024-03-14T12:16:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1cfe5640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d07b0 0x7f1e180999a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c460 tx=0x7f1e0800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e0dffb640 1 -- 172.21.15.82:0/224968623 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08016020 con 0x7f1e180d07b0 2024-03-14T12:16:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e0dffb640 1 -- 172.21.15.82:0/224968623 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08012870 con 0x7f1e180d07b0 2024-03-14T12:16:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e0dffb640 1 -- 172.21.15.82:0/224968623 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08012c60 con 0x7f1e180d07b0 2024-03-14T12:16:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e181f5030 con 0x7f1e180d07b0 2024-03-14T12:16:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.343+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e180cd4c0 con 0x7f1e180d07b0 2024-03-14T12:16:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.344+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1804f050 con 0x7f1e180d07b0 2024-03-14T12:16:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.344+0000 7f1e0dffb640 1 -- 172.21.15.82:0/224968623 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1e08019020 con 0x7f1e180d07b0 2024-03-14T12:16:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.345+0000 7f1e0dffb640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1dfc06bbd0 0x7f1dfc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.345+0000 7f1e0dffb640 1 -- 172.21.15.82:0/224968623 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e0808a480 con 0x7f1e180d07b0 2024-03-14T12:16:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.345+0000 7f1e1d7e6640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1dfc06bbd0 0x7f1dfc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.345+0000 7f1e1d7e6640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1dfc06bbd0 0x7f1dfc06e090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1e14035000 tx=0x7f1e14030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.349+0000 7f1e0dffb640 1 -- 172.21.15.82:0/224968623 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1e08020030 con 0x7f1e180d07b0 2024-03-14T12:16:29.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.523+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e180cd690 con 0x7f1e180d07b0 2024-03-14T12:16:29.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.524+0000 7f1e0dffb640 1 -- 172.21.15.82:0/224968623 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f1e180cd690 con 0x7f1e180d07b0 2024-03-14T12:16:29.524 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:29.524 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":54,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:28.850638+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.525+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1dfc06bbd0 msgr2=0x7f1dfc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.525+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1dfc06bbd0 0x7f1dfc06e090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1e14035000 tx=0x7f1e14030000 comp rx=0 tx=0).stop 2024-03-14T12:16:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.525+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d07b0 msgr2=0x7f1e180999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.525+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d07b0 0x7f1e180999a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c460 tx=0x7f1e0800c920 comp rx=0 tx=0).stop 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.525+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 shutdown_connections 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.525+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1815e180 0x7f1e180cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.525+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e180d07b0 0x7f1e180999a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.526+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1dfc06bbd0 0x7f1dfc06e090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.526+0000 7f1e1fa4a640 1 --2- 172.21.15.82:0/224968623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e18066a20 0x7f1e180c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.526+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 >> 172.21.15.82:0/224968623 conn(0x7f1e180721a0 msgr2=0x7f1e180ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.526+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 shutdown_connections 2024-03-14T12:16:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.526+0000 7f1e1fa4a640 1 -- 172.21.15.82:0/224968623 wait complete. 2024-03-14T12:16:29.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:29.537 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:29.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.640+0000 7f0e66ade640 1 Processor -- start 2024-03-14T12:16:29.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.640+0000 7f0e66ade640 1 -- start start 2024-03-14T12:16:29.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e66ade640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e600d0db0 0x7f0e60162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e66ade640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e600b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e66ade640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6006e8e0 con 0x7f0e601478b0 2024-03-14T12:16:29.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e66ade640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6006eb80 con 0x7f0e600d0db0 2024-03-14T12:16:29.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e66ade640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6006f100 con 0x7f0e60147360 2024-03-14T12:16:29.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e6487a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e600d0db0 0x7f0e60162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e6487a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e600d0db0 0x7f0e60162560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38950/0 (socket says 172.21.15.82:38950) 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e6487a640 1 -- 172.21.15.82:0/43714413 learned_addr learned my addr 172.21.15.82:0/43714413 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e5ffff640 1 --2- 172.21.15.82:0/43714413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e600b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e6487a640 1 -- 172.21.15.82:0/43714413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 msgr2=0x7f0e600b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e6487a640 1 --2- 172.21.15.82:0/43714413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e600b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e6487a640 1 -- 172.21.15.82:0/43714413 >> v1:172.21.15.164:6789/0 conn(0x7f0e601478b0 legacy=0x7f0e6015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e6487a640 1 -- 172.21.15.82:0/43714413 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6006ee20 con 0x7f0e600d0db0 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.641+0000 7f0e5ffff640 1 --2- 172.21.15.82:0/43714413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e600b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e6487a640 1 --2- 172.21.15.82:0/43714413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e600d0db0 0x7f0e60162560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009870 tx=0x7f0e5402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309bf5f8dbf43a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e5f7fe640 1 -- 172.21.15.82:0/43714413 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540317c0 con 0x7f0e600d0db0 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e5f7fe640 1 -- 172.21.15.82:0/43714413 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54037070 con 0x7f0e600d0db0 2024-03-14T12:16:29.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e5f7fe640 1 -- 172.21.15.82:0/43714413 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54031b80 con 0x7f0e600d0db0 2024-03-14T12:16:29.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 -- 172.21.15.82:0/43714413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e600d0db0 msgr2=0x7f0e60162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 --2- 172.21.15.82:0/43714413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e600d0db0 0x7f0e60162560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009870 tx=0x7f0e5402b860 comp rx=0 tx=0).stop 2024-03-14T12:16:29.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 -- 172.21.15.82:0/43714413 shutdown_connections 2024-03-14T12:16:29.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 --2- 172.21.15.82:0/43714413 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e600b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 --2- 172.21.15.82:0/43714413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e600d0db0 0x7f0e60162560 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 -- 172.21.15.82:0/43714413 >> 172.21.15.82:0/43714413 conn(0x7f0e600b4fd0 msgr2=0x7f0e600b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 -- 172.21.15.82:0/43714413 shutdown_connections 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.642+0000 7f0e66ade640 1 -- 172.21.15.82:0/43714413 wait complete. 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.644+0000 7f0e66ade640 1 Processor -- start 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.644+0000 7f0e66ade640 1 -- start start 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.644+0000 7f0e66ade640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600d0db0 0x7f0e601f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.644+0000 7f0e66ade640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e6009d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.644+0000 7f0e66ade640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e601478b0 0x7f0e6009ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.644+0000 7f0e66ade640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60162aa0 con 0x7f0e601478b0 2024-03-14T12:16:29.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e66ade640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60162040 con 0x7f0e600d0db0 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e66ade640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6006d330 con 0x7f0e60147360 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e601478b0 0x7f0e6009ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e601478b0 0x7f0e6009ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38966/0 (socket says 172.21.15.82:38966) 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 -- 172.21.15.82:0/2880645613 learned_addr learned my addr 172.21.15.82:0/2880645613 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e5ffff640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e6009d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6487a640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600d0db0 0x7f0e601f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 -- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 msgr2=0x7f0e6009d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e6009d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 -- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600d0db0 msgr2=0x7f0e601f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600d0db0 0x7f0e601f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 -- 172.21.15.82:0/2880645613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60150100 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6487a640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600d0db0 0x7f0e601f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e6507b640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e601478b0 0x7f0e6009ddb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800c780 tx=0x7f0e5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e5d7fa640 1 -- 172.21.15.82:0/2880645613 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58004b60 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e5d7fa640 1 -- 172.21.15.82:0/2880645613 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e58004d00 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e5d7fa640 1 -- 172.21.15.82:0/2880645613 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800aa90 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e601503e0 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.645+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e601f4270 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.646+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e60071c40 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.647+0000 7f0e5d7fa640 1 -- 172.21.15.82:0/2880645613 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0e5800ac30 con 0x7f0e601478b0 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.647+0000 7f0e5d7fa640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0e4406bbd0 0x7f0e4406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.647+0000 7f0e5d7fa640 1 -- 172.21.15.82:0/2880645613 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0e5808a9f0 con 0x7f0e601478b0 2024-03-14T12:16:29.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.647+0000 7f0e6487a640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0e4406bbd0 0x7f0e4406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.648+0000 7f0e6487a640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0e4406bbd0 0x7f0e4406e090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e54036000 tx=0x7f0e5403d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.651+0000 7f0e5d7fa640 1 -- 172.21.15.82:0/2880645613 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0e60071c40 con 0x7f0e601478b0 2024-03-14T12:16:29.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.821+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e60150900 con 0x7f0e601478b0 2024-03-14T12:16:29.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.822+0000 7f0e5d7fa640 1 -- 172.21.15.82:0/2880645613 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e58020070 con 0x7f0e601478b0 2024-03-14T12:16:29.822 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.823+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0e4406bbd0 msgr2=0x7f0e4406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.823+0000 7f0e66ade640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0e4406bbd0 0x7f0e4406e090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e54036000 tx=0x7f0e5403d040 comp rx=0 tx=0).stop 2024-03-14T12:16:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.823+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e601478b0 msgr2=0x7f0e6009ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.823+0000 7f0e66ade640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e601478b0 0x7f0e6009ddb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800c780 tx=0x7f0e5800cc40 comp rx=0 tx=0).stop 2024-03-14T12:16:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 shutdown_connections 2024-03-14T12:16:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e601478b0 0x7f0e6009ddb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0e4406bbd0 0x7f0e4406e090 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e60147360 0x7f0e6009d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 --2- 172.21.15.82:0/2880645613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e600d0db0 0x7f0e601f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 >> 172.21.15.82:0/2880645613 conn(0x7f0e600b4fd0 msgr2=0x7f0e6015fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 shutdown_connections 2024-03-14T12:16:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.824+0000 7f0e66ade640 1 -- 172.21.15.82:0/2880645613 wait complete. 2024-03-14T12:16:29.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T12:16:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.936+0000 7fbe15905640 1 Processor -- start 2024-03-14T12:16:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.936+0000 7fbe15905640 1 -- start start 2024-03-14T12:16:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.936+0000 7fbe15905640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.936+0000 7fbe15905640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe100cf370 0x7fbe100d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.936+0000 7fbe15905640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe100d1080 con 0x7fbe100d1da0 2024-03-14T12:16:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.936+0000 7fbe15905640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe100cf8c0 con 0x7fbe100cf370 2024-03-14T12:16:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.936+0000 7fbe15905640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100cfba0 con 0x7fbe100d0760 2024-03-14T12:16:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0e7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe100cf370 0x7fbe100d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0e7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe100cf370 0x7fbe100d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38982/0 (socket says 172.21.15.82:38982) 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0e7fc640 1 -- 172.21.15.82:0/1880394357 learned_addr learned my addr 172.21.15.82:0/1880394357 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0effd640 1 --2- 172.21.15.82:0/1880394357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0e7fc640 1 -- 172.21.15.82:0/1880394357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 msgr2=0x7fbe100d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0e7fc640 1 --2- 172.21.15.82:0/1880394357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0e7fc640 1 -- 172.21.15.82:0/1880394357 >> v1:172.21.15.164:6789/0 conn(0x7fbe100d1da0 legacy=0x7fbe100d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.937+0000 7fbe0e7fc640 1 -- 172.21.15.82:0/1880394357 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe100b6c40 con 0x7fbe100cf370 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe0e7fc640 1 --2- 172.21.15.82:0/1880394357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100cf370 0x7fbe100d4660 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009870 tx=0x7fbe0402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748ad39b3a6b7d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe0dffb640 1 -- 172.21.15.82:0/1880394357 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0402f840 con 0x7fbe100cf370 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe0dffb640 1 -- 172.21.15.82:0/1880394357 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0402f9e0 con 0x7fbe100cf370 2024-03-14T12:16:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe0dffb640 1 -- 172.21.15.82:0/1880394357 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0402fdb0 con 0x7fbe100cf370 2024-03-14T12:16:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 -- 172.21.15.82:0/1880394357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100cf370 msgr2=0x7fbe100d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 --2- 172.21.15.82:0/1880394357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100cf370 0x7fbe100d4660 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009870 tx=0x7fbe0402ba20 comp rx=0 tx=0).stop 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 -- 172.21.15.82:0/1880394357 shutdown_connections 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 --2- 172.21.15.82:0/1880394357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100cf370 0x7fbe100d4660 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 --2- 172.21.15.82:0/1880394357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 -- 172.21.15.82:0/1880394357 >> 172.21.15.82:0/1880394357 conn(0x7fbe1009d030 msgr2=0x7fbe100d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 -- 172.21.15.82:0/1880394357 shutdown_connections 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.938+0000 7fbe15905640 1 -- 172.21.15.82:0/1880394357 wait complete. 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.939+0000 7fbe15905640 1 Processor -- start 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe15905640 1 -- start start 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe15905640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100cf370 0x7fbe100cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe15905640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe15905640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100d1da0 0x7fbe1015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe15905640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100d4ba0 con 0x7fbe100d1da0 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe15905640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100b7200 con 0x7fbe100cf370 2024-03-14T12:16:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe15905640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe101617a0 con 0x7fbe100d0760 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.940+0000 7fbe0f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100d1da0 0x7fbe1015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100d1da0 0x7fbe1015d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38994/0 (socket says 172.21.15.82:38994) 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 -- 172.21.15.82:0/422368645 learned_addr learned my addr 172.21.15.82:0/422368645 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0e7fc640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0effd640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100cf370 0x7fbe100cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 -- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 msgr2=0x7fbe100cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 -- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100cf370 msgr2=0x7fbe100cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100cf370 0x7fbe100cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 -- 172.21.15.82:0/422368645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1015dbe0 con 0x7fbe100d1da0 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0e7fc640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0effd640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100cf370 0x7fbe100cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe0f7fe640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100d1da0 0x7fbe1015d6a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c730 tx=0x7fbe0800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbdf7fff640 1 -- 172.21.15.82:0/422368645 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08016020 con 0x7fbe100d1da0 2024-03-14T12:16:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbdf7fff640 1 -- 172.21.15.82:0/422368645 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe08006d50 con 0x7fbe100d1da0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe100cc790 con 0x7fbe100d1da0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbdf7fff640 1 -- 172.21.15.82:0/422368645 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08003bb0 con 0x7fbe100d1da0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.941+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe100ccd50 con 0x7fbe100d1da0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.942+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1004f050 con 0x7fbe100d1da0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.943+0000 7fbdf7fff640 1 -- 172.21.15.82:0/422368645 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbe080050d0 con 0x7fbe100d1da0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.943+0000 7fbdf7fff640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbdfc06bbd0 0x7fbdfc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.943+0000 7fbdf7fff640 1 -- 172.21.15.82:0/422368645 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbe0808a2b0 con 0x7fbe100d1da0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.943+0000 7fbe0effd640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbdfc06bbd0 0x7fbdfc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.943+0000 7fbe0effd640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbdfc06bbd0 0x7fbdfc06e090 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe00006fd0 tx=0x7fbe00008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:29.947+0000 7fbdf7fff640 1 -- 172.21.15.82:0/422368645 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbe08020020 con 0x7fbe100d1da0 2024-03-14T12:16:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.117+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbe100d0540 con 0x7fbe100d1da0 2024-03-14T12:16:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.118+0000 7fbdf7fff640 1 -- 172.21.15.82:0/422368645 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbe08059180 con 0x7fbe100d1da0 2024-03-14T12:16:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:30.118 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-03-14T12:16:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-03-14T12:16:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-03-14T12:16:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:30.119 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-03-14T12:16:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbdfc06bbd0 msgr2=0x7fbdfc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbdfc06bbd0 0x7fbdfc06e090 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe00006fd0 tx=0x7fbe00008040 comp rx=0 tx=0).stop 2024-03-14T12:16:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100d1da0 msgr2=0x7fbe1015d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100d1da0 0x7fbe1015d6a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c730 tx=0x7fbe0800cb30 comp rx=0 tx=0).stop 2024-03-14T12:16:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 shutdown_connections 2024-03-14T12:16:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe100d1da0 0x7fbe1015d6a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbdfc06bbd0 0x7fbdfc06e090 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe100d0760 0x7fbe100cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 --2- 172.21.15.82:0/422368645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100cf370 0x7fbe100cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.119+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 >> 172.21.15.82:0/422368645 conn(0x7fbe1009d030 msgr2=0x7fbe1006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.120+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 shutdown_connections 2024-03-14T12:16:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.120+0000 7fbe15905640 1 -- 172.21.15.82:0/422368645 wait complete. 2024-03-14T12:16:30.130 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T12:16:30.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T12:16:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.234+0000 7fec87e9a640 1 Processor -- start 2024-03-14T12:16:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.234+0000 7fec87e9a640 1 -- start start 2024-03-14T12:16:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec87e9a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec8015dfd0 0x7fec800ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec87e9a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec800cfaa0 0x7fec8006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec87e9a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec800cfef0 con 0x7fec8006bf80 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec87e9a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8006ad10 con 0x7fec8015dfd0 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec87e9a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec800ae4f0 con 0x7fec800cfaa0 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec85c36640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec8015dfd0 0x7fec800ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec85c36640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec8015dfd0 0x7fec800ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39008/0 (socket says 172.21.15.82:39008) 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec85c36640 1 -- 172.21.15.82:0/4228154606 learned_addr learned my addr 172.21.15.82:0/4228154606 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec85c36640 1 -- 172.21.15.82:0/4228154606 >> v2:172.21.15.164:3300/0 conn(0x7fec800cfaa0 msgr2=0x7fec8006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec85c36640 1 --2- 172.21.15.82:0/4228154606 >> v2:172.21.15.164:3300/0 conn(0x7fec800cfaa0 0x7fec8006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec85c36640 1 -- 172.21.15.82:0/4228154606 >> v1:172.21.15.164:6789/0 conn(0x7fec8006bf80 legacy=0x7fec8006cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.235+0000 7fec85c36640 1 -- 172.21.15.82:0/4228154606 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8006af60 con 0x7fec8015dfd0 2024-03-14T12:16:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.236+0000 7fec85c36640 1 --2- 172.21.15.82:0/4228154606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec800ade30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fec800d0eb0 tx=0x7fec7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0854eadbe09441 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.236+0000 7fec84c34640 1 -- 172.21.15.82:0/4228154606 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c035020 con 0x7fec8015dfd0 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.236+0000 7fec84c34640 1 -- 172.21.15.82:0/4228154606 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c031870 con 0x7fec8015dfd0 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.236+0000 7fec84c34640 1 -- 172.21.15.82:0/4228154606 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c031c40 con 0x7fec8015dfd0 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 -- 172.21.15.82:0/4228154606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 msgr2=0x7fec800ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 --2- 172.21.15.82:0/4228154606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec800ade30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fec800d0eb0 tx=0x7fec7c02b910 comp rx=0 tx=0).stop 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 -- 172.21.15.82:0/4228154606 shutdown_connections 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 --2- 172.21.15.82:0/4228154606 >> v2:172.21.15.164:3300/0 conn(0x7fec800cfaa0 0x7fec8006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 --2- 172.21.15.82:0/4228154606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec800ade30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 -- 172.21.15.82:0/4228154606 >> 172.21.15.82:0/4228154606 conn(0x7fec800d0110 msgr2=0x7fec800d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 -- 172.21.15.82:0/4228154606 shutdown_connections 2024-03-14T12:16:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.237+0000 7fec87e9a640 1 -- 172.21.15.82:0/4228154606 wait complete. 2024-03-14T12:16:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.238+0000 7fec87e9a640 1 Processor -- start 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.238+0000 7fec87e9a640 1 -- start start 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec87e9a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec8006bf80 0x7fec8015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec87e9a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec800cfaa0 0x7fec8015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec87e9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec801e8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec87e9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8015e3b0 con 0x7fec8015dfd0 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec87e9a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8006b520 con 0x7fec800cfaa0 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec87e9a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8015e530 con 0x7fec8006bf80 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec801e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec801e8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39024/0 (socket says 172.21.15.82:39024) 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 -- 172.21.15.82:0/130541557 learned_addr learned my addr 172.21.15.82:0/130541557 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec85c36640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec8006bf80 0x7fec8015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec85435640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec800cfaa0 0x7fec8015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 -- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec8006bf80 msgr2=0x7fec8015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec8006bf80 0x7fec8015fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 -- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec800cfaa0 msgr2=0x7fec8015a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec800cfaa0 0x7fec8015a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 -- 172.21.15.82:0/130541557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80160010 con 0x7fec8015dfd0 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.239+0000 7fec86437640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec801e8300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fec7800b9c0 tx=0x7fec7800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.240+0000 7fec767fc640 1 -- 172.21.15.82:0/130541557 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78019070 con 0x7fec8015dfd0 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.240+0000 7fec767fc640 1 -- 172.21.15.82:0/130541557 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7800cba0 con 0x7fec8015dfd0 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.240+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8015ea30 con 0x7fec8015dfd0 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.240+0000 7fec767fc640 1 -- 172.21.15.82:0/130541557 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78014440 con 0x7fec8015dfd0 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.240+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8015eff0 con 0x7fec8015dfd0 2024-03-14T12:16:30.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.240+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec80072720 con 0x7fec8015dfd0 2024-03-14T12:16:30.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.241+0000 7fec767fc640 1 -- 172.21.15.82:0/130541557 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fec7801c070 con 0x7fec8015dfd0 2024-03-14T12:16:30.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.241+0000 7fec767fc640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec6406bbd0 0x7fec6406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.241+0000 7fec85c36640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec6406bbd0 0x7fec6406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.241+0000 7fec767fc640 1 -- 172.21.15.82:0/130541557 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec7808a480 con 0x7fec8015dfd0 2024-03-14T12:16:30.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.242+0000 7fec85c36640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec6406bbd0 0x7fec6406e090 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec7c036000 tx=0x7fec7c03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:30.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.246+0000 7fec767fc640 1 -- 172.21.15.82:0/130541557 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fec80072720 con 0x7fec8015dfd0 2024-03-14T12:16:30.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.415+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fec8004f050 con 0x7fec8015dfd0 2024-03-14T12:16:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.860+0000 7fec767fc640 1 -- 172.21.15.82:0/130541557 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7fec780592c0 con 0x7fec8015dfd0 2024-03-14T12:16:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec6406bbd0 msgr2=0x7fec6406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec6406bbd0 0x7fec6406e090 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec7c036000 tx=0x7fec7c03c040 comp rx=0 tx=0).stop 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 msgr2=0x7fec801e8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec801e8300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fec7800b9c0 tx=0x7fec7800be80 comp rx=0 tx=0).stop 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 shutdown_connections 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8015dfd0 0x7fec801e8300 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec800cfaa0 0x7fec8015a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec6406bbd0 0x7fec6406e090 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 --2- 172.21.15.82:0/130541557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec8006bf80 0x7fec8015fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 >> 172.21.15.82:0/130541557 conn(0x7fec800d0110 msgr2=0x7fec800b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:30.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 shutdown_connections 2024-03-14T12:16:30.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.862+0000 7fec87e9a640 1 -- 172.21.15.82:0/130541557 wait complete. 2024-03-14T12:16:30.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T12:16:30.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.984+0000 7f9e53d74640 1 Processor -- start 2024-03-14T12:16:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.984+0000 7f9e53d74640 1 -- start start 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e53d74640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e4c145400 0x7f9e4c0723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e53d74640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c0714e0 con 0x7f9e4c15f800 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e53d74640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c1457e0 con 0x7f9e4c06d690 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e53d74640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c0ce800 con 0x7f9e4c145400 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e51b10640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e4c145400 0x7f9e4c0723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e51b10640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e4c145400 0x7f9e4c0723d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39040/0 (socket says 172.21.15.82:39040) 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e51b10640 1 -- 172.21.15.82:0/506909057 learned_addr learned my addr 172.21.15.82:0/506909057 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.985+0000 7f9e5130f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9e4c15f800 0x7f9e4c0706c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33582/0 (socket says 172.21.15.82:33582) 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.986+0000 7f9e51b10640 1 -- 172.21.15.82:0/506909057 >> v1:172.21.15.164:6789/0 conn(0x7f9e4c06d690 legacy=0x7f9e4c070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.986+0000 7f9e51b10640 1 -- 172.21.15.82:0/506909057 >> v1:172.21.15.82:6789/0 conn(0x7f9e4c15f800 legacy=0x7f9e4c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.986+0000 7f9e50b0e640 1 -- 172.21.15.82:0/506909057 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197152349 0 0) 0x7f9e4c0714e0 con 0x7f9e4c15f800 2024-03-14T12:16:30.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.986+0000 7f9e51b10640 1 -- 172.21.15.82:0/506909057 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0cea50 con 0x7f9e4c145400 2024-03-14T12:16:30.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.986+0000 7f9e51b10640 1 --2- 172.21.15.82:0/506909057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c145400 0x7f9e4c0723d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009e50 tx=0x7f9e4802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf168523c2ace13 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:30.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.986+0000 7f9e50b0e640 1 -- 172.21.15.82:0/506909057 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4802ca80 con 0x7f9e4c145400 2024-03-14T12:16:30.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e50b0e640 1 -- 172.21.15.82:0/506909057 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4802cc20 con 0x7f9e4c145400 2024-03-14T12:16:30.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e50b0e640 1 -- 172.21.15.82:0/506909057 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480343f0 con 0x7f9e4c145400 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e53d74640 1 -- 172.21.15.82:0/506909057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c145400 msgr2=0x7f9e4c0723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e53d74640 1 --2- 172.21.15.82:0/506909057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c145400 0x7f9e4c0723d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009e50 tx=0x7f9e4802bee0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e53d74640 1 -- 172.21.15.82:0/506909057 shutdown_connections 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e53d74640 1 --2- 172.21.15.82:0/506909057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c145400 0x7f9e4c0723d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e53d74640 1 -- 172.21.15.82:0/506909057 >> 172.21.15.82:0/506909057 conn(0x7f9e4c0cf910 msgr2=0x7f9e4c0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e53d74640 1 -- 172.21.15.82:0/506909057 shutdown_connections 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.987+0000 7f9e53d74640 1 -- 172.21.15.82:0/506909057 wait complete. 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.988+0000 7f9e53d74640 1 Processor -- start 2024-03-14T12:16:30.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.988+0000 7f9e53d74640 1 -- start start 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e53d74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e4c06d690 0x7f9e4c138540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e53d74640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e4c145400 0x7f9e4c1420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e53d74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c15f800 0x7f9e4c15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e53d74640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0cf2f0 con 0x7f9e4c15f800 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e53d74640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0cf010 con 0x7f9e4c06d690 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e53d74640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0cf5d0 con 0x7f9e4c145400 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c15f800 0x7f9e4c15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c15f800 0x7f9e4c15c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39048/0 (socket says 172.21.15.82:39048) 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 -- 172.21.15.82:0/3398307277 learned_addr learned my addr 172.21.15.82:0/3398307277 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e51b10640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e4c06d690 0x7f9e4c138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e5130f640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e4c145400 0x7f9e4c1420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 -- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e4c145400 msgr2=0x7f9e4c1420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e4c145400 0x7f9e4c1420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 -- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e4c06d690 msgr2=0x7f9e4c138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e4c06d690 0x7f9e4c138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.989+0000 7f9e52311640 1 -- 172.21.15.82:0/3398307277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c139aa0 con 0x7f9e4c15f800 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.990+0000 7f9e52311640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c15f800 0x7f9e4c15c0a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c410 tx=0x7f9e4400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.990+0000 7f9e427fc640 1 -- 172.21.15.82:0/3398307277 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44017070 con 0x7f9e4c15f800 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.990+0000 7f9e427fc640 1 -- 172.21.15.82:0/3398307277 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e44005230 con 0x7f9e4c15f800 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.990+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c139d80 con 0x7f9e4c15f800 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.990+0000 7f9e427fc640 1 -- 172.21.15.82:0/3398307277 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44011590 con 0x7f9e4c15f800 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.990+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c13a340 con 0x7f9e4c15f800 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.991+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c04f050 con 0x7f9e4c15f800 2024-03-14T12:16:30.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.992+0000 7f9e427fc640 1 -- 172.21.15.82:0/3398307277 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9e4401a070 con 0x7f9e4c15f800 2024-03-14T12:16:30.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.992+0000 7f9e427fc640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9e3006bbd0 0x7f9e3006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:30.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.992+0000 7f9e427fc640 1 -- 172.21.15.82:0/3398307277 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9e440898d0 con 0x7f9e4c15f800 2024-03-14T12:16:30.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.992+0000 7f9e51b10640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9e3006bbd0 0x7f9e3006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:30.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.992+0000 7f9e51b10640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9e3006bbd0 0x7f9e3006e090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e48035000 tx=0x7f9e48030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:30.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:30.997+0000 7f9e427fc640 1 -- 172.21.15.82:0/3398307277 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9e44057fe0 con 0x7f9e4c15f800 2024-03-14T12:16:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.169+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9e4c15f570 con 0x7f9e4c15f800 2024-03-14T12:16:31.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.865+0000 7f9e427fc640 1 -- 172.21.15.82:0/3398307277 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9e4405c080 con 0x7f9e4c15f800 2024-03-14T12:16:31.866 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-03-14T12:16:31.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.868+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9e3006bbd0 msgr2=0x7f9e3006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:31.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.868+0000 7f9e53d74640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9e3006bbd0 0x7f9e3006e090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e48035000 tx=0x7f9e48030000 comp rx=0 tx=0).stop 2024-03-14T12:16:31.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c15f800 msgr2=0x7f9e4c15c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:31.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c15f800 0x7f9e4c15c0a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c410 tx=0x7f9e4400c8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 shutdown_connections 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e4c15f800 0x7f9e4c15c0a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9e3006bbd0 0x7f9e3006e090 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e4c145400 0x7f9e4c1420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 --2- 172.21.15.82:0/3398307277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e4c06d690 0x7f9e4c138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 >> 172.21.15.82:0/3398307277 conn(0x7f9e4c0cf910 msgr2=0x7f9e4c15fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 shutdown_connections 2024-03-14T12:16:31.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:31.869+0000 7f9e53d74640 1 -- 172.21.15.82:0/3398307277 wait complete. 2024-03-14T12:16:31.894 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T12:16:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.001+0000 7fcd39220640 1 Processor -- start 2024-03-14T12:16:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.001+0000 7fcd39220640 1 -- start start 2024-03-14T12:16:32.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd39220640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd34140e70 0x7fcd340b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:32.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd39220640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcd341626f0 0x7fcd3415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:32.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd39220640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd340b41c0 con 0x7fcd34143550 2024-03-14T12:16:32.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd39220640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3415f3f0 con 0x7fcd34140e70 2024-03-14T12:16:32.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd39220640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd34161c10 con 0x7fcd341626f0 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd32d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd34140e70 0x7fcd340b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd32d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd34140e70 0x7fcd340b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50410/0 (socket says 172.21.15.82:50410) 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd32d9d640 1 -- 172.21.15.82:0/500375105 learned_addr learned my addr 172.21.15.82:0/500375105 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd3259c640 1 --2- 172.21.15.82:0/500375105 >> v2:172.21.15.164:3300/0 conn(0x7fcd341626f0 0x7fcd3415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd32d9d640 1 -- 172.21.15.82:0/500375105 >> v2:172.21.15.164:3300/0 conn(0x7fcd341626f0 msgr2=0x7fcd3415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd32d9d640 1 --2- 172.21.15.82:0/500375105 >> v2:172.21.15.164:3300/0 conn(0x7fcd341626f0 0x7fcd3415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd32d9d640 1 -- 172.21.15.82:0/500375105 >> v1:172.21.15.164:6789/0 conn(0x7fcd34143550 legacy=0x7fcd340af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.002+0000 7fcd32d9d640 1 -- 172.21.15.82:0/500375105 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34161ef0 con 0x7fcd34140e70 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.003+0000 7fcd32d9d640 1 --2- 172.21.15.82:0/500375105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34140e70 0x7fcd340b3b30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcd340b4a20 tx=0x7fcd2c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20eeb6582c506284 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.003+0000 7fcd31d9b640 1 -- 172.21.15.82:0/500375105 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c02cb20 con 0x7fcd34140e70 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.003+0000 7fcd31d9b640 1 -- 172.21.15.82:0/500375105 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c02ccc0 con 0x7fcd34140e70 2024-03-14T12:16:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.003+0000 7fcd31d9b640 1 -- 172.21.15.82:0/500375105 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c034610 con 0x7fcd34140e70 2024-03-14T12:16:32.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 -- 172.21.15.82:0/500375105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34140e70 msgr2=0x7fcd340b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:32.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 --2- 172.21.15.82:0/500375105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34140e70 0x7fcd340b3b30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcd340b4a20 tx=0x7fcd2c02bf80 comp rx=0 tx=0).stop 2024-03-14T12:16:32.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 -- 172.21.15.82:0/500375105 shutdown_connections 2024-03-14T12:16:32.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 --2- 172.21.15.82:0/500375105 >> v2:172.21.15.164:3300/0 conn(0x7fcd341626f0 0x7fcd3415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 --2- 172.21.15.82:0/500375105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34140e70 0x7fcd340b3b30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 -- 172.21.15.82:0/500375105 >> 172.21.15.82:0/500375105 conn(0x7fcd340b2830 msgr2=0x7fcd340b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:32.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 -- 172.21.15.82:0/500375105 shutdown_connections 2024-03-14T12:16:32.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.004+0000 7fcd39220640 1 -- 172.21.15.82:0/500375105 wait complete. 2024-03-14T12:16:32.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.005+0000 7fcd39220640 1 Processor -- start 2024-03-14T12:16:32.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.005+0000 7fcd39220640 1 -- start start 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd39220640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd34140e70 0x7fcd34130390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd39220640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34143550 0x7fcd340ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd39220640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd341626f0 0x7fcd340add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd39220640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34143b40 con 0x7fcd34143550 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd39220640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd341439c0 con 0x7fcd341626f0 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd39220640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34143cc0 con 0x7fcd34140e70 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34143550 0x7fcd340ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34143550 0x7fcd340ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50418/0 (socket says 172.21.15.82:50418) 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 -- 172.21.15.82:0/1404179373 learned_addr learned my addr 172.21.15.82:0/1404179373 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd32d9d640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd34140e70 0x7fcd34130390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3359e640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd341626f0 0x7fcd340add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 -- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd34140e70 msgr2=0x7fcd34130390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd34140e70 0x7fcd34130390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 -- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd341626f0 msgr2=0x7fcd340add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd341626f0 0x7fcd340add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 -- 172.21.15.82:0/1404179373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34133380 con 0x7fcd34143550 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.006+0000 7fcd3259c640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34143550 0x7fcd340ad850 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d5f0 tx=0x7fcd2800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.007+0000 7fcd1b7fe640 1 -- 172.21.15.82:0/1404179373 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280125f0 con 0x7fcd34143550 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.007+0000 7fcd1b7fe640 1 -- 172.21.15.82:0/1404179373 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28019070 con 0x7fcd34143550 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.007+0000 7fcd1b7fe640 1 -- 172.21.15.82:0/1404179373 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280129d0 con 0x7fcd34143550 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.007+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd341330a0 con 0x7fcd34143550 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.007+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd340ce950 con 0x7fcd34143550 2024-03-14T12:16:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.008+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3404f050 con 0x7fcd34143550 2024-03-14T12:16:32.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.008+0000 7fcd1b7fe640 1 -- 172.21.15.82:0/1404179373 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcd28012b70 con 0x7fcd34143550 2024-03-14T12:16:32.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.008+0000 7fcd1b7fe640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2006bc00 0x7fcd2006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:32.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.009+0000 7fcd1b7fe640 1 -- 172.21.15.82:0/1404179373 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fcd2808ab90 con 0x7fcd34143550 2024-03-14T12:16:32.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.009+0000 7fcd32d9d640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2006bc00 0x7fcd2006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:32.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.009+0000 7fcd32d9d640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2006bc00 0x7fcd2006e0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c035000 tx=0x7fcd2c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:32.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.013+0000 7fcd1b7fe640 1 -- 172.21.15.82:0/1404179373 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fcd28059120 con 0x7fcd34143550 2024-03-14T12:16:32.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.220+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcd340b3700 con 0x7fcd34143550 2024-03-14T12:16:32.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.867+0000 7fcd1b7fe640 1 -- 172.21.15.82:0/1404179373 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v25) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcd2805d1c0 con 0x7fcd34143550 2024-03-14T12:16:32.867 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 8 and data pool 9 2024-03-14T12:16:32.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2006bc00 msgr2=0x7fcd2006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:32.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2006bc00 0x7fcd2006e0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c035000 tx=0x7fcd2c030000 comp rx=0 tx=0).stop 2024-03-14T12:16:32.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34143550 msgr2=0x7fcd340ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:32.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34143550 0x7fcd340ad850 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d5f0 tx=0x7fcd2800dab0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 shutdown_connections 2024-03-14T12:16:32.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd341626f0 0x7fcd340add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34143550 0x7fcd340ad850 secure :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d5f0 tx=0x7fcd2800dab0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2006bc00 0x7fcd2006e0c0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c035000 tx=0x7fcd2c030000 comp rx=0 tx=0).stop 2024-03-14T12:16:32.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 --2- 172.21.15.82:0/1404179373 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd34140e70 0x7fcd34130390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:32.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.872+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 >> 172.21.15.82:0/1404179373 conn(0x7fcd340b2830 msgr2=0x7fcd34161170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:32.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.881+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 shutdown_connections 2024-03-14T12:16:32.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:32.882+0000 7fcd39220640 1 -- 172.21.15.82:0/1404179373 wait complete. 2024-03-14T12:16:32.907 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.008+0000 7f62b53c4640 1 Processor -- start 2024-03-14T12:16:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.008+0000 7f62b53c4640 1 -- start start 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.009+0000 7f62b53c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f62b015f1f0 0x7f62b006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.009+0000 7f62b53c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b00b01d0 con 0x7f62b00b58f0 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.009+0000 7f62b53c4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b015e710 con 0x7f62b00af150 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.009+0000 7f62b53c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b00aff90 con 0x7f62b015f1f0 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.009+0000 7f62af7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f62b00b58f0 0x7f62b0157cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41070/0 (socket says 172.21.15.82:41070) 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.009+0000 7f62af7fe640 1 -- 172.21.15.82:0/434117295 learned_addr learned my addr 172.21.15.82:0/434117295 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336774501 0 0) 0x7f62b00b01d0 con 0x7f62b00b58f0 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6298003520 con 0x7f62b00b58f0 2024-03-14T12:16:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62ae7fc640 1 --2- 172.21.15.82:0/434117295 >> v2:172.21.15.164:3300/0 conn(0x7f62b015f1f0 0x7f62b006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1572628235 0 0) 0x7f6298003520 con 0x7f62b00b58f0 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 >> v2:172.21.15.164:3300/0 conn(0x7f62b015f1f0 msgr2=0x7f62b006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 --2- 172.21.15.82:0/434117295 >> v2:172.21.15.164:3300/0 conn(0x7f62b015f1f0 0x7f62b006ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 >> v1:172.21.15.164:6789/0 conn(0x7f62b00af150 legacy=0x7f62b00af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b015e9f0 con 0x7f62b00b58f0 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f62a8001670 con 0x7f62b00b58f0 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 >> v1:172.21.15.82:6789/0 conn(0x7f62b00b58f0 legacy=0x7f62b0157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62980039b0 0x7f6298003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6298004800 0x7f6298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62980051c0 0x7f629800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00aff90 con 0x7f62980039b0 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629800c160 con 0x7f62980051c0 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629800c360 con 0x7f6298004800 2024-03-14T12:16:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62aeffd640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62980039b0 0x7f6298003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62ae7fc640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6298004800 0x7f6298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62af7fe640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62980051c0 0x7f629800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62aeffd640 1 -- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6298004800 msgr2=0x7f6298004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62aeffd640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6298004800 0x7f6298004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62aeffd640 1 -- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62980051c0 msgr2=0x7f629800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62aeffd640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62980051c0 0x7f629800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62aeffd640 1 -- 172.21.15.82:0/434117295 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298003520 con 0x7f62980039b0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62af7fe640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62980051c0 0x7f629800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.010+0000 7f62af7fe640 1 -- 172.21.15.82:0/434117295 reap_dead start 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.011+0000 7f62aeffd640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62980039b0 0x7f6298003dd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f62a0001e50 tx=0x7f6298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5087f47059639339 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.011+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0001a70 con 0x7f62980039b0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.011+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a0003b70 con 0x7f62980039b0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.011+0000 7f62adffb640 1 -- 172.21.15.82:0/434117295 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00050d0 con 0x7f62980039b0 2024-03-14T12:16:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.012+0000 7f62b53c4640 1 -- 172.21.15.82:0/434117295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62980039b0 msgr2=0x7f6298003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.012+0000 7f62b53c4640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62980039b0 0x7f6298003dd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f62a0001e50 tx=0x7f6298005830 comp rx=0 tx=0).stop 2024-03-14T12:16:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.012+0000 7f62b53c4640 1 -- 172.21.15.82:0/434117295 shutdown_connections 2024-03-14T12:16:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.012+0000 7f62b53c4640 1 --2- 172.21.15.82:0/434117295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62980039b0 0x7f6298003dd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.012+0000 7f62b53c4640 1 -- 172.21.15.82:0/434117295 >> 172.21.15.82:0/434117295 conn(0x7f62b00d0a20 msgr2=0x7f62b00d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.012+0000 7f62b53c4640 1 -- 172.21.15.82:0/434117295 shutdown_connections 2024-03-14T12:16:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.012+0000 7f62b53c4640 1 -- 172.21.15.82:0/434117295 wait complete. 2024-03-14T12:16:33.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.013+0000 7f62b53c4640 1 Processor -- start 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.013+0000 7f62b53c4640 1 -- start start 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62b53c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62980039b0 0x7f62b01f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62b53c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b015b020 0x7f62b00cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62b53c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b01f6a20 0x7f62b00cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62b53c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00af880 con 0x7f62b01f6a20 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62b53c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b015e400 con 0x7f62b015b020 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62b53c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00afa00 con 0x7f62980039b0 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b01f6a20 0x7f62b00cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b01f6a20 0x7f62b00cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50432/0 (socket says 172.21.15.82:50432) 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 -- 172.21.15.82:0/102436360 learned_addr learned my addr 172.21.15.82:0/102436360 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62ae7fc640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b015b020 0x7f62b00cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62aeffd640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62980039b0 0x7f62b01f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 -- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62980039b0 msgr2=0x7f62b01f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62980039b0 0x7f62b01f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 -- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b015b020 msgr2=0x7f62b00cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b015b020 0x7f62b00cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 -- 172.21.15.82:0/102436360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b0158120 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62ae7fc640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b015b020 0x7f62b00cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62aeffd640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62980039b0 0x7f62b01f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62af7fe640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b01f6a20 0x7f62b00cee00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f62a8000fb0 tx=0x7f62a800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62977fe640 1 -- 172.21.15.82:0/102436360 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a800c590 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62977fe640 1 -- 172.21.15.82:0/102436360 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a8018070 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b0158400 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.014+0000 7f62977fe640 1 -- 172.21.15.82:0/102436360 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a800c950 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.015+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b01589c0 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.016+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b004f050 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.016+0000 7f62977fe640 1 -- 172.21.15.82:0/102436360 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f62a800caf0 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.016+0000 7f62977fe640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629c06bbd0 0x7f629c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.016+0000 7f62977fe640 1 -- 172.21.15.82:0/102436360 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62a808a6a0 con 0x7f62b01f6a20 2024-03-14T12:16:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.016+0000 7f62aeffd640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629c06bbd0 0x7f629c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.017+0000 7f62aeffd640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629c06bbd0 0x7f629c06e090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f62a0034000 tx=0x7f62a0009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.021+0000 7f62977fe640 1 -- 172.21.15.82:0/102436360 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f62a8059190 con 0x7f62b01f6a20 2024-03-14T12:16:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.194+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62b0158b90 con 0x7f62b01f6a20 2024-03-14T12:16:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.195+0000 7f62977fe640 1 -- 172.21.15.82:0/102436360 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f62a8058a90 con 0x7f62b01f6a20 2024-03-14T12:16:33.196 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:33.196 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":57,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:32.222493+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629c06bbd0 msgr2=0x7f629c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629c06bbd0 0x7f629c06e090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f62a0034000 tx=0x7f62a0009030 comp rx=0 tx=0).stop 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b01f6a20 msgr2=0x7f62b00cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b01f6a20 0x7f62b00cee00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f62a8000fb0 tx=0x7f62a800eb90 comp rx=0 tx=0).stop 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 shutdown_connections 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62b01f6a20 0x7f62b00cee00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62b015b020 0x7f62b00cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f629c06bbd0 0x7f629c06e090 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.197+0000 7f62b53c4640 1 --2- 172.21.15.82:0/102436360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62980039b0 0x7f62b01f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.198+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 >> 172.21.15.82:0/102436360 conn(0x7f62b00d0a20 msgr2=0x7f62b015f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:33.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.198+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 shutdown_connections 2024-03-14T12:16:33.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.198+0000 7f62b53c4640 1 -- 172.21.15.82:0/102436360 wait complete. 2024-03-14T12:16:33.209 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:33.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.309+0000 7f0c84929640 1 Processor -- start 2024-03-14T12:16:33.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.309+0000 7f0c84929640 1 -- start start 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c84929640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0c8015bf60 0x7f0c8006f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c84929640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8006ea90 con 0x7f0c8006ad10 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c84929640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8006ed30 con 0x7f0c800d1810 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c84929640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c80072000 con 0x7f0c8015bf60 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7ed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0c8006ad10 0x7f0c800b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41076/0 (socket says 172.21.15.82:41076) 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7ed9d640 1 -- 172.21.15.82:0/210195586 learned_addr learned my addr 172.21.15.82:0/210195586 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7e59c640 1 --2- 172.21.15.82:0/210195586 >> v2:172.21.15.164:3300/0 conn(0x7f0c8015bf60 0x7f0c8006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7dd9b640 1 -- 172.21.15.82:0/210195586 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075937780 0 0) 0x7f0c8006ea90 con 0x7f0c8006ad10 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7dd9b640 1 -- 172.21.15.82:0/210195586 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c003520 con 0x7f0c8006ad10 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7e59c640 1 -- 172.21.15.82:0/210195586 >> v1:172.21.15.164:6789/0 conn(0x7f0c800d1810 legacy=0x7f0c80072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7dd9b640 1 -- 172.21.15.82:0/210195586 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2159390297 0 0) 0x7f0c6c003520 con 0x7f0c8006ad10 2024-03-14T12:16:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7e59c640 1 -- 172.21.15.82:0/210195586 >> v1:172.21.15.82:6789/0 conn(0x7f0c8006ad10 legacy=0x7f0c800b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.310+0000 7f0c7e59c640 1 -- 172.21.15.82:0/210195586 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800719b0 con 0x7f0c8015bf60 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.311+0000 7f0c7e59c640 1 --2- 172.21.15.82:0/210195586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c8006f190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c700097a0 tx=0x7f0c7002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8168faf020c3a9a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.311+0000 7f0c7dd9b640 1 -- 172.21.15.82:0/210195586 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70037070 con 0x7f0c8015bf60 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.311+0000 7f0c7dd9b640 1 -- 172.21.15.82:0/210195586 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70030510 con 0x7f0c8015bf60 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.311+0000 7f0c7dd9b640 1 -- 172.21.15.82:0/210195586 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c780019f0 con 0x7f0c8015bf60 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.312+0000 7f0c84929640 1 -- 172.21.15.82:0/210195586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 msgr2=0x7f0c8006f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.312+0000 7f0c84929640 1 --2- 172.21.15.82:0/210195586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c8006f190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c700097a0 tx=0x7f0c7002b790 comp rx=0 tx=0).stop 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.312+0000 7f0c84929640 1 -- 172.21.15.82:0/210195586 shutdown_connections 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.312+0000 7f0c84929640 1 --2- 172.21.15.82:0/210195586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c8006f190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.312+0000 7f0c84929640 1 -- 172.21.15.82:0/210195586 >> 172.21.15.82:0/210195586 conn(0x7f0c800d0b60 msgr2=0x7f0c800d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.312+0000 7f0c84929640 1 -- 172.21.15.82:0/210195586 shutdown_connections 2024-03-14T12:16:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.312+0000 7f0c84929640 1 -- 172.21.15.82:0/210195586 wait complete. 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.313+0000 7f0c84929640 1 Processor -- start 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.313+0000 7f0c84929640 1 -- start start 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c84929640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c8006ad10 0x7f0c80149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c84929640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0c800d1810 0x7f0c80146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c84929640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c80147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c84929640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800b42c0 con 0x7f0c8006ad10 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c84929640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800b4140 con 0x7f0c8015bf60 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c84929640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800b4440 con 0x7f0c800d1810 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c8006ad10 0x7f0c80149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c8006ad10 0x7f0c80149330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50444/0 (socket says 172.21.15.82:50444) 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 -- 172.21.15.82:0/132647932 learned_addr learned my addr 172.21.15.82:0/132647932 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7ed9d640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c80147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c77fff640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0c800d1810 0x7f0c80146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 -- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0c800d1810 msgr2=0x7f0c80146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0c800d1810 0x7f0c80146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 -- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 msgr2=0x7f0c80147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c80147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7e59c640 1 -- 172.21.15.82:0/132647932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80146f10 con 0x7f0c8006ad10 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.314+0000 7f0c7ed9d640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c80147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.315+0000 7f0c7e59c640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c8006ad10 0x7f0c80149330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0c7002b7d0 tx=0x7f0c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.315+0000 7f0c76ffd640 1 -- 172.21.15.82:0/132647932 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70042070 con 0x7f0c8006ad10 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.315+0000 7f0c76ffd640 1 -- 172.21.15.82:0/132647932 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7003e870 con 0x7f0c8006ad10 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.315+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c801471f0 con 0x7f0c8006ad10 2024-03-14T12:16:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.315+0000 7f0c76ffd640 1 -- 172.21.15.82:0/132647932 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70037040 con 0x7f0c8006ad10 2024-03-14T12:16:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.315+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c801477b0 con 0x7f0c8006ad10 2024-03-14T12:16:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.316+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8004f050 con 0x7f0c8006ad10 2024-03-14T12:16:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.316+0000 7f0c76ffd640 1 -- 172.21.15.82:0/132647932 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0c70045070 con 0x7f0c8006ad10 2024-03-14T12:16:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.317+0000 7f0c76ffd640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c6006bbd0 0x7f0c6006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.317+0000 7f0c76ffd640 1 -- 172.21.15.82:0/132647932 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c700ac680 con 0x7f0c8006ad10 2024-03-14T12:16:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.317+0000 7f0c77fff640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c6006bbd0 0x7f0c6006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.317+0000 7f0c77fff640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c6006bbd0 0x7f0c6006e090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c800c76b0 tx=0x7f0c68009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.321+0000 7f0c76ffd640 1 -- 172.21.15.82:0/132647932 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0c7007b200 con 0x7f0c8006ad10 2024-03-14T12:16:33.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.492+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c800740b0 con 0x7f0c8006ad10 2024-03-14T12:16:33.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.493+0000 7f0c76ffd640 1 -- 172.21.15.82:0/132647932 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f0c7007ab00 con 0x7f0c8006ad10 2024-03-14T12:16:33.494 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:33.494 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":57,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:32.222493+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c6006bbd0 msgr2=0x7f0c6006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c6006bbd0 0x7f0c6006e090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c800c76b0 tx=0x7f0c68009040 comp rx=0 tx=0).stop 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c8006ad10 msgr2=0x7f0c80149330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c8006ad10 0x7f0c80149330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0c7002b7d0 tx=0x7f0c700046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 shutdown_connections 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c8015bf60 0x7f0c80147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0c6006bbd0 0x7f0c6006e090 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0c800d1810 0x7f0c80146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 --2- 172.21.15.82:0/132647932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c8006ad10 0x7f0c80149330 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 >> 172.21.15.82:0/132647932 conn(0x7f0c800d0b60 msgr2=0x7f0c800cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 shutdown_connections 2024-03-14T12:16:33.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.495+0000 7f0c84929640 1 -- 172.21.15.82:0/132647932 wait complete. 2024-03-14T12:16:33.507 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T12:16:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.610+0000 7fe2fe6ab640 1 Processor -- start 2024-03-14T12:16:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.610+0000 7fe2fe6ab640 1 -- start start 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2fe6ab640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2f81587f0 0x7fe2f80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2fe6ab640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2f80b4d20 0x7fe2f8071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2fe6ab640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f80721b0 con 0x7fe2f8071da0 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2fe6ab640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f806f6f0 con 0x7fe2f80b4d20 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2fe6ab640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f815f5f0 con 0x7fe2f81587f0 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2f80b4d20 0x7fe2f8071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2f80b4d20 0x7fe2f8071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50456/0 (socket says 172.21.15.82:50456) 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2fcc48640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe2f8071da0 0x7fe2f806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41086/0 (socket says 172.21.15.82:41086) 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f77fe640 1 -- 172.21.15.82:0/1332000663 learned_addr learned my addr 172.21.15.82:0/1332000663 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f7fff640 1 --2- 172.21.15.82:0/1332000663 >> v2:172.21.15.164:3300/0 conn(0x7fe2f81587f0 0x7fe2f80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f77fe640 1 -- 172.21.15.82:0/1332000663 >> v2:172.21.15.164:3300/0 conn(0x7fe2f81587f0 msgr2=0x7fe2f80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:33.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f77fe640 1 --2- 172.21.15.82:0/1332000663 >> v2:172.21.15.164:3300/0 conn(0x7fe2f81587f0 0x7fe2f80b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f77fe640 1 -- 172.21.15.82:0/1332000663 >> v1:172.21.15.82:6789/0 conn(0x7fe2f8071da0 legacy=0x7fe2f806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f77fe640 1 -- 172.21.15.82:0/1332000663 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f815e830 con 0x7fe2f80b4d20 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.611+0000 7fe2f6ffd640 1 -- 172.21.15.82:0/1332000663 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201896300 0 0) 0x7fe2f0001490 con 0x7fe2f8071da0 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2f77fe640 1 --2- 172.21.15.82:0/1332000663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f80b4d20 0x7fe2f8071860 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8073820 tx=0x7fe2e802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f0eef24feb8d0a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2f6ffd640 1 -- 172.21.15.82:0/1332000663 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e803b070 con 0x7fe2f80b4d20 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2f6ffd640 1 -- 172.21.15.82:0/1332000663 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e8036440 con 0x7fe2f80b4d20 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2f6ffd640 1 -- 172.21.15.82:0/1332000663 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8036810 con 0x7fe2f80b4d20 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/1332000663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f80b4d20 msgr2=0x7fe2f8071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/1332000663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f80b4d20 0x7fe2f8071860 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8073820 tx=0x7fe2e802e0e0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/1332000663 shutdown_connections 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.612+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/1332000663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f80b4d20 0x7fe2f8071860 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.613+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/1332000663 >> v2:172.21.15.164:3300/0 conn(0x7fe2f81587f0 0x7fe2f80b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.613+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/1332000663 >> 172.21.15.82:0/1332000663 conn(0x7fe2f80b2830 msgr2=0x7fe2f80b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:33.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.613+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/1332000663 shutdown_connections 2024-03-14T12:16:33.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.613+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/1332000663 wait complete. 2024-03-14T12:16:33.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.614+0000 7fe2fe6ab640 1 Processor -- start 2024-03-14T12:16:33.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.614+0000 7fe2fe6ab640 1 -- start start 2024-03-14T12:16:33.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.614+0000 7fe2fe6ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f8071da0 0x7fe2f8156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.614+0000 7fe2fe6ab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2f80b4d20 0x7fe2f80b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2fe6ab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2f81587f0 0x7fe2f8154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2fe6ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f815f0d0 con 0x7fe2f8071da0 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2fe6ab640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f815edf0 con 0x7fe2f81587f0 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2fe6ab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f815c1b0 con 0x7fe2f80b4d20 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f8071da0 0x7fe2f8156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f8071da0 0x7fe2f8156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50470/0 (socket says 172.21.15.82:50470) 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 -- 172.21.15.82:0/2807284440 learned_addr learned my addr 172.21.15.82:0/2807284440 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 -- 172.21.15.82:0/2807284440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2f80b4d20 msgr2=0x7fe2f80b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2f80b4d20 0x7fe2f80b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 -- 172.21.15.82:0/2807284440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2f81587f0 msgr2=0x7fe2f8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2f81587f0 0x7fe2f8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 -- 172.21.15.82:0/2807284440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f80b1f90 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f7fff640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f8071da0 0x7fe2f8156f50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00d830 tx=0x7fe2ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f4ff9640 1 -- 172.21.15.82:0/2807284440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00b6c0 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f4ff9640 1 -- 172.21.15.82:0/2807284440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec019070 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f8156190 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2f4ff9640 1 -- 172.21.15.82:0/2807284440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00ba80 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.615+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f8156750 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.616+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f804f050 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.617+0000 7fe2f4ff9640 1 -- 172.21.15.82:0/2807284440 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe2ec01e460 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.617+0000 7fe2f4ff9640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2e406bbd0 0x7fe2e406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.617+0000 7fe2f77fe640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2e406bbd0 0x7fe2e406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.617+0000 7fe2f4ff9640 1 -- 172.21.15.82:0/2807284440 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2ec08b150 con 0x7fe2f8071da0 2024-03-14T12:16:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.618+0000 7fe2f77fe640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2e406bbd0 0x7fe2e406e090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8038000 tx=0x7fe2e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:33.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.621+0000 7fe2f4ff9640 1 -- 172.21.15.82:0/2807284440 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe2ec059c40 con 0x7fe2f8071da0 2024-03-14T12:16:33.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.828+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe2f80725d0 con 0x7fe2f8071da0 2024-03-14T12:16:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.879+0000 7fe2f4ff9640 1 -- 172.21.15.82:0/2807284440 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe2ec059540 con 0x7fe2f8071da0 2024-03-14T12:16:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2e406bbd0 msgr2=0x7fe2e406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2e406bbd0 0x7fe2e406e090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8038000 tx=0x7fe2e8008040 comp rx=0 tx=0).stop 2024-03-14T12:16:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f8071da0 msgr2=0x7fe2f8156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f8071da0 0x7fe2f8156f50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00d830 tx=0x7fe2ec00dcf0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 shutdown_connections 2024-03-14T12:16:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2f81587f0 0x7fe2f8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2e406bbd0 0x7fe2e406e090 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2f80b4d20 0x7fe2f80b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 --2- 172.21.15.82:0/2807284440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2f8071da0 0x7fe2f8156f50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:33.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 >> 172.21.15.82:0/2807284440 conn(0x7fe2f80b2830 msgr2=0x7fe2f8159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:33.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 shutdown_connections 2024-03-14T12:16:33.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.881+0000 7fe2fe6ab640 1 -- 172.21.15.82:0/2807284440 wait complete. 2024-03-14T12:16:33.892 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T12:16:33.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T12:16:33.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.997+0000 7fec21c6c640 1 Processor -- start 2024-03-14T12:16:33.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.997+0000 7fec21c6c640 1 -- start start 2024-03-14T12:16:34.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec21c6c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec1c15bf10 0x7fec1c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:34.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec21c6c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec1c15c2f0 0x7fec1c0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:34.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec21c6c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c0b1e00 0x7fec1c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:34.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec21c6c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c0b62e0 con 0x7fec1c15bf10 2024-03-14T12:16:34.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec21c6c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c06c740 con 0x7fec1c15c2f0 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec21c6c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0670c0 con 0x7fec1c0b1e00 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec1c15bf10 0x7fec1c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec1c15bf10 0x7fec1c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50476/0 (socket says 172.21.15.82:50476) 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 -- 172.21.15.82:0/1834081015 learned_addr learned my addr 172.21.15.82:0/1834081015 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1bfff640 1 --2- 172.21.15.82:0/1834081015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c0b1e00 0x7fec1c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1affd640 1 --2- 172.21.15.82:0/1834081015 >> v2:172.21.15.164:3300/0 conn(0x7fec1c15c2f0 0x7fec1c0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 -- 172.21.15.82:0/1834081015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c0b1e00 msgr2=0x7fec1c0b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 --2- 172.21.15.82:0/1834081015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c0b1e00 0x7fec1c0b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 -- 172.21.15.82:0/1834081015 >> v2:172.21.15.164:3300/0 conn(0x7fec1c15c2f0 msgr2=0x7fec1c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 --2- 172.21.15.82:0/1834081015 >> v2:172.21.15.164:3300/0 conn(0x7fec1c15c2f0 0x7fec1c0af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.998+0000 7fec1b7fe640 1 -- 172.21.15.82:0/1834081015 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c06c130 con 0x7fec1c15bf10 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.999+0000 7fec1b7fe640 1 --2- 172.21.15.82:0/1834081015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c0b5da0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fec1c069e40 tx=0x7fec0c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb349bae88f69d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.999+0000 7fec1a7fc640 1 -- 172.21.15.82:0/1834081015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c02e070 con 0x7fec1c15bf10 2024-03-14T12:16:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.999+0000 7fec1a7fc640 1 -- 172.21.15.82:0/1834081015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c037070 con 0x7fec1c15bf10 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:33.999+0000 7fec1a7fc640 1 -- 172.21.15.82:0/1834081015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c031a00 con 0x7fec1c15bf10 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 -- 172.21.15.82:0/1834081015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 msgr2=0x7fec1c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 --2- 172.21.15.82:0/1834081015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c0b5da0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fec1c069e40 tx=0x7fec0c02b880 comp rx=0 tx=0).stop 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 -- 172.21.15.82:0/1834081015 shutdown_connections 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 --2- 172.21.15.82:0/1834081015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c0b1e00 0x7fec1c0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 --2- 172.21.15.82:0/1834081015 >> v2:172.21.15.164:3300/0 conn(0x7fec1c15c2f0 0x7fec1c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 --2- 172.21.15.82:0/1834081015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c0b5da0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 -- 172.21.15.82:0/1834081015 >> 172.21.15.82:0/1834081015 conn(0x7fec1c0b4fd0 msgr2=0x7fec1c0b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 -- 172.21.15.82:0/1834081015 shutdown_connections 2024-03-14T12:16:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.000+0000 7fec21c6c640 1 -- 172.21.15.82:0/1834081015 wait complete. 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.001+0000 7fec21c6c640 1 Processor -- start 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.001+0000 7fec21c6c640 1 -- start start 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.002+0000 7fec21c6c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c0b1e00 0x7fec1c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.002+0000 7fec21c6c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c09d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.002+0000 7fec21c6c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c15c2f0 0x7fec1c15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.002+0000 7fec21c6c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0afb10 con 0x7fec1c15bf10 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.002+0000 7fec21c6c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c06c3d0 con 0x7fec1c0b1e00 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.002+0000 7fec21c6c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c145b60 con 0x7fec1c15c2f0 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.002+0000 7fec1affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c09d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c09d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50492/0 (socket says 172.21.15.82:50492) 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 -- 172.21.15.82:0/3962902258 learned_addr learned my addr 172.21.15.82:0/3962902258 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1bfff640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c15c2f0 0x7fec1c15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 -- 172.21.15.82:0/3962902258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c15c2f0 msgr2=0x7fec1c15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:34.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c15c2f0 0x7fec1c15ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 -- 172.21.15.82:0/3962902258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c0b1e00 msgr2=0x7fec1c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c0b1e00 0x7fec1c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 -- 172.21.15.82:0/3962902258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c09e280 con 0x7fec1c15bf10 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec1affd640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c09d250 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c7c0 tx=0x7fec1000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec03fff640 1 -- 172.21.15.82:0/3962902258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10015070 con 0x7fec1c15bf10 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec03fff640 1 -- 172.21.15.82:0/3962902258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10007820 con 0x7fec1c15bf10 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec03fff640 1 -- 172.21.15.82:0/3962902258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10007bd0 con 0x7fec1c15bf10 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c06f0a0 con 0x7fec1c15bf10 2024-03-14T12:16:34.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.003+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c09dfa0 con 0x7fec1c15bf10 2024-03-14T12:16:34.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.004+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c04f050 con 0x7fec1c15bf10 2024-03-14T12:16:34.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.005+0000 7fec03fff640 1 -- 172.21.15.82:0/3962902258 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fec10018070 con 0x7fec1c15bf10 2024-03-14T12:16:34.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.006+0000 7fec03fff640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec0806bbd0 0x7fec0806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:34.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.006+0000 7fec03fff640 1 -- 172.21.15.82:0/3962902258 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec1008a7d0 con 0x7fec1c15bf10 2024-03-14T12:16:34.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.006+0000 7fec1b7fe640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec0806bbd0 0x7fec0806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:34.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.006+0000 7fec1b7fe640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec0806bbd0 0x7fec0806e090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec0c036000 tx=0x7fec0c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:34.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.010+0000 7fec03fff640 1 -- 172.21.15.82:0/3962902258 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fec1c09e280 con 0x7fec1c15bf10 2024-03-14T12:16:34.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.218+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fec1c15b730 con 0x7fec1c15bf10 2024-03-14T12:16:34.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.884+0000 7fec03fff640 1 -- 172.21.15.82:0/3962902258 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fec10059350 con 0x7fec1c15bf10 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec0806bbd0 msgr2=0x7fec0806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec0806bbd0 0x7fec0806e090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec0c036000 tx=0x7fec0c030000 comp rx=0 tx=0).stop 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 msgr2=0x7fec1c09d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c09d250 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c7c0 tx=0x7fec1000cc80 comp rx=0 tx=0).stop 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 shutdown_connections 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fec0806bbd0 0x7fec0806e090 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec1c15c2f0 0x7fec1c15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1c15bf10 0x7fec1c09d250 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 --2- 172.21.15.82:0/3962902258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec1c0b1e00 0x7fec1c0ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 >> 172.21.15.82:0/3962902258 conn(0x7fec1c0b4fd0 msgr2=0x7fec1c159150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:34.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 shutdown_connections 2024-03-14T12:16:34.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:34.886+0000 7fec21c6c640 1 -- 172.21.15.82:0/3962902258 wait complete. 2024-03-14T12:16:34.900 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T12:16:34.900 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:35.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.004+0000 7ff688b7e640 1 Processor -- start 2024-03-14T12:16:35.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.004+0000 7ff688b7e640 1 -- start start 2024-03-14T12:16:35.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff688b7e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff684151230 0x7ff6840cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff688b7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840b4b70 0x7ff684070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff688b7e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff6840d0040 0x7ff68415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff688b7e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68406c3b0 con 0x7ff684151230 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff688b7e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68406c690 con 0x7ff6840d0040 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff688b7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6840b3da0 con 0x7ff6840b4b70 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff684151230 0x7ff6840cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff684151230 0x7ff6840cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50496/0 (socket says 172.21.15.82:50496) 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 -- 172.21.15.82:0/2527866435 learned_addr learned my addr 172.21.15.82:0/2527866435 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 -- 172.21.15.82:0/2527866435 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840b4b70 msgr2=0x7ff684070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 --2- 172.21.15.82:0/2527866435 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840b4b70 0x7ff684070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 -- 172.21.15.82:0/2527866435 >> v2:172.21.15.164:3300/0 conn(0x7ff6840d0040 msgr2=0x7ff68415dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 --2- 172.21.15.82:0/2527866435 >> v2:172.21.15.164:3300/0 conn(0x7ff6840d0040 0x7ff68415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.005+0000 7ff68259c640 1 -- 172.21.15.82:0/2527866435 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840b4080 con 0x7ff684151230 2024-03-14T12:16:35.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff68259c640 1 --2- 172.21.15.82:0/2527866435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff684151230 0x7ff6840cf0a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff684072410 tx=0x7ff67402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98371fd2353a5207 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff681d9b640 1 -- 172.21.15.82:0/2527866435 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67402e070 con 0x7ff684151230 2024-03-14T12:16:35.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff681d9b640 1 -- 172.21.15.82:0/2527866435 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674037070 con 0x7ff684151230 2024-03-14T12:16:35.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff681d9b640 1 -- 172.21.15.82:0/2527866435 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674031a00 con 0x7ff684151230 2024-03-14T12:16:35.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff688b7e640 1 -- 172.21.15.82:0/2527866435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff684151230 msgr2=0x7ff6840cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff688b7e640 1 --2- 172.21.15.82:0/2527866435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff684151230 0x7ff6840cf0a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff684072410 tx=0x7ff67402b880 comp rx=0 tx=0).stop 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff688b7e640 1 -- 172.21.15.82:0/2527866435 shutdown_connections 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff688b7e640 1 --2- 172.21.15.82:0/2527866435 >> v2:172.21.15.164:3300/0 conn(0x7ff6840d0040 0x7ff68415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff688b7e640 1 --2- 172.21.15.82:0/2527866435 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840b4b70 0x7ff684070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff688b7e640 1 --2- 172.21.15.82:0/2527866435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff684151230 0x7ff6840cf0a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.006+0000 7ff688b7e640 1 -- 172.21.15.82:0/2527866435 >> 172.21.15.82:0/2527866435 conn(0x7ff6840b1c70 msgr2=0x7ff6840cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.007+0000 7ff688b7e640 1 -- 172.21.15.82:0/2527866435 shutdown_connections 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.007+0000 7ff688b7e640 1 -- 172.21.15.82:0/2527866435 wait complete. 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.008+0000 7ff688b7e640 1 Processor -- start 2024-03-14T12:16:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.008+0000 7ff688b7e640 1 -- start start 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840b4b70 0x7ff6841544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840d0040 0x7ff68414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff684151230 0x7ff6841503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6840cf5e0 con 0x7ff6840b4b70 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684151610 con 0x7ff684151230 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff684070b70 con 0x7ff6840d0040 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840b4b70 0x7ff6841544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840b4b70 0x7ff6841544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50502/0 (socket says 172.21.15.82:50502) 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 -- 172.21.15.82:0/1610838623 learned_addr learned my addr 172.21.15.82:0/1610838623 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 -- 172.21.15.82:0/1610838623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840d0040 msgr2=0x7ff68414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840d0040 0x7ff68414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 -- 172.21.15.82:0/1610838623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff684151230 msgr2=0x7ff6841503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff684151230 0x7ff6841503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 -- 172.21.15.82:0/1610838623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68413ffc0 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff68259c640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840b4b70 0x7ff6841544e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff6740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff67affd640 1 -- 172.21.15.82:0/1610838623 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674032880 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff67affd640 1 -- 172.21.15.82:0/1610838623 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674032a40 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6841402a0 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff67affd640 1 -- 172.21.15.82:0/1610838623 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674032e30 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.009+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff684140860 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.010+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68404f050 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.011+0000 7ff67affd640 1 -- 172.21.15.82:0/1610838623 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff67404c060 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.011+0000 7ff67affd640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff66406bc00 0x7ff66406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.011+0000 7ff67affd640 1 -- 172.21.15.82:0/1610838623 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6740ac6c0 con 0x7ff6840b4b70 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.011+0000 7ff67bfff640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff66406bc00 0x7ff66406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.012+0000 7ff67bfff640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff66406bc00 0x7ff66406e0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff684153cf0 tx=0x7ff66c006c90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.016+0000 7ff67affd640 1 -- 172.21.15.82:0/1610838623 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff674040070 con 0x7ff6840b4b70 2024-03-14T12:16:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.220+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6840b2f40 con 0x7ff6840b4b70 2024-03-14T12:16:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.221+0000 7ff67affd640 1 -- 172.21.15.82:0/1610838623 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4941 (secure 0 0 0) 0x7ff67407b1b0 con 0x7ff6840b4b70 2024-03-14T12:16:35.222 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:35.222 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5111,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/4078909817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4078909817},{"type":"v1","addr":"172.21.15.164:6834","nonce":4078909817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2349002857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2349002857},{"type":"v1","addr":"172.21.15.164:6835","nonce":2349002857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/285826310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":285826310},{"type":"v1","addr":"172.21.15.82:6835","nonce":285826310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:32.222787+0000","modified":"2024-03-14T12:16:34.220226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1422543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1422543802},{"type":"v1","addr":"172.21.15.82:6837","nonce":1422543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T12:16:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-03-14T12:16:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff66406bc00 msgr2=0x7ff66406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff66406bc00 0x7ff66406e0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff684153cf0 tx=0x7ff66c006c90 comp rx=0 tx=0).stop 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840b4b70 msgr2=0x7ff6841544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840b4b70 0x7ff6841544e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff6740046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 shutdown_connections 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff684151230 0x7ff6841503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff66406bc00 0x7ff66406e0c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6840d0040 0x7ff68414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 --2- 172.21.15.82:0/1610838623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840b4b70 0x7ff6841544e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 >> 172.21.15.82:0/1610838623 conn(0x7ff6840b1c70 msgr2=0x7ff68414f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 shutdown_connections 2024-03-14T12:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.223+0000 7ff688b7e640 1 -- 172.21.15.82:0/1610838623 wait complete. 2024-03-14T12:16:35.234 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.334+0000 7f363c7c4640 1 Processor -- start 2024-03-14T12:16:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.334+0000 7f363c7c4640 1 -- start start 2024-03-14T12:16:35.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363c7c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f363415e670 0x7f36340b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363c7c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363415ebe0 con 0x7f3634071740 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363c7c4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363415eec0 con 0x7f3634143f60 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363c7c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363415f1a0 con 0x7f363415e670 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363a560640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3634071740 0x7f3634071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41088/0 (socket says 172.21.15.82:41088) 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363a560640 1 -- 172.21.15.82:0/2258566936 learned_addr learned my addr 172.21.15.82:0/2258566936 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f3639d5f640 1 --2- 172.21.15.82:0/2258566936 >> v2:172.21.15.164:3300/0 conn(0x7f363415e670 0x7f36340b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260836006 0 0) 0x7f363415ebe0 con 0x7f3634071740 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3620003520 con 0x7f3634071740 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1841758464 0 0) 0x7f3620003520 con 0x7f3634071740 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 >> v2:172.21.15.164:3300/0 conn(0x7f363415e670 msgr2=0x7f36340b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:35.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 --2- 172.21.15.82:0/2258566936 >> v2:172.21.15.164:3300/0 conn(0x7f363415e670 0x7f36340b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 >> v1:172.21.15.164:6789/0 conn(0x7f3634143f60 legacy=0x7f363415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363415f480 con 0x7f3634071740 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f3630001670 con 0x7f3634071740 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 >> v1:172.21.15.82:6789/0 conn(0x7f3634071740 legacy=0x7f3634071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36200039e0 0x7f3620003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3620004800 0x7f3620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36200051c0 0x7f362000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363415f1a0 con 0x7f36200051c0 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362000c090 con 0x7f3620004800 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362000c290 con 0x7f36200039e0 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.335+0000 7f363ad61640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36200051c0 0x7f362000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363a560640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36200039e0 0x7f3620003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f3639d5f640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3620004800 0x7f3620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363ad61640 1 -- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36200039e0 msgr2=0x7f3620003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363ad61640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36200039e0 0x7f3620003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363ad61640 1 -- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3620004800 msgr2=0x7f3620004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363ad61640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3620004800 0x7f3620004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363ad61640 1 -- 172.21.15.82:0/2258566936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620003520 con 0x7f36200051c0 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363ad61640 1 -- 172.21.15.82:0/2258566936 reap_dead start 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.336+0000 7f363ad61640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36200051c0 0x7f362000b9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f362c001e50 tx=0x7f3620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde4906cd9fc567b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c001a70 con 0x7f36200051c0 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c003bb0 con 0x7f36200051c0 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363955e640 1 -- 172.21.15.82:0/2258566936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0050d0 con 0x7f36200051c0 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363c7c4640 1 -- 172.21.15.82:0/2258566936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36200051c0 msgr2=0x7f362000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36200051c0 0x7f362000b9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f362c001e50 tx=0x7f3620005830 comp rx=0 tx=0).stop 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363c7c4640 1 -- 172.21.15.82:0/2258566936 shutdown_connections 2024-03-14T12:16:35.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2258566936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36200051c0 0x7f362000b9c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363c7c4640 1 -- 172.21.15.82:0/2258566936 >> 172.21.15.82:0/2258566936 conn(0x7f36340b1c70 msgr2=0x7f36340cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:35.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363c7c4640 1 -- 172.21.15.82:0/2258566936 shutdown_connections 2024-03-14T12:16:35.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.337+0000 7f363c7c4640 1 -- 172.21.15.82:0/2258566936 wait complete. 2024-03-14T12:16:35.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.338+0000 7f363c7c4640 1 Processor -- start 2024-03-14T12:16:35.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.338+0000 7f363c7c4640 1 -- start start 2024-03-14T12:16:35.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363c7c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36200051c0 0x7f36341f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363c7c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3634158210 0x7f363415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363c7c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f363415bf70 0x7f363415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363c7c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634144670 con 0x7f3634158210 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363c7c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363415b080 con 0x7f36200051c0 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363c7c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36341447f0 con 0x7f363415bf70 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3634158210 0x7f363415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3634158210 0x7f363415ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50510/0 (socket says 172.21.15.82:50510) 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 -- 172.21.15.82:0/2284214903 learned_addr learned my addr 172.21.15.82:0/2284214903 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363ad61640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f363415bf70 0x7f363415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363a560640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36200051c0 0x7f36341f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 -- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f363415bf70 msgr2=0x7f363415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f363415bf70 0x7f363415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 -- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36200051c0 msgr2=0x7f36341f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36200051c0 0x7f36341f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f3639d5f640 1 -- 172.21.15.82:0/2284214903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340ce1f0 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.339+0000 7f363a560640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36200051c0 0x7f36341f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.340+0000 7f3639d5f640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3634158210 0x7f363415ba30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f36240118e0 tx=0x7f3624011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.340+0000 7f362affd640 1 -- 172.21.15.82:0/2284214903 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400fa90 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.340+0000 7f362affd640 1 -- 172.21.15.82:0/2284214903 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362400fc30 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.340+0000 7f362affd640 1 -- 172.21.15.82:0/2284214903 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624014440 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.340+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36340ce4d0 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.340+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36340cea90 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.341+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363415e340 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.341+0000 7f362affd640 1 -- 172.21.15.82:0/2284214903 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3624012040 con 0x7f3634158210 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.341+0000 7f362affd640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361806bbd0 0x7f361806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.342+0000 7f363a560640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361806bbd0 0x7f361806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.342+0000 7f362affd640 1 -- 172.21.15.82:0/2284214903 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f362408b0f0 con 0x7f3634158210 2024-03-14T12:16:35.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.342+0000 7f363a560640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361806bbd0 0x7f361806e090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3630007c70 tx=0x7f3630009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.346+0000 7f362affd640 1 -- 172.21.15.82:0/2284214903 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3624019030 con 0x7f3634158210 2024-03-14T12:16:35.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.517+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f363404f050 con 0x7f3634158210 2024-03-14T12:16:35.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.518+0000 7f362affd640 1 -- 172.21.15.82:0/2284214903 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f3624059be0 con 0x7f3634158210 2024-03-14T12:16:35.518 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:35.518 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":58,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:33.876540+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:35.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361806bbd0 msgr2=0x7f361806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361806bbd0 0x7f361806e090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3630007c70 tx=0x7f3630009040 comp rx=0 tx=0).stop 2024-03-14T12:16:35.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3634158210 msgr2=0x7f363415ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3634158210 0x7f363415ba30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f36240118e0 tx=0x7f3624011da0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 shutdown_connections 2024-03-14T12:16:35.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f361806bbd0 0x7f361806e090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f363415bf70 0x7f363415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3634158210 0x7f363415ba30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 --2- 172.21.15.82:0/2284214903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36200051c0 0x7f36341f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 >> 172.21.15.82:0/2284214903 conn(0x7f36340b1c70 msgr2=0x7f3634071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:35.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 shutdown_connections 2024-03-14T12:16:35.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.520+0000 7f363c7c4640 1 -- 172.21.15.82:0/2284214903 wait complete. 2024-03-14T12:16:35.531 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T12:16:35.531 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:16:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.631+0000 7f6490297640 1 Processor -- start 2024-03-14T12:16:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.631+0000 7f6490297640 1 -- start start 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f6490297640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648815fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f6490297640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f64880cdd60 0x7f64880ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f6490297640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64881600c0 con 0x7f6488144ac0 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f6490297640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64880cf150 con 0x7f64880cdd60 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f6490297640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64880ce8b0 con 0x7f648815f7a0 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648d832640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f64880cdd60 0x7f64880ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648e033640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648815fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648e834640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6488144ac0 0x7f6488144f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:49808/0 (socket says 172.21.15.82:49808) 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648e834640 1 -- 172.21.15.82:0/1780093909 learned_addr learned my addr 172.21.15.82:0/1780093909 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648d031640 1 -- 172.21.15.82:0/1780093909 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903139554 0 0) 0x7f64881600c0 con 0x7f6488144ac0 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648d031640 1 -- 172.21.15.82:0/1780093909 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6474003520 con 0x7f6488144ac0 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648e033640 1 -- 172.21.15.82:0/1780093909 >> v2:172.21.15.164:3300/0 conn(0x7f64880cdd60 msgr2=0x7f64880ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648e033640 1 --2- 172.21.15.82:0/1780093909 >> v2:172.21.15.164:3300/0 conn(0x7f64880cdd60 0x7f64880ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648e033640 1 -- 172.21.15.82:0/1780093909 >> v1:172.21.15.164:6789/0 conn(0x7f6488144ac0 legacy=0x7f6488144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648e033640 1 -- 172.21.15.82:0/1780093909 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880ceb90 con 0x7f648815f7a0 2024-03-14T12:16:35.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.632+0000 7f648d832640 1 --2- 172.21.15.82:0/1780093909 >> v2:172.21.15.164:3300/0 conn(0x7f64880cdd60 0x7f64880ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.633+0000 7f648e033640 1 --2- 172.21.15.82:0/1780093909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648815fb80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64880cf2d0 tx=0x7f648402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625350510e305784 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.633+0000 7f648d031640 1 -- 172.21.15.82:0/1780093909 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484031770 con 0x7f648815f7a0 2024-03-14T12:16:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.633+0000 7f648d031640 1 -- 172.21.15.82:0/1780093909 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6484037070 con 0x7f648815f7a0 2024-03-14T12:16:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.633+0000 7f648d031640 1 -- 172.21.15.82:0/1780093909 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484031b30 con 0x7f648815f7a0 2024-03-14T12:16:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 -- 172.21.15.82:0/1780093909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 msgr2=0x7f648815fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 --2- 172.21.15.82:0/1780093909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648815fb80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64880cf2d0 tx=0x7f648402b810 comp rx=0 tx=0).stop 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 -- 172.21.15.82:0/1780093909 shutdown_connections 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 --2- 172.21.15.82:0/1780093909 >> v2:172.21.15.164:3300/0 conn(0x7f64880cdd60 0x7f64880ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 --2- 172.21.15.82:0/1780093909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648815fb80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 -- 172.21.15.82:0/1780093909 >> 172.21.15.82:0/1780093909 conn(0x7f64880b1c70 msgr2=0x7f64880cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 -- 172.21.15.82:0/1780093909 shutdown_connections 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.634+0000 7f6490297640 1 -- 172.21.15.82:0/1780093909 wait complete. 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.635+0000 7f6490297640 1 Processor -- start 2024-03-14T12:16:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.635+0000 7f6490297640 1 -- start start 2024-03-14T12:16:35.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f6490297640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdd60 0x7f6488141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f6490297640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6488144ac0 0x7f648813b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f6490297640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648813b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f6490297640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880cc4f0 con 0x7f64880cdd60 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f6490297640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648806c4e0 con 0x7f6488144ac0 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f6490297640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64880cc670 con 0x7f648815f7a0 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdd60 0x7f6488141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdd60 0x7f6488141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50518/0 (socket says 172.21.15.82:50518) 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 -- 172.21.15.82:0/817214582 learned_addr learned my addr 172.21.15.82:0/817214582 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e834640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648813b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648d832640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6488144ac0 0x7f648813b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 -- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 msgr2=0x7f648813b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648813b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 -- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6488144ac0 msgr2=0x7f648813b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6488144ac0 0x7f648813b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 -- 172.21.15.82:0/817214582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648813a080 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648d832640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6488144ac0 0x7f648813b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.636+0000 7f648e033640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdd60 0x7f6488141e40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6484009890 tx=0x7f64840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.637+0000 7f647effd640 1 -- 172.21.15.82:0/817214582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484031ed0 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.637+0000 7f647effd640 1 -- 172.21.15.82:0/817214582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6484032600 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.637+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648813a360 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.637+0000 7f647effd640 1 -- 172.21.15.82:0/817214582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840329f0 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.637+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648813a920 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.638+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648813bd60 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.638+0000 7f647effd640 1 -- 172.21.15.82:0/817214582 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6484031070 con 0x7f64880cdd60 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.638+0000 7f647effd640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f646c06bbd0 0x7f646c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.639+0000 7f648d832640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f646c06bbd0 0x7f646c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.639+0000 7f647effd640 1 -- 172.21.15.82:0/817214582 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64840ac480 con 0x7f64880cdd60 2024-03-14T12:16:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.639+0000 7f648d832640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f646c06bbd0 0x7f646c06e090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6488139e80 tx=0x7f6478008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.643+0000 7f647effd640 1 -- 172.21.15.82:0/817214582 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6484040070 con 0x7f64880cdd60 2024-03-14T12:16:35.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.810+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f648804f050 con 0x7f646c06bbd0 2024-03-14T12:16:35.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.816+0000 7f647effd640 1 -- 172.21.15.82:0/817214582 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251883 (secure 0 0 0) 0x7f648804f050 con 0x7f646c06bbd0 2024-03-14T12:16:35.819 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:35.821 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:16:35.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f646c06bbd0 msgr2=0x7f646c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f646c06bbd0 0x7f646c06e090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6488139e80 tx=0x7f6478008040 comp rx=0 tx=0).stop 2024-03-14T12:16:35.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdd60 msgr2=0x7f6488141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdd60 0x7f6488141e40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6484009890 tx=0x7f64840046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 shutdown_connections 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f646c06bbd0 0x7f646c06e090 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f648815f7a0 0x7f648813b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6488144ac0 0x7f648813b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 --2- 172.21.15.82:0/817214582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdd60 0x7f6488141e40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 >> 172.21.15.82:0/817214582 conn(0x7f64880b1c70 msgr2=0x7f6488142fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 shutdown_connections 2024-03-14T12:16:35.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.819+0000 7f6490297640 1 -- 172.21.15.82:0/817214582 wait complete. 2024-03-14T12:16:35.833 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-03-14T12:16:35.531735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260184,"kb_used_data":6552,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714536,"statfs":{"total":773094113280,"available":772827684864,"internally_reserved":0,"allocated":6709248,"data_stored":3430083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":259575347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.669293"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900025+0000","last_change":"2024-03-14T12:16:32.897956+0000","last_active":"2024-03-14T12:16:32.900025+0000","last_peered":"2024-03-14T12:16:32.900025+0000","last_clean":"2024-03-14T12:16:32.900025+0000","last_became_active":"2024-03-14T12:16:32.897507+0000","last_became_peered":"2024-03-14T12:16:32.897507+0000","last_unstale":"2024-03-14T12:16:32.900025+0000","last_undegraded":"2024-03-14T12:16:32.900025+0000","last_fullsized":"2024-03-14T12:16:32.900025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:29:30.619269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896721+0000","last_change":"2024-03-14T12:16:32.885186+0000","last_active":"2024-03-14T12:16:32.896721+0000","last_peered":"2024-03-14T12:16:32.896721+0000","last_clean":"2024-03-14T12:16:32.896721+0000","last_became_active":"2024-03-14T12:16:32.884928+0000","last_became_peered":"2024-03-14T12:16:32.884928+0000","last_unstale":"2024-03-14T12:16:32.896721+0000","last_undegraded":"2024-03-14T12:16:32.896721+0000","last_fullsized":"2024-03-14T12:16:32.896721+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:35:57.163688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900800+0000","last_change":"2024-03-14T12:16:32.895727+0000","last_active":"2024-03-14T12:16:32.900800+0000","last_peered":"2024-03-14T12:16:32.900800+0000","last_clean":"2024-03-14T12:16:32.900800+0000","last_became_active":"2024-03-14T12:16:32.895399+0000","last_became_peered":"2024-03-14T12:16:32.895399+0000","last_unstale":"2024-03-14T12:16:32.900800+0000","last_undegraded":"2024-03-14T12:16:32.900800+0000","last_fullsized":"2024-03-14T12:16:32.900800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:19.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900005+0000","last_change":"2024-03-14T12:16:32.890627+0000","last_active":"2024-03-14T12:16:32.900005+0000","last_peered":"2024-03-14T12:16:32.900005+0000","last_clean":"2024-03-14T12:16:32.900005+0000","last_became_active":"2024-03-14T12:16:32.890365+0000","last_became_peered":"2024-03-14T12:16:32.890365+0000","last_unstale":"2024-03-14T12:16:32.900005+0000","last_undegraded":"2024-03-14T12:16:32.900005+0000","last_fullsized":"2024-03-14T12:16:32.900005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:22.674104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897241+0000","last_change":"2024-03-14T12:16:32.896084+0000","last_active":"2024-03-14T12:16:32.897241+0000","last_peered":"2024-03-14T12:16:32.897241+0000","last_clean":"2024-03-14T12:16:32.897241+0000","last_became_active":"2024-03-14T12:16:32.895788+0000","last_became_peered":"2024-03-14T12:16:32.895788+0000","last_unstale":"2024-03-14T12:16:32.897241+0000","last_undegraded":"2024-03-14T12:16:32.897241+0000","last_fullsized":"2024-03-14T12:16:32.897241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:20.830516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896907+0000","last_change":"2024-03-14T12:16:32.892737+0000","last_active":"2024-03-14T12:16:32.896907+0000","last_peered":"2024-03-14T12:16:32.896907+0000","last_clean":"2024-03-14T12:16:32.896907+0000","last_became_active":"2024-03-14T12:16:32.892466+0000","last_became_peered":"2024-03-14T12:16:32.892466+0000","last_unstale":"2024-03-14T12:16:32.896907+0000","last_undegraded":"2024-03-14T12:16:32.896907+0000","last_fullsized":"2024-03-14T12:16:32.896907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:24.983599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894538+0000","last_change":"2024-03-14T12:16:32.884016+0000","last_active":"2024-03-14T12:16:32.894538+0000","last_peered":"2024-03-14T12:16:32.894538+0000","last_clean":"2024-03-14T12:16:32.894538+0000","last_became_active":"2024-03-14T12:16:32.883695+0000","last_became_peered":"2024-03-14T12:16:32.883695+0000","last_unstale":"2024-03-14T12:16:32.894538+0000","last_undegraded":"2024-03-14T12:16:32.894538+0000","last_fullsized":"2024-03-14T12:16:32.894538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:03:25.351354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.891771+0000","last_change":"2024-03-14T12:16:32.883406+0000","last_active":"2024-03-14T12:16:32.891771+0000","last_peered":"2024-03-14T12:16:32.891771+0000","last_clean":"2024-03-14T12:16:32.891771+0000","last_became_active":"2024-03-14T12:16:32.882582+0000","last_became_peered":"2024-03-14T12:16:32.882582+0000","last_unstale":"2024-03-14T12:16:32.891771+0000","last_undegraded":"2024-03-14T12:16:32.891771+0000","last_fullsized":"2024-03-14T12:16:32.891771+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:07:10.557782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894716+0000","last_change":"2024-03-14T12:16:32.885107+0000","last_active":"2024-03-14T12:16:32.894716+0000","last_peered":"2024-03-14T12:16:32.894716+0000","last_clean":"2024-03-14T12:16:32.894716+0000","last_became_active":"2024-03-14T12:16:32.884742+0000","last_became_peered":"2024-03-14T12:16:32.884742+0000","last_unstale":"2024-03-14T12:16:32.894716+0000","last_undegraded":"2024-03-14T12:16:32.894716+0000","last_fullsized":"2024-03-14T12:16:32.894716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:10.878467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889105+0000","last_change":"2024-03-14T12:16:32.883744+0000","last_active":"2024-03-14T12:16:32.889105+0000","last_peered":"2024-03-14T12:16:32.889105+0000","last_clean":"2024-03-14T12:16:32.889105+0000","last_became_active":"2024-03-14T12:16:32.883483+0000","last_became_peered":"2024-03-14T12:16:32.883483+0000","last_unstale":"2024-03-14T12:16:32.889105+0000","last_undegraded":"2024-03-14T12:16:32.889105+0000","last_fullsized":"2024-03-14T12:16:32.889105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:27.750978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888405+0000","last_change":"2024-03-14T12:16:32.881473+0000","last_active":"2024-03-14T12:16:32.888405+0000","last_peered":"2024-03-14T12:16:32.888405+0000","last_clean":"2024-03-14T12:16:32.888405+0000","last_became_active":"2024-03-14T12:16:32.881233+0000","last_became_peered":"2024-03-14T12:16:32.881233+0000","last_unstale":"2024-03-14T12:16:32.888405+0000","last_undegraded":"2024-03-14T12:16:32.888405+0000","last_fullsized":"2024-03-14T12:16:32.888405+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:58:27.218417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.895041+0000","last_change":"2024-03-14T12:16:32.884493+0000","last_active":"2024-03-14T12:16:32.895041+0000","last_peered":"2024-03-14T12:16:32.895041+0000","last_clean":"2024-03-14T12:16:32.895041+0000","last_became_active":"2024-03-14T12:16:32.884220+0000","last_became_peered":"2024-03-14T12:16:32.884220+0000","last_unstale":"2024-03-14T12:16:32.895041+0000","last_undegraded":"2024-03-14T12:16:32.895041+0000","last_fullsized":"2024-03-14T12:16:32.895041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:19.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889250+0000","last_change":"2024-03-14T12:16:32.887567+0000","last_active":"2024-03-14T12:16:32.889250+0000","last_peered":"2024-03-14T12:16:32.889250+0000","last_clean":"2024-03-14T12:16:32.889250+0000","last_became_active":"2024-03-14T12:16:32.887048+0000","last_became_peered":"2024-03-14T12:16:32.887048+0000","last_unstale":"2024-03-14T12:16:32.889250+0000","last_undegraded":"2024-03-14T12:16:32.889250+0000","last_fullsized":"2024-03-14T12:16:32.889250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:42:34.697540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897353+0000","last_change":"2024-03-14T12:16:32.890660+0000","last_active":"2024-03-14T12:16:32.897353+0000","last_peered":"2024-03-14T12:16:32.897353+0000","last_clean":"2024-03-14T12:16:32.897353+0000","last_became_active":"2024-03-14T12:16:32.890374+0000","last_became_peered":"2024-03-14T12:16:32.890374+0000","last_unstale":"2024-03-14T12:16:32.897353+0000","last_undegraded":"2024-03-14T12:16:32.897353+0000","last_fullsized":"2024-03-14T12:16:32.897353+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:10.857734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.887661+0000","last_change":"2024-03-14T12:16:32.879744+0000","last_active":"2024-03-14T12:16:32.887661+0000","last_peered":"2024-03-14T12:16:32.887661+0000","last_clean":"2024-03-14T12:16:32.887661+0000","last_became_active":"2024-03-14T12:16:32.879468+0000","last_became_peered":"2024-03-14T12:16:32.879468+0000","last_unstale":"2024-03-14T12:16:32.887661+0000","last_undegraded":"2024-03-14T12:16:32.887661+0000","last_fullsized":"2024-03-14T12:16:32.887661+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:17:16.261933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889964+0000","last_change":"2024-03-14T12:16:32.886008+0000","last_active":"2024-03-14T12:16:32.889964+0000","last_peered":"2024-03-14T12:16:32.889964+0000","last_clean":"2024-03-14T12:16:32.889964+0000","last_became_active":"2024-03-14T12:16:32.885737+0000","last_became_peered":"2024-03-14T12:16:32.885737+0000","last_unstale":"2024-03-14T12:16:32.889964+0000","last_undegraded":"2024-03-14T12:16:32.889964+0000","last_fullsized":"2024-03-14T12:16:32.889964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:24.479669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898365+0000","last_change":"2024-03-14T12:16:32.892148+0000","last_active":"2024-03-14T12:16:32.898365+0000","last_peered":"2024-03-14T12:16:32.898365+0000","last_clean":"2024-03-14T12:16:32.898365+0000","last_became_active":"2024-03-14T12:16:32.891876+0000","last_became_peered":"2024-03-14T12:16:32.891876+0000","last_unstale":"2024-03-14T12:16:32.898365+0000","last_undegraded":"2024-03-14T12:16:32.898365+0000","last_fullsized":"2024-03-14T12:16:32.898365+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:34:04.174384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900253+0000","last_change":"2024-03-14T12:16:32.889876+0000","last_active":"2024-03-14T12:16:32.900253+0000","last_peered":"2024-03-14T12:16:32.900253+0000","last_clean":"2024-03-14T12:16:32.900253+0000","last_became_active":"2024-03-14T12:16:32.889238+0000","last_became_peered":"2024-03-14T12:16:32.889238+0000","last_unstale":"2024-03-14T12:16:32.900253+0000","last_undegraded":"2024-03-14T12:16:32.900253+0000","last_fullsized":"2024-03-14T12:16:32.900253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:25:12.759059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896436+0000","last_change":"2024-03-14T12:16:32.893974+0000","last_active":"2024-03-14T12:16:32.896436+0000","last_peered":"2024-03-14T12:16:32.896436+0000","last_clean":"2024-03-14T12:16:32.896436+0000","last_became_active":"2024-03-14T12:16:32.893706+0000","last_became_peered":"2024-03-14T12:16:32.893706+0000","last_unstale":"2024-03-14T12:16:32.896436+0000","last_undegraded":"2024-03-14T12:16:32.896436+0000","last_fullsized":"2024-03-14T12:16:32.896436+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:41.261103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897944+0000","last_change":"2024-03-14T12:16:32.892682+0000","last_active":"2024-03-14T12:16:32.897944+0000","last_peered":"2024-03-14T12:16:32.897944+0000","last_clean":"2024-03-14T12:16:32.897944+0000","last_became_active":"2024-03-14T12:16:32.892416+0000","last_became_peered":"2024-03-14T12:16:32.892416+0000","last_unstale":"2024-03-14T12:16:32.897944+0000","last_undegraded":"2024-03-14T12:16:32.897944+0000","last_fullsized":"2024-03-14T12:16:32.897944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:36:13.000073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900763+0000","last_change":"2024-03-14T12:16:32.895919+0000","last_active":"2024-03-14T12:16:32.900763+0000","last_peered":"2024-03-14T12:16:32.900763+0000","last_clean":"2024-03-14T12:16:32.900763+0000","last_became_active":"2024-03-14T12:16:32.895571+0000","last_became_peered":"2024-03-14T12:16:32.895571+0000","last_unstale":"2024-03-14T12:16:32.900763+0000","last_undegraded":"2024-03-14T12:16:32.900763+0000","last_fullsized":"2024-03-14T12:16:32.900763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:08.569373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898252+0000","last_change":"2024-03-14T12:16:32.894563+0000","last_active":"2024-03-14T12:16:32.898252+0000","last_peered":"2024-03-14T12:16:32.898252+0000","last_clean":"2024-03-14T12:16:32.898252+0000","last_became_active":"2024-03-14T12:16:32.894102+0000","last_became_peered":"2024-03-14T12:16:32.894102+0000","last_unstale":"2024-03-14T12:16:32.898252+0000","last_undegraded":"2024-03-14T12:16:32.898252+0000","last_fullsized":"2024-03-14T12:16:32.898252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:38.560864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900864+0000","last_change":"2024-03-14T12:16:32.897695+0000","last_active":"2024-03-14T12:16:32.900864+0000","last_peered":"2024-03-14T12:16:32.900864+0000","last_clean":"2024-03-14T12:16:32.900864+0000","last_became_active":"2024-03-14T12:16:32.897274+0000","last_became_peered":"2024-03-14T12:16:32.897274+0000","last_unstale":"2024-03-14T12:16:32.900864+0000","last_undegraded":"2024-03-14T12:16:32.900864+0000","last_fullsized":"2024-03-14T12:16:32.900864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:22.674104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889282+0000","last_change":"2024-03-14T12:16:32.887118+0000","last_active":"2024-03-14T12:16:32.889282+0000","last_peered":"2024-03-14T12:16:32.889282+0000","last_clean":"2024-03-14T12:16:32.889282+0000","last_became_active":"2024-03-14T12:16:32.886822+0000","last_became_peered":"2024-03-14T12:16:32.886822+0000","last_unstale":"2024-03-14T12:16:32.889282+0000","last_undegraded":"2024-03-14T12:16:32.889282+0000","last_fullsized":"2024-03-14T12:16:32.889282+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:27.750978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894499+0000","last_change":"2024-03-14T12:16:32.890670+0000","last_active":"2024-03-14T12:16:32.894499+0000","last_peered":"2024-03-14T12:16:32.894499+0000","last_clean":"2024-03-14T12:16:32.894499+0000","last_became_active":"2024-03-14T12:16:32.890305+0000","last_became_peered":"2024-03-14T12:16:32.890305+0000","last_unstale":"2024-03-14T12:16:32.894499+0000","last_undegraded":"2024-03-14T12:16:32.894499+0000","last_fullsized":"2024-03-14T12:16:32.894499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:28.029527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.882245+0000","last_change":"2024-03-14T12:16:32.879318+0000","last_active":"2024-03-14T12:16:32.882245+0000","last_peered":"2024-03-14T12:16:32.882245+0000","last_clean":"2024-03-14T12:16:32.882245+0000","last_became_active":"2024-03-14T12:16:32.879058+0000","last_became_peered":"2024-03-14T12:16:32.879058+0000","last_unstale":"2024-03-14T12:16:32.882245+0000","last_undegraded":"2024-03-14T12:16:32.882245+0000","last_fullsized":"2024-03-14T12:16:32.882245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:48:07.046028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889979+0000","last_change":"2024-03-14T12:16:32.887107+0000","last_active":"2024-03-14T12:16:32.889979+0000","last_peered":"2024-03-14T12:16:32.889979+0000","last_clean":"2024-03-14T12:16:32.889979+0000","last_became_active":"2024-03-14T12:16:32.886631+0000","last_became_peered":"2024-03-14T12:16:32.886631+0000","last_unstale":"2024-03-14T12:16:32.889979+0000","last_undegraded":"2024-03-14T12:16:32.889979+0000","last_fullsized":"2024-03-14T12:16:32.889979+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:23.111789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897101+0000","last_change":"2024-03-14T12:16:32.896082+0000","last_active":"2024-03-14T12:16:32.897101+0000","last_peered":"2024-03-14T12:16:32.897101+0000","last_clean":"2024-03-14T12:16:32.897101+0000","last_became_active":"2024-03-14T12:16:32.895805+0000","last_became_peered":"2024-03-14T12:16:32.895805+0000","last_unstale":"2024-03-14T12:16:32.897101+0000","last_undegraded":"2024-03-14T12:16:32.897101+0000","last_fullsized":"2024-03-14T12:16:32.897101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:33.586925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898244+0000","last_change":"2024-03-14T12:16:32.895105+0000","last_active":"2024-03-14T12:16:32.898244+0000","last_peered":"2024-03-14T12:16:32.898244+0000","last_clean":"2024-03-14T12:16:32.898244+0000","last_became_active":"2024-03-14T12:16:32.894808+0000","last_became_peered":"2024-03-14T12:16:32.894808+0000","last_unstale":"2024-03-14T12:16:32.898244+0000","last_undegraded":"2024-03-14T12:16:32.898244+0000","last_fullsized":"2024-03-14T12:16:32.898244+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:11.934373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894562+0000","last_change":"2024-03-14T12:16:32.888432+0000","last_active":"2024-03-14T12:16:32.894562+0000","last_peered":"2024-03-14T12:16:32.894562+0000","last_clean":"2024-03-14T12:16:32.894562+0000","last_became_active":"2024-03-14T12:16:32.888159+0000","last_became_peered":"2024-03-14T12:16:32.888159+0000","last_unstale":"2024-03-14T12:16:32.894562+0000","last_undegraded":"2024-03-14T12:16:32.894562+0000","last_fullsized":"2024-03-14T12:16:32.894562+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:10.857734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.891088+0000","last_change":"2024-03-14T12:16:32.890285+0000","last_active":"2024-03-14T12:16:32.891088+0000","last_peered":"2024-03-14T12:16:32.891088+0000","last_clean":"2024-03-14T12:16:32.891088+0000","last_became_active":"2024-03-14T12:16:32.889904+0000","last_became_peered":"2024-03-14T12:16:32.889904+0000","last_unstale":"2024-03-14T12:16:32.891088+0000","last_undegraded":"2024-03-14T12:16:32.891088+0000","last_fullsized":"2024-03-14T12:16:32.891088+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:38:19.269651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888632+0000","last_change":"2024-03-14T12:16:32.885130+0000","last_active":"2024-03-14T12:16:32.888632+0000","last_peered":"2024-03-14T12:16:32.888632+0000","last_clean":"2024-03-14T12:16:32.888632+0000","last_became_active":"2024-03-14T12:16:32.884845+0000","last_became_peered":"2024-03-14T12:16:32.884845+0000","last_unstale":"2024-03-14T12:16:32.888632+0000","last_undegraded":"2024-03-14T12:16:32.888632+0000","last_fullsized":"2024-03-14T12:16:32.888632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:22.674104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889929+0000","last_change":"2024-03-14T12:16:32.886892+0000","last_active":"2024-03-14T12:16:32.889929+0000","last_peered":"2024-03-14T12:16:32.889929+0000","last_clean":"2024-03-14T12:16:32.889929+0000","last_became_active":"2024-03-14T12:16:32.886637+0000","last_became_peered":"2024-03-14T12:16:32.886637+0000","last_unstale":"2024-03-14T12:16:32.889929+0000","last_undegraded":"2024-03-14T12:16:32.889929+0000","last_fullsized":"2024-03-14T12:16:32.889929+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:18:55.168791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900968+0000","last_change":"2024-03-14T12:16:31.883324+0000","last_active":"2024-03-14T12:16:32.900968+0000","last_peered":"2024-03-14T12:16:32.900968+0000","last_clean":"2024-03-14T12:16:32.900968+0000","last_became_active":"2024-03-14T12:16:31.883049+0000","last_became_peered":"2024-03-14T12:16:31.883049+0000","last_unstale":"2024-03-14T12:16:32.900968+0000","last_undegraded":"2024-03-14T12:16:32.900968+0000","last_fullsized":"2024-03-14T12:16:32.900968+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:09.852408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889251+0000","last_change":"2024-03-14T12:16:32.886808+0000","last_active":"2024-03-14T12:16:32.889251+0000","last_peered":"2024-03-14T12:16:32.889251+0000","last_clean":"2024-03-14T12:16:32.889251+0000","last_became_active":"2024-03-14T12:16:32.886530+0000","last_became_peered":"2024-03-14T12:16:32.886530+0000","last_unstale":"2024-03-14T12:16:32.889251+0000","last_undegraded":"2024-03-14T12:16:32.889251+0000","last_fullsized":"2024-03-14T12:16:32.889251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:46:34.286986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888282+0000","last_change":"2024-03-14T12:16:31.887910+0000","last_active":"2024-03-14T12:16:32.888282+0000","last_peered":"2024-03-14T12:16:32.888282+0000","last_clean":"2024-03-14T12:16:32.888282+0000","last_became_active":"2024-03-14T12:16:31.887627+0000","last_became_peered":"2024-03-14T12:16:31.887627+0000","last_unstale":"2024-03-14T12:16:32.888282+0000","last_undegraded":"2024-03-14T12:16:32.888282+0000","last_fullsized":"2024-03-14T12:16:32.888282+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:17.462895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.887710+0000","last_change":"2024-03-14T12:16:32.883823+0000","last_active":"2024-03-14T12:16:32.887710+0000","last_peered":"2024-03-14T12:16:32.887710+0000","last_clean":"2024-03-14T12:16:32.887710+0000","last_became_active":"2024-03-14T12:16:32.883538+0000","last_became_peered":"2024-03-14T12:16:32.883538+0000","last_unstale":"2024-03-14T12:16:32.887710+0000","last_undegraded":"2024-03-14T12:16:32.887710+0000","last_fullsized":"2024-03-14T12:16:32.887710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:19.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.891962+0000","last_change":"2024-03-14T12:16:31.889056+0000","last_active":"2024-03-14T12:16:32.891962+0000","last_peered":"2024-03-14T12:16:32.891962+0000","last_clean":"2024-03-14T12:16:32.891962+0000","last_became_active":"2024-03-14T12:16:31.888770+0000","last_became_peered":"2024-03-14T12:16:31.888770+0000","last_unstale":"2024-03-14T12:16:32.891962+0000","last_undegraded":"2024-03-14T12:16:32.891962+0000","last_fullsized":"2024-03-14T12:16:32.891962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:25:11.753733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889957+0000","last_change":"2024-03-14T12:16:32.886763+0000","last_active":"2024-03-14T12:16:32.889957+0000","last_peered":"2024-03-14T12:16:32.889957+0000","last_clean":"2024-03-14T12:16:32.889957+0000","last_became_active":"2024-03-14T12:16:32.886260+0000","last_became_peered":"2024-03-14T12:16:32.886260+0000","last_unstale":"2024-03-14T12:16:32.889957+0000","last_undegraded":"2024-03-14T12:16:32.889957+0000","last_fullsized":"2024-03-14T12:16:32.889957+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:12.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900492+0000","last_change":"2024-03-14T12:16:31.881489+0000","last_active":"2024-03-14T12:16:32.900492+0000","last_peered":"2024-03-14T12:16:32.900492+0000","last_clean":"2024-03-14T12:16:32.900492+0000","last_became_active":"2024-03-14T12:16:31.881197+0000","last_became_peered":"2024-03-14T12:16:31.881197+0000","last_unstale":"2024-03-14T12:16:32.900492+0000","last_undegraded":"2024-03-14T12:16:32.900492+0000","last_fullsized":"2024-03-14T12:16:32.900492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:09.852408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898228+0000","last_change":"2024-03-14T12:16:32.894357+0000","last_active":"2024-03-14T12:16:32.898228+0000","last_peered":"2024-03-14T12:16:32.898228+0000","last_clean":"2024-03-14T12:16:32.898228+0000","last_became_active":"2024-03-14T12:16:32.893751+0000","last_became_peered":"2024-03-14T12:16:32.893751+0000","last_unstale":"2024-03-14T12:16:32.898228+0000","last_undegraded":"2024-03-14T12:16:32.898228+0000","last_fullsized":"2024-03-14T12:16:32.898228+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:51.995841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897167+0000","last_change":"2024-03-14T12:16:31.885662+0000","last_active":"2024-03-14T12:16:32.897167+0000","last_peered":"2024-03-14T12:16:32.897167+0000","last_clean":"2024-03-14T12:16:32.897167+0000","last_became_active":"2024-03-14T12:16:31.885355+0000","last_became_peered":"2024-03-14T12:16:31.885355+0000","last_unstale":"2024-03-14T12:16:32.897167+0000","last_undegraded":"2024-03-14T12:16:32.897167+0000","last_fullsized":"2024-03-14T12:16:32.897167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:35.243213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898220+0000","last_change":"2024-03-14T12:16:32.893151+0000","last_active":"2024-03-14T12:16:32.898220+0000","last_peered":"2024-03-14T12:16:32.898220+0000","last_clean":"2024-03-14T12:16:32.898220+0000","last_became_active":"2024-03-14T12:16:32.892891+0000","last_became_peered":"2024-03-14T12:16:32.892891+0000","last_unstale":"2024-03-14T12:16:32.898220+0000","last_undegraded":"2024-03-14T12:16:32.898220+0000","last_fullsized":"2024-03-14T12:16:32.898220+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:57.663772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.899888+0000","last_change":"2024-03-14T12:16:31.882606+0000","last_active":"2024-03-14T12:16:32.899888+0000","last_peered":"2024-03-14T12:16:32.899888+0000","last_clean":"2024-03-14T12:16:32.899888+0000","last_became_active":"2024-03-14T12:16:31.882280+0000","last_became_peered":"2024-03-14T12:16:31.882280+0000","last_unstale":"2024-03-14T12:16:32.899888+0000","last_undegraded":"2024-03-14T12:16:32.899888+0000","last_fullsized":"2024-03-14T12:16:32.899888+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:48:06.040703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.885169+0000","last_change":"2024-03-14T12:16:32.882844+0000","last_active":"2024-03-14T12:16:32.885169+0000","last_peered":"2024-03-14T12:16:32.885169+0000","last_clean":"2024-03-14T12:16:32.885169+0000","last_became_active":"2024-03-14T12:16:32.882466+0000","last_became_peered":"2024-03-14T12:16:32.882466+0000","last_unstale":"2024-03-14T12:16:32.885169+0000","last_undegraded":"2024-03-14T12:16:32.885169+0000","last_fullsized":"2024-03-14T12:16:32.885169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:52:45.064420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.872561+0000","last_change":"2024-03-14T12:16:31.886002+0000","last_active":"2024-03-14T12:16:32.872561+0000","last_peered":"2024-03-14T12:16:32.872561+0000","last_clean":"2024-03-14T12:16:32.872561+0000","last_became_active":"2024-03-14T12:16:31.885710+0000","last_became_peered":"2024-03-14T12:16:31.885710+0000","last_unstale":"2024-03-14T12:16:32.872561+0000","last_undegraded":"2024-03-14T12:16:32.872561+0000","last_fullsized":"2024-03-14T12:16:32.872561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:03:24.346029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896729+0000","last_change":"2024-03-14T12:16:32.885267+0000","last_active":"2024-03-14T12:16:32.896729+0000","last_peered":"2024-03-14T12:16:32.896729+0000","last_clean":"2024-03-14T12:16:32.896729+0000","last_became_active":"2024-03-14T12:16:32.884979+0000","last_became_peered":"2024-03-14T12:16:32.884979+0000","last_unstale":"2024-03-14T12:16:32.896729+0000","last_undegraded":"2024-03-14T12:16:32.896729+0000","last_fullsized":"2024-03-14T12:16:32.896729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:33:19.774417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.874359+0000","last_change":"2024-03-14T12:16:31.886284+0000","last_active":"2024-03-14T12:16:32.874359+0000","last_peered":"2024-03-14T12:16:32.874359+0000","last_clean":"2024-03-14T12:16:32.874359+0000","last_became_active":"2024-03-14T12:16:31.885976+0000","last_became_peered":"2024-03-14T12:16:31.885976+0000","last_unstale":"2024-03-14T12:16:32.874359+0000","last_undegraded":"2024-03-14T12:16:32.874359+0000","last_fullsized":"2024-03-14T12:16:32.874359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:01.343460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.886533+0000","last_change":"2024-03-14T12:16:32.879719+0000","last_active":"2024-03-14T12:16:32.886533+0000","last_peered":"2024-03-14T12:16:32.886533+0000","last_clean":"2024-03-14T12:16:32.886533+0000","last_became_active":"2024-03-14T12:16:32.879397+0000","last_became_peered":"2024-03-14T12:16:32.879397+0000","last_unstale":"2024-03-14T12:16:32.886533+0000","last_undegraded":"2024-03-14T12:16:32.886533+0000","last_fullsized":"2024-03-14T12:16:32.886533+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:26:14.651402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.880199+0000","last_change":"2024-03-14T12:16:31.884864+0000","last_active":"2024-03-14T12:16:32.880199+0000","last_peered":"2024-03-14T12:16:32.880199+0000","last_clean":"2024-03-14T12:16:32.880199+0000","last_became_active":"2024-03-14T12:16:31.884470+0000","last_became_peered":"2024-03-14T12:16:31.884470+0000","last_unstale":"2024-03-14T12:16:32.880199+0000","last_undegraded":"2024-03-14T12:16:32.880199+0000","last_fullsized":"2024-03-14T12:16:32.880199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:43:16.206207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900137+0000","last_change":"2024-03-14T12:16:32.893064+0000","last_active":"2024-03-14T12:16:32.900137+0000","last_peered":"2024-03-14T12:16:32.900137+0000","last_clean":"2024-03-14T12:16:32.900137+0000","last_became_active":"2024-03-14T12:16:32.892541+0000","last_became_peered":"2024-03-14T12:16:32.892541+0000","last_unstale":"2024-03-14T12:16:32.900137+0000","last_undegraded":"2024-03-14T12:16:32.900137+0000","last_fullsized":"2024-03-14T12:16:32.900137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:20.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.875312+0000","last_change":"2024-03-14T12:16:31.889669+0000","last_active":"2024-03-14T12:16:32.875312+0000","last_peered":"2024-03-14T12:16:32.875312+0000","last_clean":"2024-03-14T12:16:32.875312+0000","last_became_active":"2024-03-14T12:16:31.889373+0000","last_became_peered":"2024-03-14T12:16:31.889373+0000","last_unstale":"2024-03-14T12:16:32.875312+0000","last_undegraded":"2024-03-14T12:16:32.875312+0000","last_fullsized":"2024-03-14T12:16:32.875312+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:09.596105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.887691+0000","last_change":"2024-03-14T12:16:32.880312+0000","last_active":"2024-03-14T12:16:32.887691+0000","last_peered":"2024-03-14T12:16:32.887691+0000","last_clean":"2024-03-14T12:16:32.887691+0000","last_became_active":"2024-03-14T12:16:32.879821+0000","last_became_peered":"2024-03-14T12:16:32.879821+0000","last_unstale":"2024-03-14T12:16:32.887691+0000","last_undegraded":"2024-03-14T12:16:32.887691+0000","last_fullsized":"2024-03-14T12:16:32.887691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:01.676794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900739+0000","last_change":"2024-03-14T12:16:31.886478+0000","last_active":"2024-03-14T12:16:32.900739+0000","last_peered":"2024-03-14T12:16:32.900739+0000","last_clean":"2024-03-14T12:16:32.900739+0000","last_became_active":"2024-03-14T12:16:31.885986+0000","last_became_peered":"2024-03-14T12:16:31.885986+0000","last_unstale":"2024-03-14T12:16:32.900739+0000","last_undegraded":"2024-03-14T12:16:32.900739+0000","last_fullsized":"2024-03-14T12:16:32.900739+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:03:24.346029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900234+0000","last_change":"2024-03-14T12:16:32.893176+0000","last_active":"2024-03-14T12:16:32.900234+0000","last_peered":"2024-03-14T12:16:32.900234+0000","last_clean":"2024-03-14T12:16:32.900234+0000","last_became_active":"2024-03-14T12:16:32.892615+0000","last_became_peered":"2024-03-14T12:16:32.892615+0000","last_unstale":"2024-03-14T12:16:32.900234+0000","last_undegraded":"2024-03-14T12:16:32.900234+0000","last_fullsized":"2024-03-14T12:16:32.900234+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:18:14.483997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900574+0000","last_change":"2024-03-14T12:16:31.875858+0000","last_active":"2024-03-14T12:16:32.900574+0000","last_peered":"2024-03-14T12:16:32.900574+0000","last_clean":"2024-03-14T12:16:32.900574+0000","last_became_active":"2024-03-14T12:16:31.875552+0000","last_became_peered":"2024-03-14T12:16:31.875552+0000","last_unstale":"2024-03-14T12:16:32.900574+0000","last_undegraded":"2024-03-14T12:16:32.900574+0000","last_fullsized":"2024-03-14T12:16:32.900574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:51:14.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.884081+0000","last_change":"2024-03-14T12:16:32.880604+0000","last_active":"2024-03-14T12:16:32.884081+0000","last_peered":"2024-03-14T12:16:32.884081+0000","last_clean":"2024-03-14T12:16:32.884081+0000","last_became_active":"2024-03-14T12:16:32.880301+0000","last_became_peered":"2024-03-14T12:16:32.880301+0000","last_unstale":"2024-03-14T12:16:32.884081+0000","last_undegraded":"2024-03-14T12:16:32.884081+0000","last_fullsized":"2024-03-14T12:16:32.884081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:18:14.483997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894596+0000","last_change":"2024-03-14T12:16:31.886919+0000","last_active":"2024-03-14T12:16:32.894596+0000","last_peered":"2024-03-14T12:16:32.894596+0000","last_clean":"2024-03-14T12:16:32.894596+0000","last_became_active":"2024-03-14T12:16:31.886603+0000","last_became_peered":"2024-03-14T12:16:31.886603+0000","last_unstale":"2024-03-14T12:16:32.894596+0000","last_undegraded":"2024-03-14T12:16:32.894596+0000","last_fullsized":"2024-03-14T12:16:32.894596+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:19.867306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888144+0000","last_change":"2024-03-14T12:16:32.886021+0000","last_active":"2024-03-14T12:16:32.888144+0000","last_peered":"2024-03-14T12:16:32.888144+0000","last_clean":"2024-03-14T12:16:32.888144+0000","last_became_active":"2024-03-14T12:16:32.885661+0000","last_became_peered":"2024-03-14T12:16:32.885661+0000","last_unstale":"2024-03-14T12:16:32.888144+0000","last_undegraded":"2024-03-14T12:16:32.888144+0000","last_fullsized":"2024-03-14T12:16:32.888144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:10.878467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876775+0000","last_change":"2024-03-14T12:16:31.885073+0000","last_active":"2024-03-14T12:16:32.876775+0000","last_peered":"2024-03-14T12:16:32.876775+0000","last_clean":"2024-03-14T12:16:32.876775+0000","last_became_active":"2024-03-14T12:16:31.884785+0000","last_became_peered":"2024-03-14T12:16:31.884785+0000","last_unstale":"2024-03-14T12:16:32.876775+0000","last_undegraded":"2024-03-14T12:16:32.876775+0000","last_fullsized":"2024-03-14T12:16:32.876775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:16.142831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894427+0000","last_change":"2024-03-14T12:16:32.882923+0000","last_active":"2024-03-14T12:16:32.894427+0000","last_peered":"2024-03-14T12:16:32.894427+0000","last_clean":"2024-03-14T12:16:32.894427+0000","last_became_active":"2024-03-14T12:16:32.882634+0000","last_became_peered":"2024-03-14T12:16:32.882634+0000","last_unstale":"2024-03-14T12:16:32.894427+0000","last_undegraded":"2024-03-14T12:16:32.894427+0000","last_fullsized":"2024-03-14T12:16:32.894427+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:43:17.211532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.872716+0000","last_change":"2024-03-14T12:16:31.881166+0000","last_active":"2024-03-14T12:16:32.872716+0000","last_peered":"2024-03-14T12:16:32.872716+0000","last_clean":"2024-03-14T12:16:32.872716+0000","last_became_active":"2024-03-14T12:16:31.880870+0000","last_became_peered":"2024-03-14T12:16:31.880870+0000","last_unstale":"2024-03-14T12:16:32.872716+0000","last_undegraded":"2024-03-14T12:16:32.872716+0000","last_fullsized":"2024-03-14T12:16:32.872716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:09.873142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900885+0000","last_change":"2024-03-14T12:16:32.897760+0000","last_active":"2024-03-14T12:16:32.900885+0000","last_peered":"2024-03-14T12:16:32.900885+0000","last_clean":"2024-03-14T12:16:32.900885+0000","last_became_active":"2024-03-14T12:16:32.897302+0000","last_became_peered":"2024-03-14T12:16:32.897302+0000","last_unstale":"2024-03-14T12:16:32.900885+0000","last_undegraded":"2024-03-14T12:16:32.900885+0000","last_fullsized":"2024-03-14T12:16:32.900885+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:24.479669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.901287+0000","last_change":"2024-03-14T12:16:31.885316+0000","last_active":"2024-03-14T12:16:32.901287+0000","last_peered":"2024-03-14T12:16:32.901287+0000","last_clean":"2024-03-14T12:16:32.901287+0000","last_became_active":"2024-03-14T12:16:31.884997+0000","last_became_peered":"2024-03-14T12:16:31.884997+0000","last_unstale":"2024-03-14T12:16:32.901287+0000","last_undegraded":"2024-03-14T12:16:32.901287+0000","last_fullsized":"2024-03-14T12:16:32.901287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:26:13.646077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900759+0000","last_change":"2024-03-14T12:16:32.895200+0000","last_active":"2024-03-14T12:16:32.900759+0000","last_peered":"2024-03-14T12:16:32.900759+0000","last_clean":"2024-03-14T12:16:32.900759+0000","last_became_active":"2024-03-14T12:16:32.894903+0000","last_became_peered":"2024-03-14T12:16:32.894903+0000","last_unstale":"2024-03-14T12:16:32.900759+0000","last_undegraded":"2024-03-14T12:16:32.900759+0000","last_fullsized":"2024-03-14T12:16:32.900759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:56:56.014598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.893207+0000","last_change":"2024-03-14T12:16:31.886947+0000","last_active":"2024-03-14T12:16:32.893207+0000","last_peered":"2024-03-14T12:16:32.893207+0000","last_clean":"2024-03-14T12:16:32.893207+0000","last_became_active":"2024-03-14T12:16:31.886429+0000","last_became_peered":"2024-03-14T12:16:31.886429+0000","last_unstale":"2024-03-14T12:16:32.893207+0000","last_undegraded":"2024-03-14T12:16:32.893207+0000","last_fullsized":"2024-03-14T12:16:32.893207+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:19.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.882276+0000","last_change":"2024-03-14T12:16:32.880478+0000","last_active":"2024-03-14T12:16:32.882276+0000","last_peered":"2024-03-14T12:16:32.882276+0000","last_clean":"2024-03-14T12:16:32.882276+0000","last_became_active":"2024-03-14T12:16:32.879920+0000","last_became_peered":"2024-03-14T12:16:32.879920+0000","last_unstale":"2024-03-14T12:16:32.882276+0000","last_undegraded":"2024-03-14T12:16:32.882276+0000","last_fullsized":"2024-03-14T12:16:32.882276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:20.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.895437+0000","last_change":"2024-03-14T12:16:31.885893+0000","last_active":"2024-03-14T12:16:32.895437+0000","last_peered":"2024-03-14T12:16:32.895437+0000","last_clean":"2024-03-14T12:16:32.895437+0000","last_became_active":"2024-03-14T12:16:31.885641+0000","last_became_peered":"2024-03-14T12:16:31.885641+0000","last_unstale":"2024-03-14T12:16:32.895437+0000","last_undegraded":"2024-03-14T12:16:32.895437+0000","last_fullsized":"2024-03-14T12:16:32.895437+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:18:13.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897922+0000","last_change":"2024-03-14T12:16:32.892330+0000","last_active":"2024-03-14T12:16:32.897922+0000","last_peered":"2024-03-14T12:16:32.897922+0000","last_clean":"2024-03-14T12:16:32.897922+0000","last_became_active":"2024-03-14T12:16:32.891083+0000","last_became_peered":"2024-03-14T12:16:32.891083+0000","last_unstale":"2024-03-14T12:16:32.897922+0000","last_undegraded":"2024-03-14T12:16:32.897922+0000","last_fullsized":"2024-03-14T12:16:32.897922+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:39:57.733790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900279+0000","last_change":"2024-03-14T12:16:31.883235+0000","last_active":"2024-03-14T12:16:32.900279+0000","last_peered":"2024-03-14T12:16:32.900279+0000","last_clean":"2024-03-14T12:16:32.900279+0000","last_became_active":"2024-03-14T12:16:31.882956+0000","last_became_peered":"2024-03-14T12:16:31.882956+0000","last_unstale":"2024-03-14T12:16:32.900279+0000","last_undegraded":"2024-03-14T12:16:32.900279+0000","last_fullsized":"2024-03-14T12:16:32.900279+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:26:13.646077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900214+0000","last_change":"2024-03-14T12:16:32.893013+0000","last_active":"2024-03-14T12:16:32.900214+0000","last_peered":"2024-03-14T12:16:32.900214+0000","last_clean":"2024-03-14T12:16:32.900214+0000","last_became_active":"2024-03-14T12:16:32.892679+0000","last_became_peered":"2024-03-14T12:16:32.892679+0000","last_unstale":"2024-03-14T12:16:32.900214+0000","last_undegraded":"2024-03-14T12:16:32.900214+0000","last_fullsized":"2024-03-14T12:16:32.900214+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:48:07.046028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.877416+0000","last_change":"2024-03-14T12:16:31.881293+0000","last_active":"2024-03-14T12:16:32.877416+0000","last_peered":"2024-03-14T12:16:32.877416+0000","last_clean":"2024-03-14T12:16:32.877416+0000","last_became_active":"2024-03-14T12:16:31.881003+0000","last_became_peered":"2024-03-14T12:16:31.881003+0000","last_unstale":"2024-03-14T12:16:32.877416+0000","last_undegraded":"2024-03-14T12:16:32.877416+0000","last_fullsized":"2024-03-14T12:16:32.877416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:19.251545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.886391+0000","last_change":"2024-03-14T12:16:32.880935+0000","last_active":"2024-03-14T12:16:32.886391+0000","last_peered":"2024-03-14T12:16:32.886391+0000","last_clean":"2024-03-14T12:16:32.886391+0000","last_became_active":"2024-03-14T12:16:32.880502+0000","last_became_peered":"2024-03-14T12:16:32.880502+0000","last_unstale":"2024-03-14T12:16:32.886391+0000","last_undegraded":"2024-03-14T12:16:32.886391+0000","last_fullsized":"2024-03-14T12:16:32.886391+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:56:56.014598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898142+0000","last_change":"2024-03-14T12:16:31.890069+0000","last_active":"2024-03-14T12:16:32.898142+0000","last_peered":"2024-03-14T12:16:32.898142+0000","last_clean":"2024-03-14T12:16:32.898142+0000","last_became_active":"2024-03-14T12:16:31.889746+0000","last_became_peered":"2024-03-14T12:16:31.889746+0000","last_unstale":"2024-03-14T12:16:32.898142+0000","last_undegraded":"2024-03-14T12:16:32.898142+0000","last_fullsized":"2024-03-14T12:16:32.898142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:12.021144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888290+0000","last_change":"2024-03-14T12:16:32.884896+0000","last_active":"2024-03-14T12:16:32.888290+0000","last_peered":"2024-03-14T12:16:32.888290+0000","last_clean":"2024-03-14T12:16:32.888290+0000","last_became_active":"2024-03-14T12:16:32.884606+0000","last_became_peered":"2024-03-14T12:16:32.884606+0000","last_unstale":"2024-03-14T12:16:32.888290+0000","last_undegraded":"2024-03-14T12:16:32.888290+0000","last_fullsized":"2024-03-14T12:16:32.888290+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:08.569373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.871643+0000","last_change":"2024-03-14T12:16:31.887671+0000","last_active":"2024-03-14T12:16:32.871643+0000","last_peered":"2024-03-14T12:16:32.871643+0000","last_clean":"2024-03-14T12:16:32.871643+0000","last_became_active":"2024-03-14T12:16:31.887221+0000","last_became_peered":"2024-03-14T12:16:31.887221+0000","last_unstale":"2024-03-14T12:16:32.871643+0000","last_undegraded":"2024-03-14T12:16:32.871643+0000","last_fullsized":"2024-03-14T12:16:32.871643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:15:29.488761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.890793+0000","last_change":"2024-03-14T12:16:32.888030+0000","last_active":"2024-03-14T12:16:32.890793+0000","last_peered":"2024-03-14T12:16:32.890793+0000","last_clean":"2024-03-14T12:16:32.890793+0000","last_became_active":"2024-03-14T12:16:32.887755+0000","last_became_peered":"2024-03-14T12:16:32.887755+0000","last_unstale":"2024-03-14T12:16:32.890793+0000","last_undegraded":"2024-03-14T12:16:32.890793+0000","last_fullsized":"2024-03-14T12:16:32.890793+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:57.398996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.872012+0000","last_change":"2024-03-14T12:16:31.882034+0000","last_active":"2024-03-14T12:16:32.872012+0000","last_peered":"2024-03-14T12:16:32.872012+0000","last_clean":"2024-03-14T12:16:32.872012+0000","last_became_active":"2024-03-14T12:16:31.881742+0000","last_became_peered":"2024-03-14T12:16:31.881742+0000","last_unstale":"2024-03-14T12:16:32.872012+0000","last_undegraded":"2024-03-14T12:16:32.872012+0000","last_fullsized":"2024-03-14T12:16:32.872012+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:43:16.206207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900273+0000","last_change":"2024-03-14T12:16:32.891551+0000","last_active":"2024-03-14T12:16:32.900273+0000","last_peered":"2024-03-14T12:16:32.900273+0000","last_clean":"2024-03-14T12:16:32.900273+0000","last_became_active":"2024-03-14T12:16:32.891208+0000","last_became_peered":"2024-03-14T12:16:32.891208+0000","last_unstale":"2024-03-14T12:16:32.900273+0000","last_undegraded":"2024-03-14T12:16:32.900273+0000","last_fullsized":"2024-03-14T12:16:32.900273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:24.479669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897359+0000","last_change":"2024-03-14T12:16:31.881543+0000","last_active":"2024-03-14T12:16:32.897359+0000","last_peered":"2024-03-14T12:16:32.897359+0000","last_clean":"2024-03-14T12:16:32.897359+0000","last_became_active":"2024-03-14T12:16:31.881296+0000","last_became_peered":"2024-03-14T12:16:31.881296+0000","last_unstale":"2024-03-14T12:16:32.897359+0000","last_undegraded":"2024-03-14T12:16:32.897359+0000","last_fullsized":"2024-03-14T12:16:32.897359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:16.142831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900169+0000","last_change":"2024-03-14T12:16:32.889400+0000","last_active":"2024-03-14T12:16:32.900169+0000","last_peered":"2024-03-14T12:16:32.900169+0000","last_clean":"2024-03-14T12:16:32.900169+0000","last_became_active":"2024-03-14T12:16:32.888938+0000","last_became_peered":"2024-03-14T12:16:32.888938+0000","last_unstale":"2024-03-14T12:16:32.900169+0000","last_undegraded":"2024-03-14T12:16:32.900169+0000","last_fullsized":"2024-03-14T12:16:32.900169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:08.569373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.877666+0000","last_change":"2024-03-14T12:16:31.881853+0000","last_active":"2024-03-14T12:16:32.877666+0000","last_peered":"2024-03-14T12:16:32.877666+0000","last_clean":"2024-03-14T12:16:32.877666+0000","last_became_active":"2024-03-14T12:16:31.881521+0000","last_became_peered":"2024-03-14T12:16:31.881521+0000","last_unstale":"2024-03-14T12:16:32.877666+0000","last_undegraded":"2024-03-14T12:16:32.877666+0000","last_fullsized":"2024-03-14T12:16:32.877666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:56:55.009273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889289+0000","last_change":"2024-03-14T12:16:32.882894+0000","last_active":"2024-03-14T12:16:32.889289+0000","last_peered":"2024-03-14T12:16:32.889289+0000","last_clean":"2024-03-14T12:16:32.889289+0000","last_became_active":"2024-03-14T12:16:32.882257+0000","last_became_peered":"2024-03-14T12:16:32.882257+0000","last_unstale":"2024-03-14T12:16:32.889289+0000","last_undegraded":"2024-03-14T12:16:32.889289+0000","last_fullsized":"2024-03-14T12:16:32.889289+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:52:45.064420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876889+0000","last_change":"2024-03-14T12:16:31.882804+0000","last_active":"2024-03-14T12:16:32.876889+0000","last_peered":"2024-03-14T12:16:32.876889+0000","last_clean":"2024-03-14T12:16:32.876889+0000","last_became_active":"2024-03-14T12:16:31.882490+0000","last_became_peered":"2024-03-14T12:16:31.882490+0000","last_unstale":"2024-03-14T12:16:32.876889+0000","last_undegraded":"2024-03-14T12:16:32.876889+0000","last_fullsized":"2024-03-14T12:16:32.876889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:00.671469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897555+0000","last_change":"2024-03-14T12:16:32.895726+0000","last_active":"2024-03-14T12:16:32.897555+0000","last_peered":"2024-03-14T12:16:32.897555+0000","last_clean":"2024-03-14T12:16:32.897555+0000","last_became_active":"2024-03-14T12:16:32.895425+0000","last_became_peered":"2024-03-14T12:16:32.895425+0000","last_unstale":"2024-03-14T12:16:32.897555+0000","last_undegraded":"2024-03-14T12:16:32.897555+0000","last_fullsized":"2024-03-14T12:16:32.897555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:18.468220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.873462+0000","last_change":"2024-03-14T12:16:31.885730+0000","last_active":"2024-03-14T12:16:32.873462+0000","last_peered":"2024-03-14T12:16:32.873462+0000","last_clean":"2024-03-14T12:16:32.873462+0000","last_became_active":"2024-03-14T12:16:31.885455+0000","last_became_peered":"2024-03-14T12:16:31.885455+0000","last_unstale":"2024-03-14T12:16:32.873462+0000","last_undegraded":"2024-03-14T12:16:32.873462+0000","last_fullsized":"2024-03-14T12:16:32.873462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:25:11.753733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897256+0000","last_change":"2024-03-14T12:16:32.891495+0000","last_active":"2024-03-14T12:16:32.897256+0000","last_peered":"2024-03-14T12:16:32.897256+0000","last_clean":"2024-03-14T12:16:32.897256+0000","last_became_active":"2024-03-14T12:16:32.891174+0000","last_became_peered":"2024-03-14T12:16:32.891174+0000","last_unstale":"2024-03-14T12:16:32.897256+0000","last_undegraded":"2024-03-14T12:16:32.897256+0000","last_fullsized":"2024-03-14T12:16:32.897256+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:09:27.650527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.871377+0000","last_change":"2024-03-14T12:16:31.886036+0000","last_active":"2024-03-14T12:16:32.871377+0000","last_peered":"2024-03-14T12:16:32.871377+0000","last_clean":"2024-03-14T12:16:32.871377+0000","last_became_active":"2024-03-14T12:16:31.885743+0000","last_became_peered":"2024-03-14T12:16:31.885743+0000","last_unstale":"2024-03-14T12:16:32.871377+0000","last_undegraded":"2024-03-14T12:16:32.871377+0000","last_fullsized":"2024-03-14T12:16:32.871377+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:01:34.400093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900852+0000","last_change":"2024-03-14T12:16:32.898040+0000","last_active":"2024-03-14T12:16:32.900852+0000","last_peered":"2024-03-14T12:16:32.900852+0000","last_clean":"2024-03-14T12:16:32.900852+0000","last_became_active":"2024-03-14T12:16:32.897767+0000","last_became_peered":"2024-03-14T12:16:32.897767+0000","last_unstale":"2024-03-14T12:16:32.900852+0000","last_undegraded":"2024-03-14T12:16:32.900852+0000","last_fullsized":"2024-03-14T12:16:32.900852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:12.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.873122+0000","last_change":"2024-03-14T12:16:31.876723+0000","last_active":"2024-03-14T12:16:32.873122+0000","last_peered":"2024-03-14T12:16:32.873122+0000","last_clean":"2024-03-14T12:16:32.873122+0000","last_became_active":"2024-03-14T12:16:31.876437+0000","last_became_peered":"2024-03-14T12:16:31.876437+0000","last_unstale":"2024-03-14T12:16:32.873122+0000","last_undegraded":"2024-03-14T12:16:32.873122+0000","last_fullsized":"2024-03-14T12:16:32.873122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:14:30.089638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889257+0000","last_change":"2024-03-14T12:16:32.886000+0000","last_active":"2024-03-14T12:16:32.889257+0000","last_peered":"2024-03-14T12:16:32.889257+0000","last_clean":"2024-03-14T12:16:32.889257+0000","last_became_active":"2024-03-14T12:16:32.885526+0000","last_became_peered":"2024-03-14T12:16:32.885526+0000","last_unstale":"2024-03-14T12:16:32.889257+0000","last_undegraded":"2024-03-14T12:16:32.889257+0000","last_fullsized":"2024-03-14T12:16:32.889257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:42:34.697540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.871506+0000","last_change":"2024-03-14T12:16:31.880908+0000","last_active":"2024-03-14T12:16:32.871506+0000","last_peered":"2024-03-14T12:16:32.871506+0000","last_clean":"2024-03-14T12:16:32.871506+0000","last_became_active":"2024-03-14T12:16:31.880643+0000","last_became_peered":"2024-03-14T12:16:31.880643+0000","last_unstale":"2024-03-14T12:16:32.871506+0000","last_undegraded":"2024-03-14T12:16:32.871506+0000","last_fullsized":"2024-03-14T12:16:32.871506+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:36:03.351328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896864+0000","last_change":"2024-03-14T12:16:32.887125+0000","last_active":"2024-03-14T12:16:32.896864+0000","last_peered":"2024-03-14T12:16:32.896864+0000","last_clean":"2024-03-14T12:16:32.896864+0000","last_became_active":"2024-03-14T12:16:32.885810+0000","last_became_peered":"2024-03-14T12:16:32.885810+0000","last_unstale":"2024-03-14T12:16:32.896864+0000","last_undegraded":"2024-03-14T12:16:32.896864+0000","last_fullsized":"2024-03-14T12:16:32.896864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:30:14.944354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876308+0000","last_change":"2024-03-14T12:16:31.885799+0000","last_active":"2024-03-14T12:16:32.876308+0000","last_peered":"2024-03-14T12:16:32.876308+0000","last_clean":"2024-03-14T12:16:32.876308+0000","last_became_active":"2024-03-14T12:16:31.885115+0000","last_became_peered":"2024-03-14T12:16:31.885115+0000","last_unstale":"2024-03-14T12:16:32.876308+0000","last_undegraded":"2024-03-14T12:16:32.876308+0000","last_fullsized":"2024-03-14T12:16:32.876308+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:20.298469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898281+0000","last_change":"2024-03-14T12:16:32.894164+0000","last_active":"2024-03-14T12:16:32.898281+0000","last_peered":"2024-03-14T12:16:32.898281+0000","last_clean":"2024-03-14T12:16:32.898281+0000","last_became_active":"2024-03-14T12:16:32.893647+0000","last_became_peered":"2024-03-14T12:16:32.893647+0000","last_unstale":"2024-03-14T12:16:32.898281+0000","last_undegraded":"2024-03-14T12:16:32.898281+0000","last_fullsized":"2024-03-14T12:16:32.898281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:02:36.750291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876971+0000","last_change":"2024-03-14T12:16:31.882679+0000","last_active":"2024-03-14T12:16:32.876971+0000","last_peered":"2024-03-14T12:16:32.876971+0000","last_clean":"2024-03-14T12:16:32.876971+0000","last_became_active":"2024-03-14T12:16:31.882405+0000","last_became_peered":"2024-03-14T12:16:31.882405+0000","last_unstale":"2024-03-14T12:16:32.876971+0000","last_undegraded":"2024-03-14T12:16:32.876971+0000","last_fullsized":"2024-03-14T12:16:32.876971+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:58:26.213092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":236,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.879234+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:16:32.879234+0000","last_peered":"2024-03-14T12:16:32.879234+0000","last_clean":"2024-03-14T12:16:32.879234+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:16:32.879234+0000","last_undegraded":"2024-03-14T12:16:32.879234+0000","last_fullsized":"2024-03-14T12:16:32.879234+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":871952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":281424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":282076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":283036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36024,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335816,"statfs":{"total":96636764160,"available":96599875584,"internally_reserved":0,"allocated":692224,"data_stored":281480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":281480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36000,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335840,"statfs":{"total":96636764160,"available":96599900160,"internally_reserved":0,"allocated":671744,"data_stored":274649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":873986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:16:35.837 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:16:35.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.970+0000 7f1fc44e9640 1 Processor -- start 2024-03-14T12:16:35.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.970+0000 7f1fc44e9640 1 -- start start 2024-03-14T12:16:35.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.971+0000 7f1fc44e9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1fbc0cf370 0x7f1fbc0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.971+0000 7f1fc44e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 0x7f1fbc0b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.971+0000 7f1fc44e9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc0cf8c0 con 0x7f1fbc0d0760 2024-03-14T12:16:35.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.971+0000 7f1fc44e9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fbc0cfba0 con 0x7f1fbc0cf370 2024-03-14T12:16:35.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.971+0000 7f1fc44e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc0b6c40 con 0x7f1fbc0b7720 2024-03-14T12:16:35.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc1a84640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1fbc0cf370 0x7f1fbc0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc2285640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1fbc0d0760 0x7f1fbc0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41104/0 (socket says 172.21.15.82:41104) 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc1a84640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1fbc0cf370 0x7f1fbc0d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50532/0 (socket says 172.21.15.82:50532) 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc2285640 1 -- 172.21.15.82:0/3874167561 learned_addr learned my addr 172.21.15.82:0/3874167561 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc2a86640 1 --2- 172.21.15.82:0/3874167561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 0x7f1fbc0b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc1a84640 1 -- 172.21.15.82:0/3874167561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 msgr2=0x7f1fbc0b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc1a84640 1 --2- 172.21.15.82:0/3874167561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 0x7f1fbc0b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc1a84640 1 -- 172.21.15.82:0/3874167561 >> v1:172.21.15.82:6789/0 conn(0x7f1fbc0d0760 legacy=0x7f1fbc0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc1a84640 1 -- 172.21.15.82:0/3874167561 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc0b6f20 con 0x7f1fbc0cf370 2024-03-14T12:16:35.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.972+0000 7f1fc1283640 1 -- 172.21.15.82:0/3874167561 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240086916 0 0) 0x7f1fbc0cf8c0 con 0x7f1fbc0d0760 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc1a84640 1 --2- 172.21.15.82:0/3874167561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc0d4660 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097d0 tx=0x7f1fac02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab30341b269b9d97 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc1283640 1 -- 172.21.15.82:0/3874167561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac035020 con 0x7f1fbc0cf370 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc1283640 1 -- 172.21.15.82:0/3874167561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac031890 con 0x7f1fbc0cf370 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc1283640 1 -- 172.21.15.82:0/3874167561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac031c60 con 0x7f1fbc0cf370 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 -- 172.21.15.82:0/3874167561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 msgr2=0x7f1fbc0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/3874167561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc0d4660 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097d0 tx=0x7f1fac02b930 comp rx=0 tx=0).stop 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 -- 172.21.15.82:0/3874167561 shutdown_connections 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/3874167561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 0x7f1fbc0b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/3874167561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc0d4660 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 -- 172.21.15.82:0/3874167561 >> 172.21.15.82:0/3874167561 conn(0x7f1fbc09d030 msgr2=0x7f1fbc0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 -- 172.21.15.82:0/3874167561 shutdown_connections 2024-03-14T12:16:35.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.973+0000 7f1fc44e9640 1 -- 172.21.15.82:0/3874167561 wait complete. 2024-03-14T12:16:35.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 Processor -- start 2024-03-14T12:16:35.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 -- start start 2024-03-14T12:16:35.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 0x7f1fbc14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fbc0d0760 0x7f1fbc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc1617a0 con 0x7f1fbc0cf370 2024-03-14T12:16:35.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc0d4ba0 con 0x7f1fbc0d0760 2024-03-14T12:16:35.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc44e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc161920 con 0x7f1fbc0b7720 2024-03-14T12:16:35.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc1a84640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc1a84640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50540/0 (socket says 172.21.15.82:50540) 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.975+0000 7f1fc1a84640 1 -- 172.21.15.82:0/2457858948 learned_addr learned my addr 172.21.15.82:0/2457858948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc1a84640 1 -- 172.21.15.82:0/2457858948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 msgr2=0x7f1fbc14d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc1a84640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 0x7f1fbc14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc1a84640 1 -- 172.21.15.82:0/2457858948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fbc0d0760 msgr2=0x7f1fbc15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc1a84640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fbc0d0760 0x7f1fbc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc1a84640 1 -- 172.21.15.82:0/2457858948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc14d870 con 0x7f1fbc0cf370 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc1a84640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc15cc80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0098f0 tx=0x7f1fac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fb2ffd640 1 -- 172.21.15.82:0/2457858948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac035050 con 0x7f1fbc0cf370 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fb2ffd640 1 -- 172.21.15.82:0/2457858948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac03bd70 con 0x7f1fbc0cf370 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc0c8ed0 con 0x7f1fbc0cf370 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fb2ffd640 1 -- 172.21.15.82:0/2457858948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac004230 con 0x7f1fbc0cf370 2024-03-14T12:16:35.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.976+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc0c8120 con 0x7f1fbc0cf370 2024-03-14T12:16:35.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.977+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc04f050 con 0x7f1fbc0cf370 2024-03-14T12:16:35.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.977+0000 7f1fb2ffd640 1 -- 172.21.15.82:0/2457858948 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1fac038020 con 0x7f1fbc0cf370 2024-03-14T12:16:35.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.978+0000 7f1fb2ffd640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fa006bbd0 0x7f1fa006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:35.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.978+0000 7f1fb2ffd640 1 -- 172.21.15.82:0/2457858948 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fac0acda0 con 0x7f1fbc0cf370 2024-03-14T12:16:35.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.978+0000 7f1fc2285640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fa006bbd0 0x7f1fa006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:35.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.978+0000 7f1fc2285640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fa006bbd0 0x7f1fa006e090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4007fd0 tx=0x7f1fb4007960 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:35.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:35.982+0000 7f1fb2ffd640 1 -- 172.21.15.82:0/2457858948 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1fbc14d870 con 0x7f1fbc0cf370 2024-03-14T12:16:36.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.151+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fbc15b4b0 con 0x7f1fa006bbd0 2024-03-14T12:16:36.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.157+0000 7f1fb2ffd640 1 -- 172.21.15.82:0/2457858948 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251883 (secure 0 0 0) 0x7f1fbc15b4b0 con 0x7f1fa006bbd0 2024-03-14T12:16:36.159 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:36.161 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:16:36.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fa006bbd0 msgr2=0x7f1fa006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fa006bbd0 0x7f1fa006e090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4007fd0 tx=0x7f1fb4007960 comp rx=0 tx=0).stop 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 msgr2=0x7f1fbc15cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc15cc80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0098f0 tx=0x7f1fac0046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 shutdown_connections 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1fbc0d0760 0x7f1fbc15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fbc0cf370 0x7f1fbc15cc80 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1fa006bbd0 0x7f1fa006e090 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.159+0000 7f1fc44e9640 1 --2- 172.21.15.82:0/2457858948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1fbc0b7720 0x7f1fbc14d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.160+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 >> 172.21.15.82:0/2457858948 conn(0x7f1fbc09d030 msgr2=0x7f1fbc0d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.160+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 shutdown_connections 2024-03-14T12:16:36.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.160+0000 7f1fc44e9640 1 -- 172.21.15.82:0/2457858948 wait complete. 2024-03-14T12:16:36.171 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-03-14T12:16:35.531735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260184,"kb_used_data":6552,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714536,"statfs":{"total":773094113280,"available":772827684864,"internally_reserved":0,"allocated":6709248,"data_stored":3430083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":259575347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.669293"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900025+0000","last_change":"2024-03-14T12:16:32.897956+0000","last_active":"2024-03-14T12:16:32.900025+0000","last_peered":"2024-03-14T12:16:32.900025+0000","last_clean":"2024-03-14T12:16:32.900025+0000","last_became_active":"2024-03-14T12:16:32.897507+0000","last_became_peered":"2024-03-14T12:16:32.897507+0000","last_unstale":"2024-03-14T12:16:32.900025+0000","last_undegraded":"2024-03-14T12:16:32.900025+0000","last_fullsized":"2024-03-14T12:16:32.900025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:29:30.619269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896721+0000","last_change":"2024-03-14T12:16:32.885186+0000","last_active":"2024-03-14T12:16:32.896721+0000","last_peered":"2024-03-14T12:16:32.896721+0000","last_clean":"2024-03-14T12:16:32.896721+0000","last_became_active":"2024-03-14T12:16:32.884928+0000","last_became_peered":"2024-03-14T12:16:32.884928+0000","last_unstale":"2024-03-14T12:16:32.896721+0000","last_undegraded":"2024-03-14T12:16:32.896721+0000","last_fullsized":"2024-03-14T12:16:32.896721+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:35:57.163688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900800+0000","last_change":"2024-03-14T12:16:32.895727+0000","last_active":"2024-03-14T12:16:32.900800+0000","last_peered":"2024-03-14T12:16:32.900800+0000","last_clean":"2024-03-14T12:16:32.900800+0000","last_became_active":"2024-03-14T12:16:32.895399+0000","last_became_peered":"2024-03-14T12:16:32.895399+0000","last_unstale":"2024-03-14T12:16:32.900800+0000","last_undegraded":"2024-03-14T12:16:32.900800+0000","last_fullsized":"2024-03-14T12:16:32.900800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:19.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900005+0000","last_change":"2024-03-14T12:16:32.890627+0000","last_active":"2024-03-14T12:16:32.900005+0000","last_peered":"2024-03-14T12:16:32.900005+0000","last_clean":"2024-03-14T12:16:32.900005+0000","last_became_active":"2024-03-14T12:16:32.890365+0000","last_became_peered":"2024-03-14T12:16:32.890365+0000","last_unstale":"2024-03-14T12:16:32.900005+0000","last_undegraded":"2024-03-14T12:16:32.900005+0000","last_fullsized":"2024-03-14T12:16:32.900005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:22.674104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897241+0000","last_change":"2024-03-14T12:16:32.896084+0000","last_active":"2024-03-14T12:16:32.897241+0000","last_peered":"2024-03-14T12:16:32.897241+0000","last_clean":"2024-03-14T12:16:32.897241+0000","last_became_active":"2024-03-14T12:16:32.895788+0000","last_became_peered":"2024-03-14T12:16:32.895788+0000","last_unstale":"2024-03-14T12:16:32.897241+0000","last_undegraded":"2024-03-14T12:16:32.897241+0000","last_fullsized":"2024-03-14T12:16:32.897241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:20.830516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896907+0000","last_change":"2024-03-14T12:16:32.892737+0000","last_active":"2024-03-14T12:16:32.896907+0000","last_peered":"2024-03-14T12:16:32.896907+0000","last_clean":"2024-03-14T12:16:32.896907+0000","last_became_active":"2024-03-14T12:16:32.892466+0000","last_became_peered":"2024-03-14T12:16:32.892466+0000","last_unstale":"2024-03-14T12:16:32.896907+0000","last_undegraded":"2024-03-14T12:16:32.896907+0000","last_fullsized":"2024-03-14T12:16:32.896907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:24.983599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894538+0000","last_change":"2024-03-14T12:16:32.884016+0000","last_active":"2024-03-14T12:16:32.894538+0000","last_peered":"2024-03-14T12:16:32.894538+0000","last_clean":"2024-03-14T12:16:32.894538+0000","last_became_active":"2024-03-14T12:16:32.883695+0000","last_became_peered":"2024-03-14T12:16:32.883695+0000","last_unstale":"2024-03-14T12:16:32.894538+0000","last_undegraded":"2024-03-14T12:16:32.894538+0000","last_fullsized":"2024-03-14T12:16:32.894538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:03:25.351354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.891771+0000","last_change":"2024-03-14T12:16:32.883406+0000","last_active":"2024-03-14T12:16:32.891771+0000","last_peered":"2024-03-14T12:16:32.891771+0000","last_clean":"2024-03-14T12:16:32.891771+0000","last_became_active":"2024-03-14T12:16:32.882582+0000","last_became_peered":"2024-03-14T12:16:32.882582+0000","last_unstale":"2024-03-14T12:16:32.891771+0000","last_undegraded":"2024-03-14T12:16:32.891771+0000","last_fullsized":"2024-03-14T12:16:32.891771+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:07:10.557782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894716+0000","last_change":"2024-03-14T12:16:32.885107+0000","last_active":"2024-03-14T12:16:32.894716+0000","last_peered":"2024-03-14T12:16:32.894716+0000","last_clean":"2024-03-14T12:16:32.894716+0000","last_became_active":"2024-03-14T12:16:32.884742+0000","last_became_peered":"2024-03-14T12:16:32.884742+0000","last_unstale":"2024-03-14T12:16:32.894716+0000","last_undegraded":"2024-03-14T12:16:32.894716+0000","last_fullsized":"2024-03-14T12:16:32.894716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:10.878467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889105+0000","last_change":"2024-03-14T12:16:32.883744+0000","last_active":"2024-03-14T12:16:32.889105+0000","last_peered":"2024-03-14T12:16:32.889105+0000","last_clean":"2024-03-14T12:16:32.889105+0000","last_became_active":"2024-03-14T12:16:32.883483+0000","last_became_peered":"2024-03-14T12:16:32.883483+0000","last_unstale":"2024-03-14T12:16:32.889105+0000","last_undegraded":"2024-03-14T12:16:32.889105+0000","last_fullsized":"2024-03-14T12:16:32.889105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:27.750978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888405+0000","last_change":"2024-03-14T12:16:32.881473+0000","last_active":"2024-03-14T12:16:32.888405+0000","last_peered":"2024-03-14T12:16:32.888405+0000","last_clean":"2024-03-14T12:16:32.888405+0000","last_became_active":"2024-03-14T12:16:32.881233+0000","last_became_peered":"2024-03-14T12:16:32.881233+0000","last_unstale":"2024-03-14T12:16:32.888405+0000","last_undegraded":"2024-03-14T12:16:32.888405+0000","last_fullsized":"2024-03-14T12:16:32.888405+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:58:27.218417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.895041+0000","last_change":"2024-03-14T12:16:32.884493+0000","last_active":"2024-03-14T12:16:32.895041+0000","last_peered":"2024-03-14T12:16:32.895041+0000","last_clean":"2024-03-14T12:16:32.895041+0000","last_became_active":"2024-03-14T12:16:32.884220+0000","last_became_peered":"2024-03-14T12:16:32.884220+0000","last_unstale":"2024-03-14T12:16:32.895041+0000","last_undegraded":"2024-03-14T12:16:32.895041+0000","last_fullsized":"2024-03-14T12:16:32.895041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:19.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889250+0000","last_change":"2024-03-14T12:16:32.887567+0000","last_active":"2024-03-14T12:16:32.889250+0000","last_peered":"2024-03-14T12:16:32.889250+0000","last_clean":"2024-03-14T12:16:32.889250+0000","last_became_active":"2024-03-14T12:16:32.887048+0000","last_became_peered":"2024-03-14T12:16:32.887048+0000","last_unstale":"2024-03-14T12:16:32.889250+0000","last_undegraded":"2024-03-14T12:16:32.889250+0000","last_fullsized":"2024-03-14T12:16:32.889250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:42:34.697540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897353+0000","last_change":"2024-03-14T12:16:32.890660+0000","last_active":"2024-03-14T12:16:32.897353+0000","last_peered":"2024-03-14T12:16:32.897353+0000","last_clean":"2024-03-14T12:16:32.897353+0000","last_became_active":"2024-03-14T12:16:32.890374+0000","last_became_peered":"2024-03-14T12:16:32.890374+0000","last_unstale":"2024-03-14T12:16:32.897353+0000","last_undegraded":"2024-03-14T12:16:32.897353+0000","last_fullsized":"2024-03-14T12:16:32.897353+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:10.857734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.887661+0000","last_change":"2024-03-14T12:16:32.879744+0000","last_active":"2024-03-14T12:16:32.887661+0000","last_peered":"2024-03-14T12:16:32.887661+0000","last_clean":"2024-03-14T12:16:32.887661+0000","last_became_active":"2024-03-14T12:16:32.879468+0000","last_became_peered":"2024-03-14T12:16:32.879468+0000","last_unstale":"2024-03-14T12:16:32.887661+0000","last_undegraded":"2024-03-14T12:16:32.887661+0000","last_fullsized":"2024-03-14T12:16:32.887661+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:17:16.261933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889964+0000","last_change":"2024-03-14T12:16:32.886008+0000","last_active":"2024-03-14T12:16:32.889964+0000","last_peered":"2024-03-14T12:16:32.889964+0000","last_clean":"2024-03-14T12:16:32.889964+0000","last_became_active":"2024-03-14T12:16:32.885737+0000","last_became_peered":"2024-03-14T12:16:32.885737+0000","last_unstale":"2024-03-14T12:16:32.889964+0000","last_undegraded":"2024-03-14T12:16:32.889964+0000","last_fullsized":"2024-03-14T12:16:32.889964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:24.479669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898365+0000","last_change":"2024-03-14T12:16:32.892148+0000","last_active":"2024-03-14T12:16:32.898365+0000","last_peered":"2024-03-14T12:16:32.898365+0000","last_clean":"2024-03-14T12:16:32.898365+0000","last_became_active":"2024-03-14T12:16:32.891876+0000","last_became_peered":"2024-03-14T12:16:32.891876+0000","last_unstale":"2024-03-14T12:16:32.898365+0000","last_undegraded":"2024-03-14T12:16:32.898365+0000","last_fullsized":"2024-03-14T12:16:32.898365+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:34:04.174384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900253+0000","last_change":"2024-03-14T12:16:32.889876+0000","last_active":"2024-03-14T12:16:32.900253+0000","last_peered":"2024-03-14T12:16:32.900253+0000","last_clean":"2024-03-14T12:16:32.900253+0000","last_became_active":"2024-03-14T12:16:32.889238+0000","last_became_peered":"2024-03-14T12:16:32.889238+0000","last_unstale":"2024-03-14T12:16:32.900253+0000","last_undegraded":"2024-03-14T12:16:32.900253+0000","last_fullsized":"2024-03-14T12:16:32.900253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:25:12.759059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896436+0000","last_change":"2024-03-14T12:16:32.893974+0000","last_active":"2024-03-14T12:16:32.896436+0000","last_peered":"2024-03-14T12:16:32.896436+0000","last_clean":"2024-03-14T12:16:32.896436+0000","last_became_active":"2024-03-14T12:16:32.893706+0000","last_became_peered":"2024-03-14T12:16:32.893706+0000","last_unstale":"2024-03-14T12:16:32.896436+0000","last_undegraded":"2024-03-14T12:16:32.896436+0000","last_fullsized":"2024-03-14T12:16:32.896436+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:30:41.261103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897944+0000","last_change":"2024-03-14T12:16:32.892682+0000","last_active":"2024-03-14T12:16:32.897944+0000","last_peered":"2024-03-14T12:16:32.897944+0000","last_clean":"2024-03-14T12:16:32.897944+0000","last_became_active":"2024-03-14T12:16:32.892416+0000","last_became_peered":"2024-03-14T12:16:32.892416+0000","last_unstale":"2024-03-14T12:16:32.897944+0000","last_undegraded":"2024-03-14T12:16:32.897944+0000","last_fullsized":"2024-03-14T12:16:32.897944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:36:13.000073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900763+0000","last_change":"2024-03-14T12:16:32.895919+0000","last_active":"2024-03-14T12:16:32.900763+0000","last_peered":"2024-03-14T12:16:32.900763+0000","last_clean":"2024-03-14T12:16:32.900763+0000","last_became_active":"2024-03-14T12:16:32.895571+0000","last_became_peered":"2024-03-14T12:16:32.895571+0000","last_unstale":"2024-03-14T12:16:32.900763+0000","last_undegraded":"2024-03-14T12:16:32.900763+0000","last_fullsized":"2024-03-14T12:16:32.900763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:08.569373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898252+0000","last_change":"2024-03-14T12:16:32.894563+0000","last_active":"2024-03-14T12:16:32.898252+0000","last_peered":"2024-03-14T12:16:32.898252+0000","last_clean":"2024-03-14T12:16:32.898252+0000","last_became_active":"2024-03-14T12:16:32.894102+0000","last_became_peered":"2024-03-14T12:16:32.894102+0000","last_unstale":"2024-03-14T12:16:32.898252+0000","last_undegraded":"2024-03-14T12:16:32.898252+0000","last_fullsized":"2024-03-14T12:16:32.898252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:38.560864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900864+0000","last_change":"2024-03-14T12:16:32.897695+0000","last_active":"2024-03-14T12:16:32.900864+0000","last_peered":"2024-03-14T12:16:32.900864+0000","last_clean":"2024-03-14T12:16:32.900864+0000","last_became_active":"2024-03-14T12:16:32.897274+0000","last_became_peered":"2024-03-14T12:16:32.897274+0000","last_unstale":"2024-03-14T12:16:32.900864+0000","last_undegraded":"2024-03-14T12:16:32.900864+0000","last_fullsized":"2024-03-14T12:16:32.900864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:22.674104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889282+0000","last_change":"2024-03-14T12:16:32.887118+0000","last_active":"2024-03-14T12:16:32.889282+0000","last_peered":"2024-03-14T12:16:32.889282+0000","last_clean":"2024-03-14T12:16:32.889282+0000","last_became_active":"2024-03-14T12:16:32.886822+0000","last_became_peered":"2024-03-14T12:16:32.886822+0000","last_unstale":"2024-03-14T12:16:32.889282+0000","last_undegraded":"2024-03-14T12:16:32.889282+0000","last_fullsized":"2024-03-14T12:16:32.889282+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:27.750978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894499+0000","last_change":"2024-03-14T12:16:32.890670+0000","last_active":"2024-03-14T12:16:32.894499+0000","last_peered":"2024-03-14T12:16:32.894499+0000","last_clean":"2024-03-14T12:16:32.894499+0000","last_became_active":"2024-03-14T12:16:32.890305+0000","last_became_peered":"2024-03-14T12:16:32.890305+0000","last_unstale":"2024-03-14T12:16:32.894499+0000","last_undegraded":"2024-03-14T12:16:32.894499+0000","last_fullsized":"2024-03-14T12:16:32.894499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:28.029527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.882245+0000","last_change":"2024-03-14T12:16:32.879318+0000","last_active":"2024-03-14T12:16:32.882245+0000","last_peered":"2024-03-14T12:16:32.882245+0000","last_clean":"2024-03-14T12:16:32.882245+0000","last_became_active":"2024-03-14T12:16:32.879058+0000","last_became_peered":"2024-03-14T12:16:32.879058+0000","last_unstale":"2024-03-14T12:16:32.882245+0000","last_undegraded":"2024-03-14T12:16:32.882245+0000","last_fullsized":"2024-03-14T12:16:32.882245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:48:07.046028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889979+0000","last_change":"2024-03-14T12:16:32.887107+0000","last_active":"2024-03-14T12:16:32.889979+0000","last_peered":"2024-03-14T12:16:32.889979+0000","last_clean":"2024-03-14T12:16:32.889979+0000","last_became_active":"2024-03-14T12:16:32.886631+0000","last_became_peered":"2024-03-14T12:16:32.886631+0000","last_unstale":"2024-03-14T12:16:32.889979+0000","last_undegraded":"2024-03-14T12:16:32.889979+0000","last_fullsized":"2024-03-14T12:16:32.889979+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:23.111789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897101+0000","last_change":"2024-03-14T12:16:32.896082+0000","last_active":"2024-03-14T12:16:32.897101+0000","last_peered":"2024-03-14T12:16:32.897101+0000","last_clean":"2024-03-14T12:16:32.897101+0000","last_became_active":"2024-03-14T12:16:32.895805+0000","last_became_peered":"2024-03-14T12:16:32.895805+0000","last_unstale":"2024-03-14T12:16:32.897101+0000","last_undegraded":"2024-03-14T12:16:32.897101+0000","last_fullsized":"2024-03-14T12:16:32.897101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:09:33.586925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898244+0000","last_change":"2024-03-14T12:16:32.895105+0000","last_active":"2024-03-14T12:16:32.898244+0000","last_peered":"2024-03-14T12:16:32.898244+0000","last_clean":"2024-03-14T12:16:32.898244+0000","last_became_active":"2024-03-14T12:16:32.894808+0000","last_became_peered":"2024-03-14T12:16:32.894808+0000","last_unstale":"2024-03-14T12:16:32.898244+0000","last_undegraded":"2024-03-14T12:16:32.898244+0000","last_fullsized":"2024-03-14T12:16:32.898244+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:11.934373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894562+0000","last_change":"2024-03-14T12:16:32.888432+0000","last_active":"2024-03-14T12:16:32.894562+0000","last_peered":"2024-03-14T12:16:32.894562+0000","last_clean":"2024-03-14T12:16:32.894562+0000","last_became_active":"2024-03-14T12:16:32.888159+0000","last_became_peered":"2024-03-14T12:16:32.888159+0000","last_unstale":"2024-03-14T12:16:32.894562+0000","last_undegraded":"2024-03-14T12:16:32.894562+0000","last_fullsized":"2024-03-14T12:16:32.894562+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:10.857734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.891088+0000","last_change":"2024-03-14T12:16:32.890285+0000","last_active":"2024-03-14T12:16:32.891088+0000","last_peered":"2024-03-14T12:16:32.891088+0000","last_clean":"2024-03-14T12:16:32.891088+0000","last_became_active":"2024-03-14T12:16:32.889904+0000","last_became_peered":"2024-03-14T12:16:32.889904+0000","last_unstale":"2024-03-14T12:16:32.891088+0000","last_undegraded":"2024-03-14T12:16:32.891088+0000","last_fullsized":"2024-03-14T12:16:32.891088+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:38:19.269651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888632+0000","last_change":"2024-03-14T12:16:32.885130+0000","last_active":"2024-03-14T12:16:32.888632+0000","last_peered":"2024-03-14T12:16:32.888632+0000","last_clean":"2024-03-14T12:16:32.888632+0000","last_became_active":"2024-03-14T12:16:32.884845+0000","last_became_peered":"2024-03-14T12:16:32.884845+0000","last_unstale":"2024-03-14T12:16:32.888632+0000","last_undegraded":"2024-03-14T12:16:32.888632+0000","last_fullsized":"2024-03-14T12:16:32.888632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:22.674104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889929+0000","last_change":"2024-03-14T12:16:32.886892+0000","last_active":"2024-03-14T12:16:32.889929+0000","last_peered":"2024-03-14T12:16:32.889929+0000","last_clean":"2024-03-14T12:16:32.889929+0000","last_became_active":"2024-03-14T12:16:32.886637+0000","last_became_peered":"2024-03-14T12:16:32.886637+0000","last_unstale":"2024-03-14T12:16:32.889929+0000","last_undegraded":"2024-03-14T12:16:32.889929+0000","last_fullsized":"2024-03-14T12:16:32.889929+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:18:55.168791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900968+0000","last_change":"2024-03-14T12:16:31.883324+0000","last_active":"2024-03-14T12:16:32.900968+0000","last_peered":"2024-03-14T12:16:32.900968+0000","last_clean":"2024-03-14T12:16:32.900968+0000","last_became_active":"2024-03-14T12:16:31.883049+0000","last_became_peered":"2024-03-14T12:16:31.883049+0000","last_unstale":"2024-03-14T12:16:32.900968+0000","last_undegraded":"2024-03-14T12:16:32.900968+0000","last_fullsized":"2024-03-14T12:16:32.900968+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:09.852408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889251+0000","last_change":"2024-03-14T12:16:32.886808+0000","last_active":"2024-03-14T12:16:32.889251+0000","last_peered":"2024-03-14T12:16:32.889251+0000","last_clean":"2024-03-14T12:16:32.889251+0000","last_became_active":"2024-03-14T12:16:32.886530+0000","last_became_peered":"2024-03-14T12:16:32.886530+0000","last_unstale":"2024-03-14T12:16:32.889251+0000","last_undegraded":"2024-03-14T12:16:32.889251+0000","last_fullsized":"2024-03-14T12:16:32.889251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:46:34.286986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888282+0000","last_change":"2024-03-14T12:16:31.887910+0000","last_active":"2024-03-14T12:16:32.888282+0000","last_peered":"2024-03-14T12:16:32.888282+0000","last_clean":"2024-03-14T12:16:32.888282+0000","last_became_active":"2024-03-14T12:16:31.887627+0000","last_became_peered":"2024-03-14T12:16:31.887627+0000","last_unstale":"2024-03-14T12:16:32.888282+0000","last_undegraded":"2024-03-14T12:16:32.888282+0000","last_fullsized":"2024-03-14T12:16:32.888282+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:17.462895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.887710+0000","last_change":"2024-03-14T12:16:32.883823+0000","last_active":"2024-03-14T12:16:32.887710+0000","last_peered":"2024-03-14T12:16:32.887710+0000","last_clean":"2024-03-14T12:16:32.887710+0000","last_became_active":"2024-03-14T12:16:32.883538+0000","last_became_peered":"2024-03-14T12:16:32.883538+0000","last_unstale":"2024-03-14T12:16:32.887710+0000","last_undegraded":"2024-03-14T12:16:32.887710+0000","last_fullsized":"2024-03-14T12:16:32.887710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:19.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.891962+0000","last_change":"2024-03-14T12:16:31.889056+0000","last_active":"2024-03-14T12:16:32.891962+0000","last_peered":"2024-03-14T12:16:32.891962+0000","last_clean":"2024-03-14T12:16:32.891962+0000","last_became_active":"2024-03-14T12:16:31.888770+0000","last_became_peered":"2024-03-14T12:16:31.888770+0000","last_unstale":"2024-03-14T12:16:32.891962+0000","last_undegraded":"2024-03-14T12:16:32.891962+0000","last_fullsized":"2024-03-14T12:16:32.891962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:25:11.753733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889957+0000","last_change":"2024-03-14T12:16:32.886763+0000","last_active":"2024-03-14T12:16:32.889957+0000","last_peered":"2024-03-14T12:16:32.889957+0000","last_clean":"2024-03-14T12:16:32.889957+0000","last_became_active":"2024-03-14T12:16:32.886260+0000","last_became_peered":"2024-03-14T12:16:32.886260+0000","last_unstale":"2024-03-14T12:16:32.889957+0000","last_undegraded":"2024-03-14T12:16:32.889957+0000","last_fullsized":"2024-03-14T12:16:32.889957+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:12.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900492+0000","last_change":"2024-03-14T12:16:31.881489+0000","last_active":"2024-03-14T12:16:32.900492+0000","last_peered":"2024-03-14T12:16:32.900492+0000","last_clean":"2024-03-14T12:16:32.900492+0000","last_became_active":"2024-03-14T12:16:31.881197+0000","last_became_peered":"2024-03-14T12:16:31.881197+0000","last_unstale":"2024-03-14T12:16:32.900492+0000","last_undegraded":"2024-03-14T12:16:32.900492+0000","last_fullsized":"2024-03-14T12:16:32.900492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:09.852408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898228+0000","last_change":"2024-03-14T12:16:32.894357+0000","last_active":"2024-03-14T12:16:32.898228+0000","last_peered":"2024-03-14T12:16:32.898228+0000","last_clean":"2024-03-14T12:16:32.898228+0000","last_became_active":"2024-03-14T12:16:32.893751+0000","last_became_peered":"2024-03-14T12:16:32.893751+0000","last_unstale":"2024-03-14T12:16:32.898228+0000","last_undegraded":"2024-03-14T12:16:32.898228+0000","last_fullsized":"2024-03-14T12:16:32.898228+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:51.995841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897167+0000","last_change":"2024-03-14T12:16:31.885662+0000","last_active":"2024-03-14T12:16:32.897167+0000","last_peered":"2024-03-14T12:16:32.897167+0000","last_clean":"2024-03-14T12:16:32.897167+0000","last_became_active":"2024-03-14T12:16:31.885355+0000","last_became_peered":"2024-03-14T12:16:31.885355+0000","last_unstale":"2024-03-14T12:16:32.897167+0000","last_undegraded":"2024-03-14T12:16:32.897167+0000","last_fullsized":"2024-03-14T12:16:32.897167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:39:35.243213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898220+0000","last_change":"2024-03-14T12:16:32.893151+0000","last_active":"2024-03-14T12:16:32.898220+0000","last_peered":"2024-03-14T12:16:32.898220+0000","last_clean":"2024-03-14T12:16:32.898220+0000","last_became_active":"2024-03-14T12:16:32.892891+0000","last_became_peered":"2024-03-14T12:16:32.892891+0000","last_unstale":"2024-03-14T12:16:32.898220+0000","last_undegraded":"2024-03-14T12:16:32.898220+0000","last_fullsized":"2024-03-14T12:16:32.898220+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:57.663772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.899888+0000","last_change":"2024-03-14T12:16:31.882606+0000","last_active":"2024-03-14T12:16:32.899888+0000","last_peered":"2024-03-14T12:16:32.899888+0000","last_clean":"2024-03-14T12:16:32.899888+0000","last_became_active":"2024-03-14T12:16:31.882280+0000","last_became_peered":"2024-03-14T12:16:31.882280+0000","last_unstale":"2024-03-14T12:16:32.899888+0000","last_undegraded":"2024-03-14T12:16:32.899888+0000","last_fullsized":"2024-03-14T12:16:32.899888+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:48:06.040703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.885169+0000","last_change":"2024-03-14T12:16:32.882844+0000","last_active":"2024-03-14T12:16:32.885169+0000","last_peered":"2024-03-14T12:16:32.885169+0000","last_clean":"2024-03-14T12:16:32.885169+0000","last_became_active":"2024-03-14T12:16:32.882466+0000","last_became_peered":"2024-03-14T12:16:32.882466+0000","last_unstale":"2024-03-14T12:16:32.885169+0000","last_undegraded":"2024-03-14T12:16:32.885169+0000","last_fullsized":"2024-03-14T12:16:32.885169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:52:45.064420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.872561+0000","last_change":"2024-03-14T12:16:31.886002+0000","last_active":"2024-03-14T12:16:32.872561+0000","last_peered":"2024-03-14T12:16:32.872561+0000","last_clean":"2024-03-14T12:16:32.872561+0000","last_became_active":"2024-03-14T12:16:31.885710+0000","last_became_peered":"2024-03-14T12:16:31.885710+0000","last_unstale":"2024-03-14T12:16:32.872561+0000","last_undegraded":"2024-03-14T12:16:32.872561+0000","last_fullsized":"2024-03-14T12:16:32.872561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:03:24.346029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896729+0000","last_change":"2024-03-14T12:16:32.885267+0000","last_active":"2024-03-14T12:16:32.896729+0000","last_peered":"2024-03-14T12:16:32.896729+0000","last_clean":"2024-03-14T12:16:32.896729+0000","last_became_active":"2024-03-14T12:16:32.884979+0000","last_became_peered":"2024-03-14T12:16:32.884979+0000","last_unstale":"2024-03-14T12:16:32.896729+0000","last_undegraded":"2024-03-14T12:16:32.896729+0000","last_fullsized":"2024-03-14T12:16:32.896729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:33:19.774417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.874359+0000","last_change":"2024-03-14T12:16:31.886284+0000","last_active":"2024-03-14T12:16:32.874359+0000","last_peered":"2024-03-14T12:16:32.874359+0000","last_clean":"2024-03-14T12:16:32.874359+0000","last_became_active":"2024-03-14T12:16:31.885976+0000","last_became_peered":"2024-03-14T12:16:31.885976+0000","last_unstale":"2024-03-14T12:16:32.874359+0000","last_undegraded":"2024-03-14T12:16:32.874359+0000","last_fullsized":"2024-03-14T12:16:32.874359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:01.343460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.886533+0000","last_change":"2024-03-14T12:16:32.879719+0000","last_active":"2024-03-14T12:16:32.886533+0000","last_peered":"2024-03-14T12:16:32.886533+0000","last_clean":"2024-03-14T12:16:32.886533+0000","last_became_active":"2024-03-14T12:16:32.879397+0000","last_became_peered":"2024-03-14T12:16:32.879397+0000","last_unstale":"2024-03-14T12:16:32.886533+0000","last_undegraded":"2024-03-14T12:16:32.886533+0000","last_fullsized":"2024-03-14T12:16:32.886533+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:26:14.651402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.880199+0000","last_change":"2024-03-14T12:16:31.884864+0000","last_active":"2024-03-14T12:16:32.880199+0000","last_peered":"2024-03-14T12:16:32.880199+0000","last_clean":"2024-03-14T12:16:32.880199+0000","last_became_active":"2024-03-14T12:16:31.884470+0000","last_became_peered":"2024-03-14T12:16:31.884470+0000","last_unstale":"2024-03-14T12:16:32.880199+0000","last_undegraded":"2024-03-14T12:16:32.880199+0000","last_fullsized":"2024-03-14T12:16:32.880199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:43:16.206207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900137+0000","last_change":"2024-03-14T12:16:32.893064+0000","last_active":"2024-03-14T12:16:32.900137+0000","last_peered":"2024-03-14T12:16:32.900137+0000","last_clean":"2024-03-14T12:16:32.900137+0000","last_became_active":"2024-03-14T12:16:32.892541+0000","last_became_peered":"2024-03-14T12:16:32.892541+0000","last_unstale":"2024-03-14T12:16:32.900137+0000","last_undegraded":"2024-03-14T12:16:32.900137+0000","last_fullsized":"2024-03-14T12:16:32.900137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:20.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.875312+0000","last_change":"2024-03-14T12:16:31.889669+0000","last_active":"2024-03-14T12:16:32.875312+0000","last_peered":"2024-03-14T12:16:32.875312+0000","last_clean":"2024-03-14T12:16:32.875312+0000","last_became_active":"2024-03-14T12:16:31.889373+0000","last_became_peered":"2024-03-14T12:16:31.889373+0000","last_unstale":"2024-03-14T12:16:32.875312+0000","last_undegraded":"2024-03-14T12:16:32.875312+0000","last_fullsized":"2024-03-14T12:16:32.875312+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:09.596105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.887691+0000","last_change":"2024-03-14T12:16:32.880312+0000","last_active":"2024-03-14T12:16:32.887691+0000","last_peered":"2024-03-14T12:16:32.887691+0000","last_clean":"2024-03-14T12:16:32.887691+0000","last_became_active":"2024-03-14T12:16:32.879821+0000","last_became_peered":"2024-03-14T12:16:32.879821+0000","last_unstale":"2024-03-14T12:16:32.887691+0000","last_undegraded":"2024-03-14T12:16:32.887691+0000","last_fullsized":"2024-03-14T12:16:32.887691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:01.676794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900739+0000","last_change":"2024-03-14T12:16:31.886478+0000","last_active":"2024-03-14T12:16:32.900739+0000","last_peered":"2024-03-14T12:16:32.900739+0000","last_clean":"2024-03-14T12:16:32.900739+0000","last_became_active":"2024-03-14T12:16:31.885986+0000","last_became_peered":"2024-03-14T12:16:31.885986+0000","last_unstale":"2024-03-14T12:16:32.900739+0000","last_undegraded":"2024-03-14T12:16:32.900739+0000","last_fullsized":"2024-03-14T12:16:32.900739+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:03:24.346029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900234+0000","last_change":"2024-03-14T12:16:32.893176+0000","last_active":"2024-03-14T12:16:32.900234+0000","last_peered":"2024-03-14T12:16:32.900234+0000","last_clean":"2024-03-14T12:16:32.900234+0000","last_became_active":"2024-03-14T12:16:32.892615+0000","last_became_peered":"2024-03-14T12:16:32.892615+0000","last_unstale":"2024-03-14T12:16:32.900234+0000","last_undegraded":"2024-03-14T12:16:32.900234+0000","last_fullsized":"2024-03-14T12:16:32.900234+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:18:14.483997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900574+0000","last_change":"2024-03-14T12:16:31.875858+0000","last_active":"2024-03-14T12:16:32.900574+0000","last_peered":"2024-03-14T12:16:32.900574+0000","last_clean":"2024-03-14T12:16:32.900574+0000","last_became_active":"2024-03-14T12:16:31.875552+0000","last_became_peered":"2024-03-14T12:16:31.875552+0000","last_unstale":"2024-03-14T12:16:32.900574+0000","last_undegraded":"2024-03-14T12:16:32.900574+0000","last_fullsized":"2024-03-14T12:16:32.900574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:51:14.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.884081+0000","last_change":"2024-03-14T12:16:32.880604+0000","last_active":"2024-03-14T12:16:32.884081+0000","last_peered":"2024-03-14T12:16:32.884081+0000","last_clean":"2024-03-14T12:16:32.884081+0000","last_became_active":"2024-03-14T12:16:32.880301+0000","last_became_peered":"2024-03-14T12:16:32.880301+0000","last_unstale":"2024-03-14T12:16:32.884081+0000","last_undegraded":"2024-03-14T12:16:32.884081+0000","last_fullsized":"2024-03-14T12:16:32.884081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:18:14.483997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894596+0000","last_change":"2024-03-14T12:16:31.886919+0000","last_active":"2024-03-14T12:16:32.894596+0000","last_peered":"2024-03-14T12:16:32.894596+0000","last_clean":"2024-03-14T12:16:32.894596+0000","last_became_active":"2024-03-14T12:16:31.886603+0000","last_became_peered":"2024-03-14T12:16:31.886603+0000","last_unstale":"2024-03-14T12:16:32.894596+0000","last_undegraded":"2024-03-14T12:16:32.894596+0000","last_fullsized":"2024-03-14T12:16:32.894596+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:19.867306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888144+0000","last_change":"2024-03-14T12:16:32.886021+0000","last_active":"2024-03-14T12:16:32.888144+0000","last_peered":"2024-03-14T12:16:32.888144+0000","last_clean":"2024-03-14T12:16:32.888144+0000","last_became_active":"2024-03-14T12:16:32.885661+0000","last_became_peered":"2024-03-14T12:16:32.885661+0000","last_unstale":"2024-03-14T12:16:32.888144+0000","last_undegraded":"2024-03-14T12:16:32.888144+0000","last_fullsized":"2024-03-14T12:16:32.888144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:10.878467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876775+0000","last_change":"2024-03-14T12:16:31.885073+0000","last_active":"2024-03-14T12:16:32.876775+0000","last_peered":"2024-03-14T12:16:32.876775+0000","last_clean":"2024-03-14T12:16:32.876775+0000","last_became_active":"2024-03-14T12:16:31.884785+0000","last_became_peered":"2024-03-14T12:16:31.884785+0000","last_unstale":"2024-03-14T12:16:32.876775+0000","last_undegraded":"2024-03-14T12:16:32.876775+0000","last_fullsized":"2024-03-14T12:16:32.876775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:16.142831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.894427+0000","last_change":"2024-03-14T12:16:32.882923+0000","last_active":"2024-03-14T12:16:32.894427+0000","last_peered":"2024-03-14T12:16:32.894427+0000","last_clean":"2024-03-14T12:16:32.894427+0000","last_became_active":"2024-03-14T12:16:32.882634+0000","last_became_peered":"2024-03-14T12:16:32.882634+0000","last_unstale":"2024-03-14T12:16:32.894427+0000","last_undegraded":"2024-03-14T12:16:32.894427+0000","last_fullsized":"2024-03-14T12:16:32.894427+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:43:17.211532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.872716+0000","last_change":"2024-03-14T12:16:31.881166+0000","last_active":"2024-03-14T12:16:32.872716+0000","last_peered":"2024-03-14T12:16:32.872716+0000","last_clean":"2024-03-14T12:16:32.872716+0000","last_became_active":"2024-03-14T12:16:31.880870+0000","last_became_peered":"2024-03-14T12:16:31.880870+0000","last_unstale":"2024-03-14T12:16:32.872716+0000","last_undegraded":"2024-03-14T12:16:32.872716+0000","last_fullsized":"2024-03-14T12:16:32.872716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:09.873142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900885+0000","last_change":"2024-03-14T12:16:32.897760+0000","last_active":"2024-03-14T12:16:32.900885+0000","last_peered":"2024-03-14T12:16:32.900885+0000","last_clean":"2024-03-14T12:16:32.900885+0000","last_became_active":"2024-03-14T12:16:32.897302+0000","last_became_peered":"2024-03-14T12:16:32.897302+0000","last_unstale":"2024-03-14T12:16:32.900885+0000","last_undegraded":"2024-03-14T12:16:32.900885+0000","last_fullsized":"2024-03-14T12:16:32.900885+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:24.479669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.901287+0000","last_change":"2024-03-14T12:16:31.885316+0000","last_active":"2024-03-14T12:16:32.901287+0000","last_peered":"2024-03-14T12:16:32.901287+0000","last_clean":"2024-03-14T12:16:32.901287+0000","last_became_active":"2024-03-14T12:16:31.884997+0000","last_became_peered":"2024-03-14T12:16:31.884997+0000","last_unstale":"2024-03-14T12:16:32.901287+0000","last_undegraded":"2024-03-14T12:16:32.901287+0000","last_fullsized":"2024-03-14T12:16:32.901287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:26:13.646077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900759+0000","last_change":"2024-03-14T12:16:32.895200+0000","last_active":"2024-03-14T12:16:32.900759+0000","last_peered":"2024-03-14T12:16:32.900759+0000","last_clean":"2024-03-14T12:16:32.900759+0000","last_became_active":"2024-03-14T12:16:32.894903+0000","last_became_peered":"2024-03-14T12:16:32.894903+0000","last_unstale":"2024-03-14T12:16:32.900759+0000","last_undegraded":"2024-03-14T12:16:32.900759+0000","last_fullsized":"2024-03-14T12:16:32.900759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:56:56.014598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.893207+0000","last_change":"2024-03-14T12:16:31.886947+0000","last_active":"2024-03-14T12:16:32.893207+0000","last_peered":"2024-03-14T12:16:32.893207+0000","last_clean":"2024-03-14T12:16:32.893207+0000","last_became_active":"2024-03-14T12:16:31.886429+0000","last_became_peered":"2024-03-14T12:16:31.886429+0000","last_unstale":"2024-03-14T12:16:32.893207+0000","last_undegraded":"2024-03-14T12:16:32.893207+0000","last_fullsized":"2024-03-14T12:16:32.893207+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:19.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.882276+0000","last_change":"2024-03-14T12:16:32.880478+0000","last_active":"2024-03-14T12:16:32.882276+0000","last_peered":"2024-03-14T12:16:32.882276+0000","last_clean":"2024-03-14T12:16:32.882276+0000","last_became_active":"2024-03-14T12:16:32.879920+0000","last_became_peered":"2024-03-14T12:16:32.879920+0000","last_unstale":"2024-03-14T12:16:32.882276+0000","last_undegraded":"2024-03-14T12:16:32.882276+0000","last_fullsized":"2024-03-14T12:16:32.882276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:20.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.895437+0000","last_change":"2024-03-14T12:16:31.885893+0000","last_active":"2024-03-14T12:16:32.895437+0000","last_peered":"2024-03-14T12:16:32.895437+0000","last_clean":"2024-03-14T12:16:32.895437+0000","last_became_active":"2024-03-14T12:16:31.885641+0000","last_became_peered":"2024-03-14T12:16:31.885641+0000","last_unstale":"2024-03-14T12:16:32.895437+0000","last_undegraded":"2024-03-14T12:16:32.895437+0000","last_fullsized":"2024-03-14T12:16:32.895437+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:18:13.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897922+0000","last_change":"2024-03-14T12:16:32.892330+0000","last_active":"2024-03-14T12:16:32.897922+0000","last_peered":"2024-03-14T12:16:32.897922+0000","last_clean":"2024-03-14T12:16:32.897922+0000","last_became_active":"2024-03-14T12:16:32.891083+0000","last_became_peered":"2024-03-14T12:16:32.891083+0000","last_unstale":"2024-03-14T12:16:32.897922+0000","last_undegraded":"2024-03-14T12:16:32.897922+0000","last_fullsized":"2024-03-14T12:16:32.897922+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:39:57.733790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900279+0000","last_change":"2024-03-14T12:16:31.883235+0000","last_active":"2024-03-14T12:16:32.900279+0000","last_peered":"2024-03-14T12:16:32.900279+0000","last_clean":"2024-03-14T12:16:32.900279+0000","last_became_active":"2024-03-14T12:16:31.882956+0000","last_became_peered":"2024-03-14T12:16:31.882956+0000","last_unstale":"2024-03-14T12:16:32.900279+0000","last_undegraded":"2024-03-14T12:16:32.900279+0000","last_fullsized":"2024-03-14T12:16:32.900279+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:26:13.646077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900214+0000","last_change":"2024-03-14T12:16:32.893013+0000","last_active":"2024-03-14T12:16:32.900214+0000","last_peered":"2024-03-14T12:16:32.900214+0000","last_clean":"2024-03-14T12:16:32.900214+0000","last_became_active":"2024-03-14T12:16:32.892679+0000","last_became_peered":"2024-03-14T12:16:32.892679+0000","last_unstale":"2024-03-14T12:16:32.900214+0000","last_undegraded":"2024-03-14T12:16:32.900214+0000","last_fullsized":"2024-03-14T12:16:32.900214+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:48:07.046028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.877416+0000","last_change":"2024-03-14T12:16:31.881293+0000","last_active":"2024-03-14T12:16:32.877416+0000","last_peered":"2024-03-14T12:16:32.877416+0000","last_clean":"2024-03-14T12:16:32.877416+0000","last_became_active":"2024-03-14T12:16:31.881003+0000","last_became_peered":"2024-03-14T12:16:31.881003+0000","last_unstale":"2024-03-14T12:16:32.877416+0000","last_undegraded":"2024-03-14T12:16:32.877416+0000","last_fullsized":"2024-03-14T12:16:32.877416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:53:19.251545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.886391+0000","last_change":"2024-03-14T12:16:32.880935+0000","last_active":"2024-03-14T12:16:32.886391+0000","last_peered":"2024-03-14T12:16:32.886391+0000","last_clean":"2024-03-14T12:16:32.886391+0000","last_became_active":"2024-03-14T12:16:32.880502+0000","last_became_peered":"2024-03-14T12:16:32.880502+0000","last_unstale":"2024-03-14T12:16:32.886391+0000","last_undegraded":"2024-03-14T12:16:32.886391+0000","last_fullsized":"2024-03-14T12:16:32.886391+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:56:56.014598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898142+0000","last_change":"2024-03-14T12:16:31.890069+0000","last_active":"2024-03-14T12:16:32.898142+0000","last_peered":"2024-03-14T12:16:32.898142+0000","last_clean":"2024-03-14T12:16:32.898142+0000","last_became_active":"2024-03-14T12:16:31.889746+0000","last_became_peered":"2024-03-14T12:16:31.889746+0000","last_unstale":"2024-03-14T12:16:32.898142+0000","last_undegraded":"2024-03-14T12:16:32.898142+0000","last_fullsized":"2024-03-14T12:16:32.898142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:12.021144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.888290+0000","last_change":"2024-03-14T12:16:32.884896+0000","last_active":"2024-03-14T12:16:32.888290+0000","last_peered":"2024-03-14T12:16:32.888290+0000","last_clean":"2024-03-14T12:16:32.888290+0000","last_became_active":"2024-03-14T12:16:32.884606+0000","last_became_peered":"2024-03-14T12:16:32.884606+0000","last_unstale":"2024-03-14T12:16:32.888290+0000","last_undegraded":"2024-03-14T12:16:32.888290+0000","last_fullsized":"2024-03-14T12:16:32.888290+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:08.569373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.871643+0000","last_change":"2024-03-14T12:16:31.887671+0000","last_active":"2024-03-14T12:16:32.871643+0000","last_peered":"2024-03-14T12:16:32.871643+0000","last_clean":"2024-03-14T12:16:32.871643+0000","last_became_active":"2024-03-14T12:16:31.887221+0000","last_became_peered":"2024-03-14T12:16:31.887221+0000","last_unstale":"2024-03-14T12:16:32.871643+0000","last_undegraded":"2024-03-14T12:16:32.871643+0000","last_fullsized":"2024-03-14T12:16:32.871643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:15:29.488761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.890793+0000","last_change":"2024-03-14T12:16:32.888030+0000","last_active":"2024-03-14T12:16:32.890793+0000","last_peered":"2024-03-14T12:16:32.890793+0000","last_clean":"2024-03-14T12:16:32.890793+0000","last_became_active":"2024-03-14T12:16:32.887755+0000","last_became_peered":"2024-03-14T12:16:32.887755+0000","last_unstale":"2024-03-14T12:16:32.890793+0000","last_undegraded":"2024-03-14T12:16:32.890793+0000","last_fullsized":"2024-03-14T12:16:32.890793+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:57.398996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.872012+0000","last_change":"2024-03-14T12:16:31.882034+0000","last_active":"2024-03-14T12:16:32.872012+0000","last_peered":"2024-03-14T12:16:32.872012+0000","last_clean":"2024-03-14T12:16:32.872012+0000","last_became_active":"2024-03-14T12:16:31.881742+0000","last_became_peered":"2024-03-14T12:16:31.881742+0000","last_unstale":"2024-03-14T12:16:32.872012+0000","last_undegraded":"2024-03-14T12:16:32.872012+0000","last_fullsized":"2024-03-14T12:16:32.872012+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:43:16.206207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900273+0000","last_change":"2024-03-14T12:16:32.891551+0000","last_active":"2024-03-14T12:16:32.900273+0000","last_peered":"2024-03-14T12:16:32.900273+0000","last_clean":"2024-03-14T12:16:32.900273+0000","last_became_active":"2024-03-14T12:16:32.891208+0000","last_became_peered":"2024-03-14T12:16:32.891208+0000","last_unstale":"2024-03-14T12:16:32.900273+0000","last_undegraded":"2024-03-14T12:16:32.900273+0000","last_fullsized":"2024-03-14T12:16:32.900273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:24.479669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897359+0000","last_change":"2024-03-14T12:16:31.881543+0000","last_active":"2024-03-14T12:16:32.897359+0000","last_peered":"2024-03-14T12:16:32.897359+0000","last_clean":"2024-03-14T12:16:32.897359+0000","last_became_active":"2024-03-14T12:16:31.881296+0000","last_became_peered":"2024-03-14T12:16:31.881296+0000","last_unstale":"2024-03-14T12:16:32.897359+0000","last_undegraded":"2024-03-14T12:16:32.897359+0000","last_fullsized":"2024-03-14T12:16:32.897359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:16.142831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900169+0000","last_change":"2024-03-14T12:16:32.889400+0000","last_active":"2024-03-14T12:16:32.900169+0000","last_peered":"2024-03-14T12:16:32.900169+0000","last_clean":"2024-03-14T12:16:32.900169+0000","last_became_active":"2024-03-14T12:16:32.888938+0000","last_became_peered":"2024-03-14T12:16:32.888938+0000","last_unstale":"2024-03-14T12:16:32.900169+0000","last_undegraded":"2024-03-14T12:16:32.900169+0000","last_fullsized":"2024-03-14T12:16:32.900169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:08.569373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.877666+0000","last_change":"2024-03-14T12:16:31.881853+0000","last_active":"2024-03-14T12:16:32.877666+0000","last_peered":"2024-03-14T12:16:32.877666+0000","last_clean":"2024-03-14T12:16:32.877666+0000","last_became_active":"2024-03-14T12:16:31.881521+0000","last_became_peered":"2024-03-14T12:16:31.881521+0000","last_unstale":"2024-03-14T12:16:32.877666+0000","last_undegraded":"2024-03-14T12:16:32.877666+0000","last_fullsized":"2024-03-14T12:16:32.877666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:56:55.009273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889289+0000","last_change":"2024-03-14T12:16:32.882894+0000","last_active":"2024-03-14T12:16:32.889289+0000","last_peered":"2024-03-14T12:16:32.889289+0000","last_clean":"2024-03-14T12:16:32.889289+0000","last_became_active":"2024-03-14T12:16:32.882257+0000","last_became_peered":"2024-03-14T12:16:32.882257+0000","last_unstale":"2024-03-14T12:16:32.889289+0000","last_undegraded":"2024-03-14T12:16:32.889289+0000","last_fullsized":"2024-03-14T12:16:32.889289+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:52:45.064420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876889+0000","last_change":"2024-03-14T12:16:31.882804+0000","last_active":"2024-03-14T12:16:32.876889+0000","last_peered":"2024-03-14T12:16:32.876889+0000","last_clean":"2024-03-14T12:16:32.876889+0000","last_became_active":"2024-03-14T12:16:31.882490+0000","last_became_peered":"2024-03-14T12:16:31.882490+0000","last_unstale":"2024-03-14T12:16:32.876889+0000","last_undegraded":"2024-03-14T12:16:32.876889+0000","last_fullsized":"2024-03-14T12:16:32.876889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:00.671469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897555+0000","last_change":"2024-03-14T12:16:32.895726+0000","last_active":"2024-03-14T12:16:32.897555+0000","last_peered":"2024-03-14T12:16:32.897555+0000","last_clean":"2024-03-14T12:16:32.897555+0000","last_became_active":"2024-03-14T12:16:32.895425+0000","last_became_peered":"2024-03-14T12:16:32.895425+0000","last_unstale":"2024-03-14T12:16:32.897555+0000","last_undegraded":"2024-03-14T12:16:32.897555+0000","last_fullsized":"2024-03-14T12:16:32.897555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:18.468220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.873462+0000","last_change":"2024-03-14T12:16:31.885730+0000","last_active":"2024-03-14T12:16:32.873462+0000","last_peered":"2024-03-14T12:16:32.873462+0000","last_clean":"2024-03-14T12:16:32.873462+0000","last_became_active":"2024-03-14T12:16:31.885455+0000","last_became_peered":"2024-03-14T12:16:31.885455+0000","last_unstale":"2024-03-14T12:16:32.873462+0000","last_undegraded":"2024-03-14T12:16:32.873462+0000","last_fullsized":"2024-03-14T12:16:32.873462+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:25:11.753733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.897256+0000","last_change":"2024-03-14T12:16:32.891495+0000","last_active":"2024-03-14T12:16:32.897256+0000","last_peered":"2024-03-14T12:16:32.897256+0000","last_clean":"2024-03-14T12:16:32.897256+0000","last_became_active":"2024-03-14T12:16:32.891174+0000","last_became_peered":"2024-03-14T12:16:32.891174+0000","last_unstale":"2024-03-14T12:16:32.897256+0000","last_undegraded":"2024-03-14T12:16:32.897256+0000","last_fullsized":"2024-03-14T12:16:32.897256+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:09:27.650527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.871377+0000","last_change":"2024-03-14T12:16:31.886036+0000","last_active":"2024-03-14T12:16:32.871377+0000","last_peered":"2024-03-14T12:16:32.871377+0000","last_clean":"2024-03-14T12:16:32.871377+0000","last_became_active":"2024-03-14T12:16:31.885743+0000","last_became_peered":"2024-03-14T12:16:31.885743+0000","last_unstale":"2024-03-14T12:16:32.871377+0000","last_undegraded":"2024-03-14T12:16:32.871377+0000","last_fullsized":"2024-03-14T12:16:32.871377+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:01:34.400093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.900852+0000","last_change":"2024-03-14T12:16:32.898040+0000","last_active":"2024-03-14T12:16:32.900852+0000","last_peered":"2024-03-14T12:16:32.900852+0000","last_clean":"2024-03-14T12:16:32.900852+0000","last_became_active":"2024-03-14T12:16:32.897767+0000","last_became_peered":"2024-03-14T12:16:32.897767+0000","last_unstale":"2024-03-14T12:16:32.900852+0000","last_undegraded":"2024-03-14T12:16:32.900852+0000","last_fullsized":"2024-03-14T12:16:32.900852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:12.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.873122+0000","last_change":"2024-03-14T12:16:31.876723+0000","last_active":"2024-03-14T12:16:32.873122+0000","last_peered":"2024-03-14T12:16:32.873122+0000","last_clean":"2024-03-14T12:16:32.873122+0000","last_became_active":"2024-03-14T12:16:31.876437+0000","last_became_peered":"2024-03-14T12:16:31.876437+0000","last_unstale":"2024-03-14T12:16:32.873122+0000","last_undegraded":"2024-03-14T12:16:32.873122+0000","last_fullsized":"2024-03-14T12:16:32.873122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:14:30.089638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.889257+0000","last_change":"2024-03-14T12:16:32.886000+0000","last_active":"2024-03-14T12:16:32.889257+0000","last_peered":"2024-03-14T12:16:32.889257+0000","last_clean":"2024-03-14T12:16:32.889257+0000","last_became_active":"2024-03-14T12:16:32.885526+0000","last_became_peered":"2024-03-14T12:16:32.885526+0000","last_unstale":"2024-03-14T12:16:32.889257+0000","last_undegraded":"2024-03-14T12:16:32.889257+0000","last_fullsized":"2024-03-14T12:16:32.889257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:42:34.697540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.871506+0000","last_change":"2024-03-14T12:16:31.880908+0000","last_active":"2024-03-14T12:16:32.871506+0000","last_peered":"2024-03-14T12:16:32.871506+0000","last_clean":"2024-03-14T12:16:32.871506+0000","last_became_active":"2024-03-14T12:16:31.880643+0000","last_became_peered":"2024-03-14T12:16:31.880643+0000","last_unstale":"2024-03-14T12:16:32.871506+0000","last_undegraded":"2024-03-14T12:16:32.871506+0000","last_fullsized":"2024-03-14T12:16:32.871506+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:36:03.351328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.896864+0000","last_change":"2024-03-14T12:16:32.887125+0000","last_active":"2024-03-14T12:16:32.896864+0000","last_peered":"2024-03-14T12:16:32.896864+0000","last_clean":"2024-03-14T12:16:32.896864+0000","last_became_active":"2024-03-14T12:16:32.885810+0000","last_became_peered":"2024-03-14T12:16:32.885810+0000","last_unstale":"2024-03-14T12:16:32.896864+0000","last_undegraded":"2024-03-14T12:16:32.896864+0000","last_fullsized":"2024-03-14T12:16:32.896864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:30:14.944354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876308+0000","last_change":"2024-03-14T12:16:31.885799+0000","last_active":"2024-03-14T12:16:32.876308+0000","last_peered":"2024-03-14T12:16:32.876308+0000","last_clean":"2024-03-14T12:16:32.876308+0000","last_became_active":"2024-03-14T12:16:31.885115+0000","last_became_peered":"2024-03-14T12:16:31.885115+0000","last_unstale":"2024-03-14T12:16:32.876308+0000","last_undegraded":"2024-03-14T12:16:32.876308+0000","last_fullsized":"2024-03-14T12:16:32.876308+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:20.298469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.898281+0000","last_change":"2024-03-14T12:16:32.894164+0000","last_active":"2024-03-14T12:16:32.898281+0000","last_peered":"2024-03-14T12:16:32.898281+0000","last_clean":"2024-03-14T12:16:32.898281+0000","last_became_active":"2024-03-14T12:16:32.893647+0000","last_became_peered":"2024-03-14T12:16:32.893647+0000","last_unstale":"2024-03-14T12:16:32.898281+0000","last_undegraded":"2024-03-14T12:16:32.898281+0000","last_fullsized":"2024-03-14T12:16:32.898281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:31.861966+0000","last_clean_scrub_stamp":"2024-03-14T12:16:31.861966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:02:36.750291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.876971+0000","last_change":"2024-03-14T12:16:31.882679+0000","last_active":"2024-03-14T12:16:32.876971+0000","last_peered":"2024-03-14T12:16:32.876971+0000","last_clean":"2024-03-14T12:16:32.876971+0000","last_became_active":"2024-03-14T12:16:31.882405+0000","last_became_peered":"2024-03-14T12:16:31.882405+0000","last_unstale":"2024-03-14T12:16:32.876971+0000","last_undegraded":"2024-03-14T12:16:32.876971+0000","last_fullsized":"2024-03-14T12:16:32.876971+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:30.856641+0000","last_clean_scrub_stamp":"2024-03-14T12:16:30.856641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:58:26.213092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":236,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-14T12:16:32.879234+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:16:32.879234+0000","last_peered":"2024-03-14T12:16:32.879234+0000","last_clean":"2024-03-14T12:16:32.879234+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:16:32.879234+0000","last_undegraded":"2024-03-14T12:16:32.879234+0000","last_fullsized":"2024-03-14T12:16:32.879234+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":871952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":281424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":282076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":283036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36024,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335816,"statfs":{"total":96636764160,"available":96599875584,"internally_reserved":0,"allocated":692224,"data_stored":281480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":281480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36000,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335840,"statfs":{"total":96636764160,"available":96599900160,"internally_reserved":0,"allocated":671744,"data_stored":274649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":873986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:16:36.175 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T12:16:36.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-14T12:16:36.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.318+0000 7f7a105c6640 1 Processor -- start 2024-03-14T12:16:36.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.318+0000 7f7a105c6640 1 -- start start 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a105c6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7a08145030 0x7f7a080cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a105c6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a080d04e0 con 0x7f7a080b29b0 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a105c6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a080b48a0 con 0x7f7a080cfbc0 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a105c6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a080ce2e0 con 0x7f7a08145030 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0eb63640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7a080b29b0 0x7f7a080ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41106/0 (socket says 172.21.15.82:41106) 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0eb63640 1 -- 172.21.15.82:0/3523639744 learned_addr learned my addr 172.21.15.82:0/3523639744 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206025042 0 0) 0x7f7a080d04e0 con 0x7f7a080b29b0 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0db61640 1 --2- 172.21.15.82:0/3523639744 >> v2:172.21.15.164:3300/0 conn(0x7f7a08145030 0x7f7a080cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f4003520 con 0x7f7a080b29b0 2024-03-14T12:16:36.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 168238454 0 0) 0x7f79f4003520 con 0x7f7a080b29b0 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 >> v2:172.21.15.164:3300/0 conn(0x7f7a08145030 msgr2=0x7f7a080cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 --2- 172.21.15.82:0/3523639744 >> v2:172.21.15.164:3300/0 conn(0x7f7a08145030 0x7f7a080cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 >> v1:172.21.15.164:6789/0 conn(0x7f7a080cfbc0 legacy=0x7f7a08160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080b4b80 con 0x7f7a080b29b0 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f7a000019f0 con 0x7f7a080b29b0 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.319+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 >> v1:172.21.15.82:6789/0 conn(0x7f7a080b29b0 legacy=0x7f7a080ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0d360640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f40039b0 0x7f79f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0d360640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f4004800 0x7f79f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0d360640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79f40051c0 0x7f79f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080ce2e0 con 0x7f79f4004800 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f400c060 con 0x7f79f40051c0 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f400c260 con 0x7f79f40039b0 2024-03-14T12:16:36.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0db61640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f4004800 0x7f79f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0eb63640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79f40051c0 0x7f79f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0e362640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f40039b0 0x7f79f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0db61640 1 -- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f40039b0 msgr2=0x7f79f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0db61640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f40039b0 0x7f79f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0db61640 1 -- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79f40051c0 msgr2=0x7f79f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0db61640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79f40051c0 0x7f79f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0db61640 1 -- 172.21.15.82:0/3523639744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4003520 con 0x7f79f4004800 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0eb63640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79f40051c0 0x7f79f400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.320+0000 7f7a0eb63640 1 -- 172.21.15.82:0/3523639744 reap_dead start 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a0db61640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f4004800 0x7f79f4004c80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009a90 tx=0x7f79f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc22456dab72ce17 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8039070 con 0x7f79f4004800 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f802c680 con 0x7f79f4004800 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a0d360640 1 -- 172.21.15.82:0/3523639744 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8009690 con 0x7f79f4004800 2024-03-14T12:16:36.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a105c6640 1 -- 172.21.15.82:0/3523639744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f4004800 msgr2=0x7f79f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a105c6640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f4004800 0x7f79f4004c80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009a90 tx=0x7f79f4005830 comp rx=0 tx=0).stop 2024-03-14T12:16:36.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a105c6640 1 -- 172.21.15.82:0/3523639744 shutdown_connections 2024-03-14T12:16:36.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a105c6640 1 --2- 172.21.15.82:0/3523639744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f4004800 0x7f79f4004c80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a105c6640 1 -- 172.21.15.82:0/3523639744 >> 172.21.15.82:0/3523639744 conn(0x7f7a0806aef0 msgr2=0x7f7a0806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:36.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a105c6640 1 -- 172.21.15.82:0/3523639744 shutdown_connections 2024-03-14T12:16:36.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.321+0000 7f7a105c6640 1 -- 172.21.15.82:0/3523639744 wait complete. 2024-03-14T12:16:36.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 Processor -- start 2024-03-14T12:16:36.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 -- start start 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f4004800 0x7f7a08140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a0813daf0 0x7f7a08151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a08142670 0x7f7a08142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08144c60 con 0x7f7a08142670 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080cdf70 con 0x7f7a0813daf0 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a105c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08144de0 con 0x7f79f4004800 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a0eb63640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a08142670 0x7f7a08142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a0eb63640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a08142670 0x7f7a08142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50552/0 (socket says 172.21.15.82:50552) 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.323+0000 7f7a0eb63640 1 -- 172.21.15.82:0/2237512846 learned_addr learned my addr 172.21.15.82:0/2237512846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0db61640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a0813daf0 0x7f7a08151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0e362640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f4004800 0x7f7a08140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0eb63640 1 -- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f4004800 msgr2=0x7f7a08140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0eb63640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f4004800 0x7f7a08140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0eb63640 1 -- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a0813daf0 msgr2=0x7f7a08151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0eb63640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a0813daf0 0x7f7a08151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0eb63640 1 -- 172.21.15.82:0/2237512846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a08141830 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0e362640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f4004800 0x7f7a08140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0db61640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a0813daf0 0x7f7a08151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a0eb63640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a08142670 0x7f7a08142af0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7a08145410 tx=0x7f7a00007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f79feffd640 1 -- 172.21.15.82:0/2237512846 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000ea20 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f79feffd640 1 -- 172.21.15.82:0/2237512846 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00018070 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a08141b10 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f79feffd640 1 -- 172.21.15.82:0/2237512846 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000ee00 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.324+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a081420d0 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.325+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0804f050 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.325+0000 7f79feffd640 1 -- 172.21.15.82:0/2237512846 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7a0001d460 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.326+0000 7f79feffd640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f79ec06bbd0 0x7f79ec06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.326+0000 7f79feffd640 1 -- 172.21.15.82:0/2237512846 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a0008ad20 con 0x7f7a08142670 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.326+0000 7f7a0e362640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f79ec06bbd0 0x7f79ec06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.326+0000 7f7a0e362640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f79ec06bbd0 0x7f79ec06e090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000c00 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:36.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.330+0000 7f79feffd640 1 -- 172.21.15.82:0/2237512846 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7a00059810 con 0x7f7a08142670 2024-03-14T12:16:36.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.568+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7a080cf6f0 con 0x7f7a08142670 2024-03-14T12:16:36.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.570+0000 7f79feffd640 1 -- 172.21.15.82:0/2237512846 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7a00059110 con 0x7f7a08142670 2024-03-14T12:16:36.570 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-03-14T12:16:36.570 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg== 2024-03-14T12:16:36.570 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-03-14T12:16:36.570 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-03-14T12:16:36.570 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-14T12:16:36.571 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f79ec06bbd0 msgr2=0x7f79ec06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f79ec06bbd0 0x7f79ec06e090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000c00 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a08142670 msgr2=0x7f7a08142af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a08142670 0x7f7a08142af0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7a08145410 tx=0x7f7a00007bc0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 shutdown_connections 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a08142670 0x7f7a08142af0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a0813daf0 0x7f7a08151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f79ec06bbd0 0x7f79ec06e090 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 --2- 172.21.15.82:0/2237512846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f79f4004800 0x7f7a08140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 >> 172.21.15.82:0/2237512846 conn(0x7f7a0806aef0 msgr2=0x7f7a0815b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 shutdown_connections 2024-03-14T12:16:36.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.572+0000 7f7a105c6640 1 -- 172.21.15.82:0/2237512846 wait complete. 2024-03-14T12:16:36.583 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:36.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.684+0000 7f3c1f4e9640 1 Processor -- start 2024-03-14T12:16:36.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.684+0000 7f3c1f4e9640 1 -- start start 2024-03-14T12:16:36.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.685+0000 7f3c1f4e9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3c180ceb90 0x7f3c180b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.685+0000 7f3c1f4e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180cdb50 0x7f3c180cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.685+0000 7f3c1f4e9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1806ed00 con 0x7f3c1815e190 2024-03-14T12:16:36.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.685+0000 7f3c1f4e9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c180d1180 con 0x7f3c180ceb90 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.685+0000 7f3c1f4e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180cc0f0 con 0x7f3c180cdb50 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.685+0000 7f3c1d285640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3c1815e190 0x7f3c180ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41114/0 (socket says 172.21.15.82:41114) 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.685+0000 7f3c1d285640 1 -- 172.21.15.82:0/1047653676 learned_addr learned my addr 172.21.15.82:0/1047653676 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089258525 0 0) 0x7f3c1806ed00 con 0x7f3c1815e190 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c04003520 con 0x7f3c1815e190 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1da86640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180cdb50 0x7f3c180cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1047653676 >> v2:172.21.15.164:3300/0 conn(0x7f3c180ceb90 0x7f3c180b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898556422 0 0) 0x7f3c04003520 con 0x7f3c1815e190 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180cdb50 msgr2=0x7f3c180cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180cdb50 0x7f3c180cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 >> v2:172.21.15.164:3300/0 conn(0x7f3c180ceb90 msgr2=0x7f3c180b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 --2- 172.21.15.82:0/1047653676 >> v2:172.21.15.164:3300/0 conn(0x7f3c180ceb90 0x7f3c180b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1806efe0 con 0x7f3c1815e190 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f3c140019f0 con 0x7f3c1815e190 2024-03-14T12:16:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 >> v1:172.21.15.82:6789/0 conn(0x7f3c1815e190 legacy=0x7f3c180ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c040039b0 0x7f3c04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c040051c0 0x7f3c0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180d1180 con 0x7f3c04004800 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180cc0f0 con 0x7f3c040051c0 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0400c140 con 0x7f3c040039b0 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1da86640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180cdb50 0x7f3c180cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1047653676 >> v2:172.21.15.164:3300/0 conn(0x7f3c180ceb90 0x7f3c180b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1d285640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c040039b0 0x7f3c04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1da86640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c040051c0 0x7f3c0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 -- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c040039b0 msgr2=0x7f3c04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c040039b0 0x7f3c04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 -- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c040051c0 msgr2=0x7f3c0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c040051c0 0x7f3c0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.686+0000 7f3c1ca84640 1 -- 172.21.15.82:0/1047653676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c04003520 con 0x7f3c04004800 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.687+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c04004c80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3c080098a0 tx=0x7f3c04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e10c10083a54ba server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.687+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800f040 con 0x7f3c04004800 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.687+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08037600 con 0x7f3c04004800 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.687+0000 7f3c0ffff640 1 -- 172.21.15.82:0/1047653676 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c080379b0 con 0x7f3c04004800 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.688+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1047653676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 msgr2=0x7f3c04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.688+0000 7f3c1f4e9640 1 --2- 172.21.15.82:0/1047653676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c04004c80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3c080098a0 tx=0x7f3c04005830 comp rx=0 tx=0).stop 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.688+0000 7f3c1da86640 1 -- 172.21.15.82:0/1047653676 reap_dead start 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.688+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1047653676 shutdown_connections 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.688+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1047653676 >> 172.21.15.82:0/1047653676 conn(0x7f3c180b3610 msgr2=0x7f3c180b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.688+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1047653676 shutdown_connections 2024-03-14T12:16:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.688+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1047653676 wait complete. 2024-03-14T12:16:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.689+0000 7f3c1f4e9640 1 Processor -- start 2024-03-14T12:16:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.689+0000 7f3c1f4e9640 1 -- start start 2024-03-14T12:16:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1f4e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c18151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1f4e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180b4700 0x7f3c180b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1f4e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c180b60d0 0x7f3c180b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1f4e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180cfc30 con 0x7f3c04004800 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1f4e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180cc6b0 con 0x7f3c180b60d0 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1f4e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180cfdb0 con 0x7f3c180b4700 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c18151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c18151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50576/0 (socket says 172.21.15.82:50576) 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 -- 172.21.15.82:0/1914654139 learned_addr learned my addr 172.21.15.82:0/1914654139 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 -- 172.21.15.82:0/1914654139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180b4700 msgr2=0x7f3c180b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180b4700 0x7f3c180b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 -- 172.21.15.82:0/1914654139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c180b60d0 msgr2=0x7f3c180b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c180b60d0 0x7f3c180b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 -- 172.21.15.82:0/1914654139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180b51c0 con 0x7f3c04004800 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1d285640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c18151e10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3c140077f0 tx=0x7f3c1400a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c0dffb640 1 -- 172.21.15.82:0/1914654139 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140133f0 con 0x7f3c04004800 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c0dffb640 1 -- 172.21.15.82:0/1914654139 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c14018070 con 0x7f3c04004800 2024-03-14T12:16:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c180b54a0 con 0x7f3c04004800 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.690+0000 7f3c0dffb640 1 -- 172.21.15.82:0/1914654139 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140137f0 con 0x7f3c04004800 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.691+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c180b5a60 con 0x7f3c04004800 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.691+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1804f050 con 0x7f3c04004800 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.692+0000 7f3c0dffb640 1 -- 172.21.15.82:0/1914654139 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3c14013990 con 0x7f3c04004800 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.692+0000 7f3c0dffb640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.692+0000 7f3c0dffb640 1 -- 172.21.15.82:0/1914654139 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c1408b2d0 con 0x7f3c04004800 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.692+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.693+0000 7f3c1ca84640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c08036000 tx=0x7f3c08030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:36.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.697+0000 7f3c0dffb640 1 -- 172.21.15.82:0/1914654139 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3c14059e50 con 0x7f3c04004800 2024-03-14T12:16:36.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.902+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c180cbdb0 con 0x7f3c04004800 2024-03-14T12:16:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.903+0000 7f3c0dffb640 1 -- 172.21.15.82:0/1914654139 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3c14059750 con 0x7f3c04004800 2024-03-14T12:16:36.903 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:36.903 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5111,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/4078909817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4078909817},{"type":"v1","addr":"172.21.15.164:6834","nonce":4078909817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2349002857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2349002857},{"type":"v1","addr":"172.21.15.164:6835","nonce":2349002857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/285826310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":285826310},{"type":"v1","addr":"172.21.15.82:6835","nonce":285826310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:32.222787+0000","modified":"2024-03-14T12:16:34.220226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1422543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1422543802},{"type":"v1","addr":"172.21.15.82:6837","nonce":1422543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T12:16:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-03-14T12:16:36.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 msgr2=0x7f3bfc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c08036000 tx=0x7f3c08030000 comp rx=0 tx=0).stop 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 msgr2=0x7f3c18151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c18151e10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3c140077f0 tx=0x7f3c1400a750 comp rx=0 tx=0).stop 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 shutdown_connections 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c180b60d0 0x7f3c180b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c180b4700 0x7f3c180b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 --2- 172.21.15.82:0/1914654139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c04004800 0x7f3c18151e10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 >> 172.21.15.82:0/1914654139 conn(0x7f3c180b3610 msgr2=0x7f3c180ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 shutdown_connections 2024-03-14T12:16:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:36.905+0000 7f3c1f4e9640 1 -- 172.21.15.82:0/1914654139 wait complete. 2024-03-14T12:16:37.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T12:16:32.222787+0000', 'modified': '2024-03-14T12:16:34.220226+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6048}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6048': {'gid': 6048, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/1422543802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 1422543802}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 1422543802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T12:16:37.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T12:16:37.318 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-14T12:16:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.420+0000 7f5ff717e640 1 Processor -- start 2024-03-14T12:16:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.420+0000 7f5ff717e640 1 -- start start 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff717e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 0x7f5ff015e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff717e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5ff006b420 0x7f5ff015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff717e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff00d1590 con 0x7f5ff015df30 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff717e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff0161b70 con 0x7f5ff006b420 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff717e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0161e50 con 0x7f5ff006f7f0 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff4f1a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5ff015df30 0x7f5ff006ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41118/0 (socket says 172.21.15.82:41118) 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff4f1a640 1 -- 172.21.15.82:0/1502228741 learned_addr learned my addr 172.21.15.82:0/1502228741 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291002030 0 0) 0x7f5ff00d1590 con 0x7f5ff015df30 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5feffff640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 0x7f5ff015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd8003520 con 0x7f5ff015df30 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.421+0000 7f5ff571b640 1 --2- 172.21.15.82:0/1502228741 >> v2:172.21.15.164:3300/0 conn(0x7f5ff006b420 0x7f5ff015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 531996855 0 0) 0x7f5fd8003520 con 0x7f5ff015df30 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 msgr2=0x7f5ff015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 0x7f5ff015e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 >> v2:172.21.15.164:3300/0 conn(0x7f5ff006b420 msgr2=0x7f5ff015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 --2- 172.21.15.82:0/1502228741 >> v2:172.21.15.164:3300/0 conn(0x7f5ff006b420 0x7f5ff015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0162130 con 0x7f5ff015df30 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f5fe0001670 con 0x7f5ff015df30 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 >> v1:172.21.15.82:6789/0 conn(0x7f5ff015df30 legacy=0x7f5ff006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fd80039b0 0x7f5fd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8004800 0x7f5fd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 0x7f5fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0161b70 con 0x7f5fd8004800 2024-03-14T12:16:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0161e50 con 0x7f5fd80039b0 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd800b9c0 con 0x7f5ff006f7f0 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5feffff640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8004800 0x7f5fd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fd80039b0 0x7f5fd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5ff571b640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 0x7f5fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5feffff640 1 -- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 msgr2=0x7f5fd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5feffff640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ff006f7f0 0x7f5fd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5feffff640 1 -- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fd80039b0 msgr2=0x7f5fd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5feffff640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fd80039b0 0x7f5fd8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.422+0000 7f5feffff640 1 -- 172.21.15.82:0/1502228741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8003520 con 0x7f5fd8004800 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.423+0000 7f5feffff640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8004800 0x7f5fd8004c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009ac0 tx=0x7f5fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e269d4d01a81ae81 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.423+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4039070 con 0x7f5fd8004800 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.423+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe402c6b0 con 0x7f5fd8004800 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.423+0000 7f5fef7fe640 1 -- 172.21.15.82:0/1502228741 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4009690 con 0x7f5fd8004800 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.423+0000 7f5ff717e640 1 -- 172.21.15.82:0/1502228741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8004800 msgr2=0x7f5fd8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.423+0000 7f5ff717e640 1 --2- 172.21.15.82:0/1502228741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd8004800 0x7f5fd8004c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009ac0 tx=0x7f5fd8005830 comp rx=0 tx=0).stop 2024-03-14T12:16:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.424+0000 7f5ff571b640 1 -- 172.21.15.82:0/1502228741 reap_dead start 2024-03-14T12:16:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.424+0000 7f5ff717e640 1 -- 172.21.15.82:0/1502228741 shutdown_connections 2024-03-14T12:16:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.424+0000 7f5ff717e640 1 -- 172.21.15.82:0/1502228741 >> 172.21.15.82:0/1502228741 conn(0x7f5ff00d09d0 msgr2=0x7f5ff00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.424+0000 7f5ff717e640 1 -- 172.21.15.82:0/1502228741 shutdown_connections 2024-03-14T12:16:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.424+0000 7f5ff717e640 1 -- 172.21.15.82:0/1502228741 wait complete. 2024-03-14T12:16:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.425+0000 7f5ff717e640 1 Processor -- start 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.425+0000 7f5ff717e640 1 -- start start 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5fd8004800 0x7f5ff0156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0146290 0x7f5ff0148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff00cd370 0x7f5ff00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0162650 con 0x7f5ff0146290 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00b3c10 con 0x7f5ff00cd370 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01627d0 con 0x7f5fd8004800 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0146290 0x7f5ff0148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0146290 0x7f5ff0148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50598/0 (socket says 172.21.15.82:50598) 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 -- 172.21.15.82:0/2835348848 learned_addr learned my addr 172.21.15.82:0/2835348848 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5fd8004800 0x7f5ff0156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff571b640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff00cd370 0x7f5ff00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 -- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5fd8004800 msgr2=0x7f5ff0156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5fd8004800 0x7f5ff0156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 -- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff00cd370 msgr2=0x7f5ff00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff00cd370 0x7f5ff00cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 -- 172.21.15.82:0/2835348848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0149090 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5feffff640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0146290 0x7f5ff0148800 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4037040 tx=0x7f5fe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5fed7fa640 1 -- 172.21.15.82:0/2835348848 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4039050 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5fed7fa640 1 -- 172.21.15.82:0/2835348848 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4045070 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff0149370 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5fed7fa640 1 -- 172.21.15.82:0/2835348848 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4003a50 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.426+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff0148d40 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.428+0000 7f5fed7fa640 1 -- 172.21.15.82:0/2835348848 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5fe4003bf0 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.428+0000 7f5fed7fa640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5fdc06bc00 0x7f5fdc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.429+0000 7f5fed7fa640 1 -- 172.21.15.82:0/2835348848 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fe40ac570 con 0x7f5ff0146290 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.429+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5fdc06bc00 0x7f5fdc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.429+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5fdc06bc00 0x7f5fdc06e0c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ff015e940 tx=0x7f5fe0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.429+0000 7f5ff9e39740 1 Processor -- start 2024-03-14T12:16:37.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.429+0000 7f5ff9e39740 1 -- start start 2024-03-14T12:16:37.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc21110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc21650 0x5625ecbf9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecbf9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ecb5e7c0 con 0x5625ecbfa9c0 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ecb07b50 con 0x5625ecc21650 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625ecb5a860 con 0x5625ecbfa5e0 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecbf9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecbf9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50610/0 (socket says 172.21.15.82:50610) 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 -- 172.21.15.82:0/2376451331 learned_addr learned my addr 172.21.15.82:0/2376451331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc21110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5feffff640 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc21650 0x5625ecbf9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 -- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 msgr2=0x5625ecc21110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc21110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 -- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc21650 msgr2=0x5625ecbf9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc21650 0x5625ecbf9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.430+0000 7f5ff571b640 1 -- 172.21.15.82:0/2376451331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625ecb1b9a0 con 0x5625ecbfa9c0 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff571b640 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecbf9b60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00b6b30 tx=0x7f5fe8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912a25e78796cbdd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5fd67fc640 1 -- 172.21.15.82:0/2376451331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8003da0 con 0x5625ecbfa9c0 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5fd67fc640 1 -- 172.21.15.82:0/2376451331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe801d070 con 0x5625ecbfa9c0 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5fd67fc640 1 -- 172.21.15.82:0/2376451331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8004230 con 0x5625ecbfa9c0 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2376451331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 msgr2=0x5625ecbf9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecbf9b60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00b6b30 tx=0x7f5fe8009750 comp rx=0 tx=0).stop 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2376451331 shutdown_connections 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecbf9b60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc21650 0x5625ecbf9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2376451331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc21110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2376451331 >> 172.21.15.82:0/2376451331 conn(0x5625ecbc1400 msgr2=0x5625ecc3f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2376451331 shutdown_connections 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.431+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2376451331 wait complete. 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.432+0000 7f5ff9e39740 1 -- start start 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.432+0000 7f5ff9e39740 1 Processor -- start 2024-03-14T12:16:37.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc18990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa9c0 0x5625ecc3adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecc1f6e0 0x5625ecc4f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ecc20c80 con 0x5625ecc1f6e0 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ecb1af90 con 0x5625ecbfa9c0 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625ecc19a00 con 0x5625ecbfa5e0 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff9e39740 10 client.0 _ll_register_callbacks cb 0x5625ecb56400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecc1f6e0 0x5625ecc4f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecc1f6e0 0x5625ecc4f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50616/0 (socket says 172.21.15.82:50616) 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 -- 172.21.15.82:0/2378206331 learned_addr learned my addr 172.21.15.82:0/2378206331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5feffff640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa9c0 0x5625ecc3adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc18990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 -- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 msgr2=0x5625ecc18990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc18990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 -- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa9c0 msgr2=0x5625ecc3adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa9c0 0x5625ecc3adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff571b640 1 -- 172.21.15.82:0/2378206331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625ec8732d0 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.433+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc18990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff571b640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecc1f6e0 0x5625ecc4f520 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0071430 tx=0x7f5fe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4482bbf9b3fc4f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5fcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/2378206331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8016b60 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/2378206331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fe8016d00 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fe801d070 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/2378206331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fe8022400 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 10 client.6192 fetch_fsmap learned FSMap version 27 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5625ec93d6b0 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/2378206331 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe801d070 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/2378206331 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5fe80225a0 con 0x5625ecc1f6e0 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 10 client.6192 fetch_fsmap finished waiting for FSMap version 27 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 10 client.6192 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 20 client.6192 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625ecc363f0 0x5625ecc367d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 4 client.6192 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 --> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625ecb6bd00 con 0x5625ecc363f0 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.434+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625ecc363f0 0x5625ecc367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5fcf7fe640 10 client.6192 ms_handle_connect on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625ecc363f0 0x5625ecc367d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47bd1fbe8411b1b server_cookie=b86d21252787e53 in_seq=0 out_seq=0 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/2378206331 <== mds.0 v2:172.21.15.82:6836/1422543802 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625ecb6bd00 con 0x5625ecc363f0 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5fcf7fe640 10 client.6192 handle_command_reply: tid=0 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 client.6192 shutdown 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecc1f6e0 msgr2=0x5625ecc4f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecc1f6e0 0x5625ecc4f520 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0071430 tx=0x7f5fe8004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 shutdown_connections 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625ecc363f0 0x5625ecc367d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecc1f6e0 0x5625ecc4f520 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa9c0 0x5625ecc3adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/2378206331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa5e0 0x5625ecc18990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5fcf7fe640 0 client.6192 ms_handle_reset on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:37.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 >> 172.21.15.82:0/2378206331 conn(0x5625ecbc1400 msgr2=0x5625ecc222e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 shutdown_connections 2024-03-14T12:16:37.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.435+0000 7f5ff9e39740 1 -- 172.21.15.82:0/2378206331 wait complete. 2024-03-14T12:16:37.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.436+0000 7f5ff9e39740 20 client.6192 trim_cache size 0 max 16384 2024-03-14T12:16:37.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.451+0000 7f5ff9e39740 1 Processor -- start 2024-03-14T12:16:37.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.451+0000 7f5ff9e39740 1 -- start start 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa5e0 0x5625ecc27310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa9c0 0x5625ecc9e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc1f6e0 0x5625ecc9eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01ffa00 con 0x5625ecbfa5e0 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff015d1e0 con 0x5625ecc1f6e0 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01ff580 con 0x5625ecbfa9c0 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa5e0 0x5625ecc27310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa5e0 0x5625ecc27310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50618/0 (socket says 172.21.15.82:50618) 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 -- 172.21.15.82:0/3157471394 learned_addr learned my addr 172.21.15.82:0/3157471394 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:37.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff571b640 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc1f6e0 0x5625ecc9eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5feffff640 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa9c0 0x5625ecc9e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 -- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa9c0 msgr2=0x5625ecc9e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa9c0 0x5625ecc9e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 -- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc1f6e0 msgr2=0x5625ecc9eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc1f6e0 0x5625ecc9eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 -- 172.21.15.82:0/3157471394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625ecc98580 con 0x5625ecbfa5e0 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa5e0 0x5625ecc27310 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00283f0 tx=0x7f5fe000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b735a7316a2b4b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.452+0000 7f5fd5ffb640 1 -- 172.21.15.82:0/3157471394 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00288b0 con 0x5625ecbfa5e0 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5fd5ffb640 1 -- 172.21.15.82:0/3157471394 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe0028b00 con 0x5625ecbfa5e0 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5fd5ffb640 1 -- 172.21.15.82:0/3157471394 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0015ad0 con 0x5625ecbfa5e0 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 -- 172.21.15.82:0/3157471394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa5e0 msgr2=0x5625ecc27310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa5e0 0x5625ecc27310 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00283f0 tx=0x7f5fe000c000 comp rx=0 tx=0).stop 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 -- 172.21.15.82:0/3157471394 shutdown_connections 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecc1f6e0 0x5625ecc9eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecbfa9c0 0x5625ecc9e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/3157471394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa5e0 0x5625ecc27310 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 -- 172.21.15.82:0/3157471394 >> 172.21.15.82:0/3157471394 conn(0x5625ecbc1400 msgr2=0x5625ecc27ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 -- 172.21.15.82:0/3157471394 shutdown_connections 2024-03-14T12:16:37.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.453+0000 7f5ff9e39740 1 -- 172.21.15.82:0/3157471394 wait complete. 2024-03-14T12:16:37.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.454+0000 7f5ff9e39740 1 -- start start 2024-03-14T12:16:37.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.454+0000 7f5ff9e39740 1 Processor -- start 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa5e0 0x5625ecc3c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecc58370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecc1f6e0 0x5625ecc588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ecc4ff30 con 0x5625ecbfa9c0 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ecc3a390 con 0x5625ecbfa5e0 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625ecc37d00 con 0x5625ecc1f6e0 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 10 client.0 _ll_register_callbacks cb 0x7f5fe8009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecc58370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecc58370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50634/0 (socket says 172.21.15.82:50634) 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 -- 172.21.15.82:0/410232066 learned_addr learned my addr 172.21.15.82:0/410232066 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa5e0 0x5625ecc3c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff571b640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecc1f6e0 0x5625ecc588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 -- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecc1f6e0 msgr2=0x5625ecc588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecc1f6e0 0x5625ecc588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 -- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa5e0 msgr2=0x5625ecc3c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa5e0 0x5625ecc3c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 -- 172.21.15.82:0/410232066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625ecc50110 con 0x5625ecbfa9c0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff571b640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecc1f6e0 0x5625ecc588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5feffff640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecc58370 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5fe407a720 tx=0x7f5fe407a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a9db3f9c4c27c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5fcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/410232066 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4082ea0 con 0x5625ecbfa9c0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/410232066 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fe40ac0b0 con 0x5625ecbfa9c0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fe0028b00 con 0x5625ecbfa9c0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/410232066 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fe407b120 con 0x5625ecbfa9c0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 10 client.6198 fetch_fsmap learned FSMap version 27 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.455+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5625ecc19e20 con 0x5625ecbfa9c0 2024-03-14T12:16:37.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/410232066 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe0028b00 con 0x5625ecbfa9c0 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/410232066 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5fe4099bc0 con 0x5625ecbfa9c0 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 10 client.6198 fetch_fsmap finished waiting for FSMap version 27 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 10 client.6198 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 20 client.6198 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625eccaa3e0 0x5625eccaa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 4 client.6198 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 --> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5625ecb66050 con 0x5625eccaa3e0 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625eccaa3e0 0x5625eccaa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5fcf7fe640 10 client.6198 ms_handle_connect on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff4f1a640 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625eccaa3e0 0x5625eccaa7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6eb7a4e9fb4702 server_cookie=eeb77df2e681883e in_seq=0 out_seq=0 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5fcf7fe640 1 -- 172.21.15.82:0/410232066 <== mds.0 v2:172.21.15.82:6836/1422543802 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5625ecb66050 con 0x5625eccaa3e0 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5fcf7fe640 10 client.6198 handle_command_reply: tid=0 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 client.6198 shutdown 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 msgr2=0x5625ecc58370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecc58370 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5fe407a720 tx=0x7f5fe407a440 comp rx=0 tx=0).stop 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 shutdown_connections 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5625ecc1f6e0 0x5625ecc588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x5625eccaa3e0 0x5625eccaa7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625ecbfa9c0 0x5625ecc58370 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 --2- 172.21.15.82:0/410232066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625ecbfa5e0 0x5625ecc3c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5fcf7fe640 0 client.6198 ms_handle_reset on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 >> 172.21.15.82:0/410232066 conn(0x5625ecbc1400 msgr2=0x5625ecc34c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 shutdown_connections 2024-03-14T12:16:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.456+0000 7f5ff9e39740 1 -- 172.21.15.82:0/410232066 wait complete. 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff9e39740 20 client.6198 trim_cache size 0 max 16384 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2", 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 6048, 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.5889244700000003, 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 58, 2024-03-14T12:16:37.464 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 58, 2024-03-14T12:16:37.465 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 9.657524961 2024-03-14T12:16:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:37.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5fdc06bc00 msgr2=0x7f5fdc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5fdc06bc00 0x7f5fdc06e0c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ff015e940 tx=0x7f5fe0009040 comp rx=0 tx=0).stop 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0146290 msgr2=0x7f5ff0148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0146290 0x7f5ff0148800 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4037040 tx=0x7f5fe4004730 comp rx=0 tx=0).stop 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 shutdown_connections 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ff00cd370 0x7f5ff00cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0146290 0x7f5ff0148800 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5fdc06bc00 0x7f5fdc06e0c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 --2- 172.21.15.82:0/2835348848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5fd8004800 0x7f5ff0156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 >> 172.21.15.82:0/2835348848 conn(0x7f5ff00d09d0 msgr2=0x7f5ff00ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 shutdown_connections 2024-03-14T12:16:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.457+0000 7f5ff717e640 1 -- 172.21.15.82:0/2835348848 wait complete. 2024-03-14T12:16:37.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:16:37.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.569+0000 7f8303a9a640 1 Processor -- start 2024-03-14T12:16:37.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.570+0000 7f8303a9a640 1 -- start start 2024-03-14T12:16:37.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.570+0000 7f8303a9a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc071740 0x7f82fc071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.570+0000 7f8303a9a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f82fc15e670 0x7f82fc0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.570+0000 7f8303a9a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc072060 con 0x7f82fc143f60 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.570+0000 7f8303a9a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc15ebe0 con 0x7f82fc15e670 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.570+0000 7f8303a9a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc15eec0 con 0x7f82fc071740 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.570+0000 7f8302037640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f82fc143f60 0x7f82fc15b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41134/0 (socket says 172.21.15.82:41134) 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8302037640 1 -- 172.21.15.82:0/438929498 learned_addr learned my addr 172.21.15.82:0/438929498 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8301836640 1 --2- 172.21.15.82:0/438929498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc071740 0x7f82fc071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8301035640 1 --2- 172.21.15.82:0/438929498 >> v2:172.21.15.164:3300/0 conn(0x7f82fc15e670 0x7f82fc0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8300834640 1 -- 172.21.15.82:0/438929498 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169017166 0 0) 0x7f82fc072060 con 0x7f82fc143f60 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8300834640 1 -- 172.21.15.82:0/438929498 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e8003520 con 0x7f82fc143f60 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8301836640 1 -- 172.21.15.82:0/438929498 >> v2:172.21.15.164:3300/0 conn(0x7f82fc15e670 msgr2=0x7f82fc0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8301836640 1 --2- 172.21.15.82:0/438929498 >> v2:172.21.15.164:3300/0 conn(0x7f82fc15e670 0x7f82fc0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8301836640 1 -- 172.21.15.82:0/438929498 >> v1:172.21.15.82:6789/0 conn(0x7f82fc143f60 legacy=0x7f82fc15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8300834640 1 -- 172.21.15.82:0/438929498 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3984980189 0 0) 0x7f82e8003520 con 0x7f82fc143f60 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8301836640 1 -- 172.21.15.82:0/438929498 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc15f1a0 con 0x7f82fc071740 2024-03-14T12:16:37.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.571+0000 7f8301035640 1 --2- 172.21.15.82:0/438929498 >> v2:172.21.15.164:3300/0 conn(0x7f82fc15e670 0x7f82fc0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.572+0000 7f8301836640 1 --2- 172.21.15.82:0/438929498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc071740 0x7f82fc071b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0097a0 tx=0x7f82ec02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f5517ceba05496 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.572+0000 7f8300834640 1 -- 172.21.15.82:0/438929498 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037070 con 0x7f82fc071740 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.572+0000 7f8300834640 1 -- 172.21.15.82:0/438929498 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec030510 con 0x7f82fc071740 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.572+0000 7f8300834640 1 -- 172.21.15.82:0/438929498 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40019f0 con 0x7f82fc071740 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 -- 172.21.15.82:0/438929498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc071740 msgr2=0x7f82fc071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 --2- 172.21.15.82:0/438929498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc071740 0x7f82fc071b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0097a0 tx=0x7f82ec02b790 comp rx=0 tx=0).stop 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 -- 172.21.15.82:0/438929498 shutdown_connections 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 --2- 172.21.15.82:0/438929498 >> v2:172.21.15.164:3300/0 conn(0x7f82fc15e670 0x7f82fc0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 --2- 172.21.15.82:0/438929498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc071740 0x7f82fc071b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 -- 172.21.15.82:0/438929498 >> 172.21.15.82:0/438929498 conn(0x7f82fc0b1c70 msgr2=0x7f82fc0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 -- 172.21.15.82:0/438929498 shutdown_connections 2024-03-14T12:16:37.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.573+0000 7f8303a9a640 1 -- 172.21.15.82:0/438929498 wait complete. 2024-03-14T12:16:37.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 Processor -- start 2024-03-14T12:16:37.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 -- start start 2024-03-14T12:16:37.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82fc071740 0x7f82fc158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82fc143f60 0x7f82fc0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc15e670 0x7f82fc0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc15b080 con 0x7f82fc071740 2024-03-14T12:16:37.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc143ce0 con 0x7f82fc143f60 2024-03-14T12:16:37.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.574+0000 7f8303a9a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc144670 con 0x7f82fc15e670 2024-03-14T12:16:37.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82fc071740 0x7f82fc158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82fc071740 0x7f82fc158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50646/0 (socket says 172.21.15.82:50646) 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 -- 172.21.15.82:0/4153931318 learned_addr learned my addr 172.21.15.82:0/4153931318 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301035640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82fc143f60 0x7f82fc0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8302037640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc15e670 0x7f82fc0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 -- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc15e670 msgr2=0x7f82fc0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc15e670 0x7f82fc0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 -- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82fc143f60 msgr2=0x7f82fc0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82fc143f60 0x7f82fc0c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 -- 172.21.15.82:0/4153931318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0cbb30 con 0x7f82fc071740 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8301836640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82fc071740 0x7f82fc158310 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f82ec02b7d0 tx=0x7f82ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f82f27fc640 1 -- 172.21.15.82:0/4153931318 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec040020 con 0x7f82fc071740 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f82f27fc640 1 -- 172.21.15.82:0/4153931318 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec038040 con 0x7f82fc071740 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc1f7be0 con 0x7f82fc071740 2024-03-14T12:16:37.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f82f27fc640 1 -- 172.21.15.82:0/4153931318 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037070 con 0x7f82fc071740 2024-03-14T12:16:37.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.575+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc1f81a0 con 0x7f82fc071740 2024-03-14T12:16:37.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.576+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc04f050 con 0x7f82fc071740 2024-03-14T12:16:37.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.577+0000 7f82f27fc640 1 -- 172.21.15.82:0/4153931318 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f82ec043020 con 0x7f82fc071740 2024-03-14T12:16:37.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.577+0000 7f82f27fc640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82e006bbd0 0x7f82e006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:37.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.577+0000 7f82f27fc640 1 -- 172.21.15.82:0/4153931318 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82ec0abc00 con 0x7f82fc071740 2024-03-14T12:16:37.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.577+0000 7f8301035640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82e006bbd0 0x7f82e006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:37.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.577+0000 7f8301035640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82e006bbd0 0x7f82e006e090 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009560 tx=0x7f82f8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:37.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.581+0000 7f82f27fc640 1 -- 172.21.15.82:0/4153931318 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f82ec0b2080 con 0x7f82fc071740 2024-03-14T12:16:37.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.788+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f82fc071520 con 0x7f82fc071740 2024-03-14T12:16:37.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.788+0000 7f82f27fc640 1 -- 172.21.15.82:0/4153931318 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f82ec04a070 con 0x7f82fc071740 2024-03-14T12:16:37.789 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82e006bbd0 msgr2=0x7f82e006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82e006bbd0 0x7f82e006e090 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009560 tx=0x7f82f8008040 comp rx=0 tx=0).stop 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82fc071740 msgr2=0x7f82fc158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82fc071740 0x7f82fc158310 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f82ec02b7d0 tx=0x7f82ec0046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 shutdown_connections 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f82e006bbd0 0x7f82e006e090 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82fc15e670 0x7f82fc0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82fc143f60 0x7f82fc0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 --2- 172.21.15.82:0/4153931318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82fc071740 0x7f82fc158310 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 >> 172.21.15.82:0/4153931318 conn(0x7f82fc0b1c70 msgr2=0x7f82fc13e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:37.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 shutdown_connections 2024-03-14T12:16:37.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:37.790+0000 7f8303a9a640 1 -- 172.21.15.82:0/4153931318 wait complete. 2024-03-14T12:16:37.801 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:16:37.802 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:37.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:37.802 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:37.817 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42522sec preferred_lft 42522sec 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:16:37.818 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c443:69ff:fec0:3488/64 scope link 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::887e:60ff:fe47:4333/64 scope link 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a6:e3:19:81:dc:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a4e3:19ff:fe81:dc41/64 scope link 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:37.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:37.819 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:16:37.882 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:16:37.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:37.884 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:16:37.939 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:16:37.939 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:16:37.939 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:16:37.939 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:16:37.939 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:16:37.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:37.940 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:16:37.940 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:16:37.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:16:37.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:16:37.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:16:37.940 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:16:38.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.035 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:16:38.049 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-14T12:16:38.049 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:16:38.049 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:38.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.049 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-14T12:16:38.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:38.116 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:38.133 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-03-14T12:16:38.133+0000 7f18177a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:16:38.147 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:16:38.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:38.148 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:38.202 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T12:16:38.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.202 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:38.286 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:38.286 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:38.300 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:38.301 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:38.301 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:16:38.301 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:16:38.301 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:16:38.302 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T12:16:38.302 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:16:38.302 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:16:38.302 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:16:38.302 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:16:38.304 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:16:38.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.304 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:38.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.331 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:16:38.391 INFO:teuthology.orchestra.run.smithi082.stdout:client5198 2024-03-14T12:16:38.391 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:38.391 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:38.446 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:16:38.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.586+0000 7f3c96f7e640 1 Processor -- start 2024-03-14T12:16:38.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.586+0000 7f3c96f7e640 1 -- start start 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c96f7e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3c9015f7a0 0x7f3c9015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c96f7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c900cdd60 0x7f3c900ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c96f7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c901600c0 con 0x7f3c90144ac0 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c96f7e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c900cf150 con 0x7f3c9015f7a0 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c96f7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900ce8b0 con 0x7f3c900cdd60 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c900cdd60 0x7f3c900ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c94d1a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3c9015f7a0 0x7f3c9015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c900cdd60 0x7f3c900ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:32936/0 (socket says 172.21.15.82:32936) 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8ffff640 1 -- 172.21.15.82:0/577725416 learned_addr learned my addr 172.21.15.82:0/577725416 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8ffff640 1 -- 172.21.15.82:0/577725416 >> v2:172.21.15.164:3300/0 conn(0x7f3c9015f7a0 msgr2=0x7f3c9015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:38.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8ffff640 1 --2- 172.21.15.82:0/577725416 >> v2:172.21.15.164:3300/0 conn(0x7f3c9015f7a0 0x7f3c9015fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8ffff640 1 -- 172.21.15.82:0/577725416 >> v1:172.21.15.164:6789/0 conn(0x7f3c90144ac0 legacy=0x7f3c90144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:38.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8f7fe640 1 -- 172.21.15.82:0/577725416 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777296070 0 0) 0x7f3c901600c0 con 0x7f3c90144ac0 2024-03-14T12:16:38.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.587+0000 7f3c8ffff640 1 -- 172.21.15.82:0/577725416 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900ceb90 con 0x7f3c900cdd60 2024-03-14T12:16:38.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.588+0000 7f3c8ffff640 1 --2- 172.21.15.82:0/577725416 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c900cdd60 0x7f3c900ce210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009870 tx=0x7f3c7c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2af2ebe6b92151b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:38.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.588+0000 7f3c8f7fe640 1 -- 172.21.15.82:0/577725416 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c035020 con 0x7f3c900cdd60 2024-03-14T12:16:38.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.588+0000 7f3c8f7fe640 1 -- 172.21.15.82:0/577725416 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c031640 con 0x7f3c900cdd60 2024-03-14T12:16:38.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.588+0000 7f3c8f7fe640 1 -- 172.21.15.82:0/577725416 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c02d040 con 0x7f3c900cdd60 2024-03-14T12:16:38.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 -- 172.21.15.82:0/577725416 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c900cdd60 msgr2=0x7f3c900ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:38.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/577725416 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c900cdd60 0x7f3c900ce210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009870 tx=0x7f3c7c02b8f0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 -- 172.21.15.82:0/577725416 shutdown_connections 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/577725416 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c900cdd60 0x7f3c900ce210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/577725416 >> v2:172.21.15.164:3300/0 conn(0x7f3c9015f7a0 0x7f3c9015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 -- 172.21.15.82:0/577725416 >> 172.21.15.82:0/577725416 conn(0x7f3c900b1c70 msgr2=0x7f3c900cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 -- 172.21.15.82:0/577725416 shutdown_connections 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.589+0000 7f3c96f7e640 1 -- 172.21.15.82:0/577725416 wait complete. 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.590+0000 7f3c96f7e640 1 Processor -- start 2024-03-14T12:16:38.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.590+0000 7f3c96f7e640 1 -- start start 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c96f7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c900cdd60 0x7f3c9013ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c96f7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c90144ac0 0x7f3c9013def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c96f7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c9015f7a0 0x7f3c901f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c96f7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900cc4f0 con 0x7f3c90144ac0 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c96f7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9006c4e0 con 0x7f3c900cdd60 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c96f7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900cc670 con 0x7f3c9015f7a0 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c90144ac0 0x7f3c9013def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c90144ac0 0x7f3c9013def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50656/0 (socket says 172.21.15.82:50656) 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 -- 172.21.15.82:0/733009791 learned_addr learned my addr 172.21.15.82:0/733009791 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c94d1a640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c900cdd60 0x7f3c9013ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c9551b640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c9015f7a0 0x7f3c901f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 -- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c9015f7a0 msgr2=0x7f3c901f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c9015f7a0 0x7f3c901f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 -- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c900cdd60 msgr2=0x7f3c9013ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c900cdd60 0x7f3c9013ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.591+0000 7f3c8ffff640 1 -- 172.21.15.82:0/733009791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9013f640 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.592+0000 7f3c8ffff640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c90144ac0 0x7f3c9013def0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c02b930 tx=0x7f3c7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.592+0000 7f3c8d7fa640 1 -- 172.21.15.82:0/733009791 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c040070 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.592+0000 7f3c8d7fa640 1 -- 172.21.15.82:0/733009791 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c03e070 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.592+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9013f920 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.592+0000 7f3c8d7fa640 1 -- 172.21.15.82:0/733009791 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c035040 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.592+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c901401c0 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.593+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9013fd10 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.593+0000 7f3c8d7fa640 1 -- 172.21.15.82:0/733009791 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3c7c038020 con 0x7f3c90144ac0 2024-03-14T12:16:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.593+0000 7f3c8d7fa640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3c7406bbd0 0x7f3c7406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:38.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.594+0000 7f3c8d7fa640 1 -- 172.21.15.82:0/733009791 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c7c0ac180 con 0x7f3c90144ac0 2024-03-14T12:16:38.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.594+0000 7f3c94d1a640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3c7406bbd0 0x7f3c7406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:38.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.594+0000 7f3c94d1a640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3c7406bbd0 0x7f3c7406e090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3c9015f540 tx=0x7f3c84009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:38.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.598+0000 7f3c8d7fa640 1 -- 172.21.15.82:0/733009791 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3c7c07a6b0 con 0x7f3c90144ac0 2024-03-14T12:16:38.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.806+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c9004f050 con 0x7f3c90144ac0 2024-03-14T12:16:38.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.806+0000 7f3c8d7fa640 1 -- 172.21.15.82:0/733009791 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3c7c07e750 con 0x7f3c90144ac0 2024-03-14T12:16:38.807 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3c7406bbd0 msgr2=0x7f3c7406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3c7406bbd0 0x7f3c7406e090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3c9015f540 tx=0x7f3c84009190 comp rx=0 tx=0).stop 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c90144ac0 msgr2=0x7f3c9013def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c90144ac0 0x7f3c9013def0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c02b930 tx=0x7f3c7c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 shutdown_connections 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3c7406bbd0 0x7f3c7406e090 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c9015f7a0 0x7f3c901f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c90144ac0 0x7f3c9013def0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.808+0000 7f3c96f7e640 1 --2- 172.21.15.82:0/733009791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c900cdd60 0x7f3c9013ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:38.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.809+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 >> 172.21.15.82:0/733009791 conn(0x7f3c900b1c70 msgr2=0x7f3c900709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:38.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.809+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 shutdown_connections 2024-03-14T12:16:38.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:38.809+0000 7f3c96f7e640 1 -- 172.21.15.82:0/733009791 wait complete. 2024-03-14T12:16:38.819 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:16:38.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:16:38.820 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-14T12:16:38.820 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T12:16:38.820 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:38.820 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:16:38.820 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:16:38.820 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:16:38.820 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:16:38.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.820 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-14T12:16:38.839 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:16:38.839 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:16:38.839 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:16:38.839 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:38.839 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:16:38.839 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42557sec preferred_lft 42557sec 2024-03-14T12:16:38.840 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::882:aaff:fed8:c2be/64 scope link 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::dc5a:2ff:fe9f:88ac/64 scope link 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether f6:de:82:8b:5d:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::f4de:82ff:fe8b:5dfb/64 scope link 2024-03-14T12:16:38.841 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:16:38.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.842 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:16:38.909 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-14T12:16:38.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.911 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-14T12:16:38.969 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-14T12:16:38.969 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:16:38.969 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:16:38.970 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:16:38.970 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:16:38.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:38.970 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:16:38.970 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:16:38.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:16:38.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:16:38.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:16:38.970 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:16:39.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:39.069 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:16:39.084 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-14T12:16:39.084 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T12:16:39.084 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:16:39.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:39.085 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-03-14T12:16:39.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:39.152 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:39.207 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:16:39.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:16:39.208 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:39.263 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T12:16:39.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:39.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:39.348 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:39.348 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:39.356 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:16:39.356 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:16:39.356 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:16:39.356 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with new device syntax: 1@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:16:39.357 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:16:39.357 INFO:teuthology.orchestra.run.smithi164.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-14T12:16:39.357 INFO:teuthology.orchestra.run.smithi164.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:16:39.357 INFO:teuthology.orchestra.run.smithi164.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:16:39.357 INFO:teuthology.orchestra.run.smithi164.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:16:39.357 INFO:teuthology.orchestra.run.smithi164.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:16:39.359 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:16:39.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:39.359 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:39.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:39.386 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:16:39.441 INFO:teuthology.orchestra.run.smithi164.stdout:client6210 2024-03-14T12:16:39.441 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:39.441 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:39.497 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:39.497 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:39.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:39.514 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:16:39.571 INFO:teuthology.orchestra.run.smithi082.stdout:client5198 2024-03-14T12:16:39.572 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:39.572 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:39.628 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:39.628 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:39.683 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:39.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.827+0000 7fee6e24b640 1 Processor -- start 2024-03-14T12:16:39.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.827+0000 7fee6e24b640 1 -- start start 2024-03-14T12:16:39.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee6e24b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee68071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee6e24b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee68143f60 0x7fee6815b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee6e24b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee680b3050 con 0x7fee6815e670 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee6e24b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee68072060 con 0x7fee68143f60 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee6e24b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815ebe0 con 0x7fee68071740 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee67fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee68143f60 0x7fee6815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee67fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee68143f60 0x7fee6815b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50658/0 (socket says 172.21.15.82:50658) 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee67fff640 1 -- 172.21.15.82:0/1319849447 learned_addr learned my addr 172.21.15.82:0/1319849447 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee677fe640 1 --2- 172.21.15.82:0/1319849447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee68071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee67fff640 1 -- 172.21.15.82:0/1319849447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 msgr2=0x7fee68071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee67fff640 1 --2- 172.21.15.82:0/1319849447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee68071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee67fff640 1 -- 172.21.15.82:0/1319849447 >> v1:172.21.15.164:6789/0 conn(0x7fee6815e670 legacy=0x7fee680b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee67fff640 1 -- 172.21.15.82:0/1319849447 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6815eec0 con 0x7fee68143f60 2024-03-14T12:16:39.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.828+0000 7fee677fe640 1 --2- 172.21.15.82:0/1319849447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee68071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:39.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.829+0000 7fee67fff640 1 --2- 172.21.15.82:0/1319849447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee68143f60 0x7fee6815b610 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fee60009e50 tx=0x7fee6002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd397cf7bdb6c7a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:39.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.829+0000 7fee667fc640 1 -- 172.21.15.82:0/1319849447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6002ca70 con 0x7fee68143f60 2024-03-14T12:16:39.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.829+0000 7fee667fc640 1 -- 172.21.15.82:0/1319849447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6002cc10 con 0x7fee68143f60 2024-03-14T12:16:39.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.829+0000 7fee667fc640 1 -- 172.21.15.82:0/1319849447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee600343f0 con 0x7fee68143f60 2024-03-14T12:16:39.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 -- 172.21.15.82:0/1319849447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee68143f60 msgr2=0x7fee6815b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:39.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 --2- 172.21.15.82:0/1319849447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee68143f60 0x7fee6815b610 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fee60009e50 tx=0x7fee6002bed0 comp rx=0 tx=0).stop 2024-03-14T12:16:39.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 -- 172.21.15.82:0/1319849447 shutdown_connections 2024-03-14T12:16:39.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 --2- 172.21.15.82:0/1319849447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee68143f60 0x7fee6815b610 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:39.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 --2- 172.21.15.82:0/1319849447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee68071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:39.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 -- 172.21.15.82:0/1319849447 >> 172.21.15.82:0/1319849447 conn(0x7fee680b1c70 msgr2=0x7fee680cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:39.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 -- 172.21.15.82:0/1319849447 shutdown_connections 2024-03-14T12:16:39.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.830+0000 7fee6e24b640 1 -- 172.21.15.82:0/1319849447 wait complete. 2024-03-14T12:16:39.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.831+0000 7fee6e24b640 1 Processor -- start 2024-03-14T12:16:39.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.831+0000 7fee6e24b640 1 -- start start 2024-03-14T12:16:39.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.831+0000 7fee6e24b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee6814eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:39.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee6e24b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee68143f60 0x7fee680c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:39.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee6e24b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee6815e670 0x7fee680c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:39.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee6e24b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68143ce0 con 0x7fee6815e670 2024-03-14T12:16:39.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee6e24b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815f480 con 0x7fee68143f60 2024-03-14T12:16:39.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee6e24b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815b080 con 0x7fee68071740 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee6815e670 0x7fee680c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee6815e670 0x7fee680c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50662/0 (socket says 172.21.15.82:50662) 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 -- 172.21.15.82:0/2969015262 learned_addr learned my addr 172.21.15.82:0/2969015262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee66ffd640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee68143f60 0x7fee680c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee677fe640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee6814eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 -- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 msgr2=0x7fee6814eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee6814eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 -- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee68143f60 msgr2=0x7fee680c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee68143f60 0x7fee680c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 -- 172.21.15.82:0/2969015262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6815dc80 con 0x7fee6815e670 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee66ffd640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee68143f60 0x7fee680c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee67fff640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee6815e670 0x7fee680c4df0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fee60009e50 tx=0x7fee60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee4ffff640 1 -- 172.21.15.82:0/2969015262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60043020 con 0x7fee6815e670 2024-03-14T12:16:39.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee4ffff640 1 -- 172.21.15.82:0/2969015262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6003e450 con 0x7fee6815e670 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6813e6c0 con 0x7fee6815e670 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee4ffff640 1 -- 172.21.15.82:0/2969015262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60009510 con 0x7fee6815e670 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.832+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6813ec80 con 0x7fee6815e670 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.833+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6804f050 con 0x7fee6815e670 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.834+0000 7fee4ffff640 1 -- 172.21.15.82:0/2969015262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fee60046020 con 0x7fee6815e670 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.834+0000 7fee4ffff640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee5406bbd0 0x7fee5406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.834+0000 7fee4ffff640 1 -- 172.21.15.82:0/2969015262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee600ad380 con 0x7fee6815e670 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.834+0000 7fee677fe640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee5406bbd0 0x7fee5406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:39.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.834+0000 7fee677fe640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee5406bbd0 0x7fee5406e090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee5800a950 tx=0x7fee58008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:39.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:39.838+0000 7fee4ffff640 1 -- 172.21.15.82:0/2969015262 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fee6004d030 con 0x7fee6815e670 2024-03-14T12:16:40.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.017+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee680d07b0 con 0x7fee6815e670 2024-03-14T12:16:40.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.018+0000 7fee4ffff640 1 -- 172.21.15.82:0/2969015262 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16653 (secure 0 0 0) 0x7fee6002ca80 con 0x7fee6815e670 2024-03-14T12:16:40.019 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:40.019 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":58,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:33.876540+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee5406bbd0 msgr2=0x7fee5406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee5406bbd0 0x7fee5406e090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee5800a950 tx=0x7fee58008040 comp rx=0 tx=0).stop 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee6815e670 msgr2=0x7fee680c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee6815e670 0x7fee680c4df0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fee60009e50 tx=0x7fee60004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 shutdown_connections 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee6815e670 0x7fee680c4df0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee68143f60 0x7fee680c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fee5406bbd0 0x7fee5406e090 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 --2- 172.21.15.82:0/2969015262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fee68071740 0x7fee6814eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 >> 172.21.15.82:0/2969015262 conn(0x7fee680b1c70 msgr2=0x7fee680cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 shutdown_connections 2024-03-14T12:16:40.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.020+0000 7fee6e24b640 1 -- 172.21.15.82:0/2969015262 wait complete. 2024-03-14T12:16:40.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:30.434900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:31.188753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:40.032 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f433dbc4640 1 Processor -- start 2024-03-14T12:16:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f433dbc4640 1 -- start start 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f433dbc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f433dbc4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f433815fd00 0x7f43381601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f433dbc4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43380cf8a0 con 0x7f433815c0e0 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f433dbc4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380d0ba0 con 0x7f433815fd00 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f433dbc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43380d01d0 con 0x7f43381608c0 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f4337fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f433815fd00 0x7f43381601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f43377fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f433815c0e0 0x7f43380cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41142/0 (socket says 172.21.15.82:41142) 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.135+0000 7f43377fe640 1 -- 172.21.15.82:0/2385255996 learned_addr learned my addr 172.21.15.82:0/2385255996 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.136+0000 7f4337fff640 1 -- 172.21.15.82:0/2385255996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 msgr2=0x7f43380d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.136+0000 7f4336ffd640 1 --2- 172.21.15.82:0/2385255996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.136+0000 7f4337fff640 1 --2- 172.21.15.82:0/2385255996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.136+0000 7f43367fc640 1 -- 172.21.15.82:0/2385255996 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040531237 0 0) 0x7f43380cf8a0 con 0x7f433815c0e0 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.136+0000 7f4337fff640 1 -- 172.21.15.82:0/2385255996 >> v1:172.21.15.82:6789/0 conn(0x7f433815c0e0 legacy=0x7f43380cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.136+0000 7f4337fff640 1 -- 172.21.15.82:0/2385255996 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380cfb80 con 0x7f433815fd00 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.136+0000 7f4336ffd640 1 --2- 172.21.15.82:0/2385255996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f4337fff640 1 --2- 172.21.15.82:0/2385255996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43381601b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4330009d90 tx=0x7f433002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1291e92608ae2d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f43367fc640 1 -- 172.21.15.82:0/2385255996 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330039070 con 0x7f433815fd00 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f43367fc640 1 -- 172.21.15.82:0/2385255996 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433002cc50 con 0x7f433815fd00 2024-03-14T12:16:40.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f43367fc640 1 -- 172.21.15.82:0/2385255996 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330034440 con 0x7f433815fd00 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 -- 172.21.15.82:0/2385255996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 msgr2=0x7f43381601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 --2- 172.21.15.82:0/2385255996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43381601b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4330009d90 tx=0x7f433002bea0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 -- 172.21.15.82:0/2385255996 shutdown_connections 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 --2- 172.21.15.82:0/2385255996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43381601b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 --2- 172.21.15.82:0/2385255996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 -- 172.21.15.82:0/2385255996 >> 172.21.15.82:0/2385255996 conn(0x7f43380b3680 msgr2=0x7f43380b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 -- 172.21.15.82:0/2385255996 shutdown_connections 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.137+0000 7f433dbc4640 1 -- 172.21.15.82:0/2385255996 wait complete. 2024-03-14T12:16:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.138+0000 7f433dbc4640 1 Processor -- start 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f433dbc4640 1 -- start start 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f433dbc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f433815c0e0 0x7f43381590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f433dbc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43380cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f433dbc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f433dbc4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433815c4c0 con 0x7f433815fd00 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f433dbc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43380cd960 con 0x7f433815c0e0 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f433dbc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433815c640 con 0x7f43381608c0 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f4336ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43380cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f4336ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43380cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50676/0 (socket says 172.21.15.82:50676) 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.139+0000 7f4336ffd640 1 -- 172.21.15.82:0/3906619937 learned_addr learned my addr 172.21.15.82:0/3906619937 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f4337fff640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f43377fe640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f433815c0e0 0x7f43381590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f4336ffd640 1 -- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 msgr2=0x7f43380ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f4336ffd640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f4336ffd640 1 -- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f433815c0e0 msgr2=0x7f43381590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f4336ffd640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f433815c0e0 0x7f43381590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f4336ffd640 1 -- 172.21.15.82:0/3906619937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43381595f0 con 0x7f433815fd00 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f4336ffd640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43380cdbc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c7f0 tx=0x7f432c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f431ffff640 1 -- 172.21.15.82:0/3906619937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c015070 con 0x7f433815fd00 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f431ffff640 1 -- 172.21.15.82:0/3906619937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c007a30 con 0x7f433815fd00 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43380c8ca0 con 0x7f433815fd00 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f431ffff640 1 -- 172.21.15.82:0/3906619937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c007e00 con 0x7f433815fd00 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.140+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43380c9260 con 0x7f433815fd00 2024-03-14T12:16:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.141+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433804f050 con 0x7f433815fd00 2024-03-14T12:16:40.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.141+0000 7f431ffff640 1 -- 172.21.15.82:0/3906619937 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f432c018070 con 0x7f433815fd00 2024-03-14T12:16:40.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.142+0000 7f431ffff640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f432406bbd0 0x7f432406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.142+0000 7f431ffff640 1 -- 172.21.15.82:0/3906619937 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f432c08a910 con 0x7f433815fd00 2024-03-14T12:16:40.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.142+0000 7f43377fe640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f432406bbd0 0x7f432406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.142+0000 7f43377fe640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f432406bbd0 0x7f432406e090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4328000f60 tx=0x7f4328007c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.146+0000 7f431ffff640 1 -- 172.21.15.82:0/3906619937 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f43381595f0 con 0x7f433815fd00 2024-03-14T12:16:40.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.314+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43380b7ab0 con 0x7f433815fd00 2024-03-14T12:16:40.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.315+0000 7f431ffff640 1 -- 172.21.15.82:0/3906619937 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f432c059490 con 0x7f433815fd00 2024-03-14T12:16:40.315 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:40.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f432406bbd0 msgr2=0x7f432406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f432406bbd0 0x7f432406e090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4328000f60 tx=0x7f4328007c00 comp rx=0 tx=0).stop 2024-03-14T12:16:40.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 msgr2=0x7f43380cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43380cdbc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c7f0 tx=0x7f432c00ccb0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 shutdown_connections 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f432406bbd0 0x7f432406e090 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43381608c0 0x7f43380ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f433815fd00 0x7f43380cdbc0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 --2- 172.21.15.82:0/3906619937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f433815c0e0 0x7f43381590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 >> 172.21.15.82:0/3906619937 conn(0x7f43380b3680 msgr2=0x7f43380b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 shutdown_connections 2024-03-14T12:16:40.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.317+0000 7f433dbc4640 1 -- 172.21.15.82:0/3906619937 wait complete. 2024-03-14T12:16:40.328 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:16:40.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.430+0000 7feb9b913640 1 Processor -- start 2024-03-14T12:16:40.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.431+0000 7feb9b913640 1 -- start start 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.431+0000 7feb9b913640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feb9415bde0 0x7feb9415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.431+0000 7feb9b913640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb940cd250 0x7feb940cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.431+0000 7feb9b913640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940ce380 con 0x7feb940d01b0 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.431+0000 7feb9b913640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb940cef90 con 0x7feb9415bde0 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.431+0000 7feb9b913640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb940b5a30 con 0x7feb940cd250 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.432+0000 7feb996af640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feb9415bde0 0x7feb9415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.432+0000 7feb99eb0640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feb940d01b0 0x7feb940b6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41144/0 (socket says 172.21.15.82:41144) 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.432+0000 7feb99eb0640 1 -- 172.21.15.82:0/1864560357 learned_addr learned my addr 172.21.15.82:0/1864560357 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.432+0000 7feb996af640 1 -- 172.21.15.82:0/1864560357 >> v2:172.21.15.164:3300/0 conn(0x7feb940cd250 msgr2=0x7feb940cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.432+0000 7feb996af640 1 --2- 172.21.15.82:0/1864560357 >> v2:172.21.15.164:3300/0 conn(0x7feb940cd250 0x7feb940cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.432+0000 7feb996af640 1 -- 172.21.15.82:0/1864560357 >> v1:172.21.15.82:6789/0 conn(0x7feb940d01b0 legacy=0x7feb940b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.432+0000 7feb996af640 1 -- 172.21.15.82:0/1864560357 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940ce660 con 0x7feb9415bde0 2024-03-14T12:16:40.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb996af640 1 --2- 172.21.15.82:0/1864560357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb9415c1c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb94071950 tx=0x7feb9002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243cdd9a9efec6a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb8bfff640 1 -- 172.21.15.82:0/1864560357 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90038070 con 0x7feb9415bde0 2024-03-14T12:16:40.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb8bfff640 1 -- 172.21.15.82:0/1864560357 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb90033640 con 0x7feb9415bde0 2024-03-14T12:16:40.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb8bfff640 1 -- 172.21.15.82:0/1864560357 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90033a10 con 0x7feb9415bde0 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 -- 172.21.15.82:0/1864560357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 msgr2=0x7feb9415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 --2- 172.21.15.82:0/1864560357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb9415c1c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb94071950 tx=0x7feb9002b7a0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 -- 172.21.15.82:0/1864560357 shutdown_connections 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 --2- 172.21.15.82:0/1864560357 >> v2:172.21.15.164:3300/0 conn(0x7feb940cd250 0x7feb940cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 --2- 172.21.15.82:0/1864560357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb9415c1c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 -- 172.21.15.82:0/1864560357 >> 172.21.15.82:0/1864560357 conn(0x7feb940d1bd0 msgr2=0x7feb940d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 -- 172.21.15.82:0/1864560357 shutdown_connections 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.433+0000 7feb9b913640 1 -- 172.21.15.82:0/1864560357 wait complete. 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.434+0000 7feb9b913640 1 Processor -- start 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.434+0000 7feb9b913640 1 -- start start 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb9b913640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb940cd250 0x7feb941589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb9b913640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb940d01b0 0x7feb941f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb9b913640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb940c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb9b913640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940cf420 con 0x7feb9415bde0 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb9b913640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940cec20 con 0x7feb940d01b0 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb9b913640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb940d0620 con 0x7feb940cd250 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb99eb0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb940c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb99eb0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb940c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50696/0 (socket says 172.21.15.82:50696) 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.435+0000 7feb99eb0640 1 -- 172.21.15.82:0/2129042424 learned_addr learned my addr 172.21.15.82:0/2129042424 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb99eb0640 1 -- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb940cd250 msgr2=0x7feb941589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb99eb0640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb940cd250 0x7feb941589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb98eae640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb940d01b0 0x7feb941f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb99eb0640 1 -- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb940d01b0 msgr2=0x7feb941f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb99eb0640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb940d01b0 0x7feb941f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb99eb0640 1 -- 172.21.15.82:0/2129042424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940c4730 con 0x7feb9415bde0 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb98eae640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb940d01b0 0x7feb941f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:40.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb99eb0640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb940c5b40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00caf0 tx=0x7feb8c0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb89ffb640 1 -- 172.21.15.82:0/2129042424 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c018070 con 0x7feb9415bde0 2024-03-14T12:16:40.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb89ffb640 1 -- 172.21.15.82:0/2129042424 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8c00a760 con 0x7feb9415bde0 2024-03-14T12:16:40.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb940b4e40 con 0x7feb9415bde0 2024-03-14T12:16:40.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb89ffb640 1 -- 172.21.15.82:0/2129042424 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00ab90 con 0x7feb9415bde0 2024-03-14T12:16:40.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.436+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb940b5400 con 0x7feb9415bde0 2024-03-14T12:16:40.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.437+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9404f050 con 0x7feb9415bde0 2024-03-14T12:16:40.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.438+0000 7feb89ffb640 1 -- 172.21.15.82:0/2129042424 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7feb8c01b070 con 0x7feb9415bde0 2024-03-14T12:16:40.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.438+0000 7feb89ffb640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.438+0000 7feb89ffb640 1 -- 172.21.15.82:0/2129042424 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb8c08a1d0 con 0x7feb9415bde0 2024-03-14T12:16:40.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.438+0000 7feb996af640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.438+0000 7feb996af640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb90035000 tx=0x7feb9002d000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.442+0000 7feb89ffb640 1 -- 172.21.15.82:0/2129042424 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7feb8c058d50 con 0x7feb9415bde0 2024-03-14T12:16:40.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.613+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb940b7b50 con 0x7feb9415bde0 2024-03-14T12:16:40.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.614+0000 7feb89ffb640 1 -- 172.21.15.82:0/2129042424 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb8c058650 con 0x7feb9415bde0 2024-03-14T12:16:40.614 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:16:40.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 msgr2=0x7feb7806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb90035000 tx=0x7feb9002d000 comp rx=0 tx=0).stop 2024-03-14T12:16:40.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 msgr2=0x7feb940c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb940c5b40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00caf0 tx=0x7feb8c0020f0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 shutdown_connections 2024-03-14T12:16:40.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb9415bde0 0x7feb940c5b40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb940d01b0 0x7feb941f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 --2- 172.21.15.82:0/2129042424 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb940cd250 0x7feb941589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 >> 172.21.15.82:0/2129042424 conn(0x7feb940d1bd0 msgr2=0x7feb940cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:40.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 shutdown_connections 2024-03-14T12:16:40.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.616+0000 7feb9b913640 1 -- 172.21.15.82:0/2129042424 wait complete. 2024-03-14T12:16:40.627 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:16:40.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.729+0000 7f3223389640 1 Processor -- start 2024-03-14T12:16:40.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.730+0000 7f3223389640 1 -- start start 2024-03-14T12:16:40.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.730+0000 7f3223389640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 0x7f321c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.730+0000 7f3223389640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c0cf8a0 con 0x7f321c15c0e0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.730+0000 7f3223389640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c0cfb80 con 0x7f321c1608c0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.730+0000 7f3223389640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0d0ba0 con 0x7f321c160410 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3221125640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f321c15c0e0 0x7f321c0cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41158/0 (socket says 172.21.15.82:41158) 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3221125640 1 -- 172.21.15.82:0/3897312712 learned_addr learned my addr 172.21.15.82:0/3897312712 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253030718 0 0) 0x7f321c0cf8a0 con 0x7f321c15c0e0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3208003520 con 0x7f321c15c0e0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3221926640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 0x7f321c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3988564983 0 0) 0x7f3208003520 con 0x7f321c15c0e0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 msgr2=0x7f321c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 0x7f321c0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 >> v1:172.21.15.164:6789/0 conn(0x7f321c1608c0 legacy=0x7f321c15fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0cfe60 con 0x7f321c15c0e0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f3218001670 con 0x7f321c15c0e0 2024-03-14T12:16:40.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 >> v1:172.21.15.82:6789/0 conn(0x7f321c15c0e0 legacy=0x7f321c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32080039e0 0x7f3208003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f3208004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 0x7f32080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0d0ba0 con 0x7f3208004800 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320800baa0 con 0x7f32080039e0 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320800bca0 con 0x7f321c160410 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3220924640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f3208004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3221125640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32080039e0 0x7f3208003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.731+0000 7f3221926640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 0x7f32080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3220924640 1 -- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 msgr2=0x7f32080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3220924640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c160410 0x7f32080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3220924640 1 -- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32080039e0 msgr2=0x7f3208003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3220924640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32080039e0 0x7f3208003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3220924640 1 -- 172.21.15.82:0/3897312712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208003520 con 0x7f3208004800 2024-03-14T12:16:40.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3220924640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f3208004c80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f320c001720 tx=0x7f3208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da303e1d41be7d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c001370 con 0x7f3208004800 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c0052d0 con 0x7f3208004800 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.732+0000 7f3213fff640 1 -- 172.21.15.82:0/3897312712 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c003830 con 0x7f3208004800 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.733+0000 7f3223389640 1 -- 172.21.15.82:0/3897312712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 msgr2=0x7f3208004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.733+0000 7f3223389640 1 --2- 172.21.15.82:0/3897312712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f3208004c80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f320c001720 tx=0x7f3208005830 comp rx=0 tx=0).stop 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.733+0000 7f3221926640 1 -- 172.21.15.82:0/3897312712 reap_dead start 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.733+0000 7f3223389640 1 -- 172.21.15.82:0/3897312712 shutdown_connections 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.733+0000 7f3223389640 1 -- 172.21.15.82:0/3897312712 >> 172.21.15.82:0/3897312712 conn(0x7f321c0b3680 msgr2=0x7f321c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.733+0000 7f3223389640 1 -- 172.21.15.82:0/3897312712 shutdown_connections 2024-03-14T12:16:40.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.733+0000 7f3223389640 1 -- 172.21.15.82:0/3897312712 wait complete. 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.734+0000 7f3223389640 1 Processor -- start 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.734+0000 7f3223389640 1 -- start start 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3223389640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f321c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3223389640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c0ca870 0x7f321c1f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3223389640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321c0cacf0 0x7f321c1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3223389640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0cd960 con 0x7f3208004800 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3223389640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0d01d0 con 0x7f321c0cacf0 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3223389640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0d0660 con 0x7f321c0ca870 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f321c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f321c0b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50710/0 (socket says 172.21.15.82:50710) 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 -- 172.21.15.82:0/1958635075 learned_addr learned my addr 172.21.15.82:0/1958635075 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221926640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321c0cacf0 0x7f321c1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 -- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c0ca870 msgr2=0x7f321c1f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3220924640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c0ca870 0x7f321c1f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c0ca870 0x7f321c1f5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 -- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321c0cacf0 msgr2=0x7f321c1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321c0cacf0 0x7f321c1f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221125640 1 -- 172.21.15.82:0/1958635075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c149d00 con 0x7f3208004800 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.735+0000 7f3221926640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321c0cacf0 0x7f321c1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.736+0000 7f3221125640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f321c0b49c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f32180072f0 tx=0x7f32180077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.736+0000 7f3211ffb640 1 -- 172.21.15.82:0/1958635075 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218001660 con 0x7f3208004800 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.736+0000 7f3211ffb640 1 -- 172.21.15.82:0/1958635075 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218018070 con 0x7f3208004800 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.736+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c0cb210 con 0x7f3208004800 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.736+0000 7f3211ffb640 1 -- 172.21.15.82:0/1958635075 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321800fb80 con 0x7f3208004800 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.736+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c0cbab0 con 0x7f3208004800 2024-03-14T12:16:40.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.737+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c149ed0 con 0x7f3208004800 2024-03-14T12:16:40.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.737+0000 7f3211ffb640 1 -- 172.21.15.82:0/1958635075 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f321800fd20 con 0x7f3208004800 2024-03-14T12:16:40.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.737+0000 7f3211ffb640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f320006bd70 0x7f320006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:40.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.738+0000 7f3211ffb640 1 -- 172.21.15.82:0/1958635075 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f321808aaa0 con 0x7f3208004800 2024-03-14T12:16:40.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.738+0000 7f3220924640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f320006bd70 0x7f320006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:40.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.738+0000 7f3220924640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f320006bd70 0x7f320006e230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f320c033000 tx=0x7f320c009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:40.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.742+0000 7f3211ffb640 1 -- 172.21.15.82:0/1958635075 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3218059590 con 0x7f3208004800 2024-03-14T12:16:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.917+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f321c04f050 con 0x7f3208004800 2024-03-14T12:16:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.918+0000 7f3211ffb640 1 -- 172.21.15.82:0/1958635075 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3218058e90 con 0x7f3208004800 2024-03-14T12:16:40.918 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:16:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f320006bd70 msgr2=0x7f320006e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f320006bd70 0x7f320006e230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f320c033000 tx=0x7f320c009030 comp rx=0 tx=0).stop 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 msgr2=0x7f321c0b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f321c0b49c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f32180072f0 tx=0x7f32180077b0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 shutdown_connections 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321c0cacf0 0x7f321c1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f320006bd70 0x7f320006e230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f321c0ca870 0x7f321c1f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 --2- 172.21.15.82:0/1958635075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3208004800 0x7f321c0b49c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 >> 172.21.15.82:0/1958635075 conn(0x7f321c0b3680 msgr2=0x7f321c0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 shutdown_connections 2024-03-14T12:16:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:40.920+0000 7f3223389640 1 -- 172.21.15.82:0/1958635075 wait complete. 2024-03-14T12:16:40.931 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:16:41.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.032+0000 7f9799659640 1 Processor -- start 2024-03-14T12:16:41.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.032+0000 7f9799659640 1 -- start start 2024-03-14T12:16:41.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9799659640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f97940d0760 0x7f97940d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9799659640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f97940b7720 0x7f97940b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9799659640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97940d1080 con 0x7f97940cf370 2024-03-14T12:16:41.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9799659640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97940cf8c0 con 0x7f97940b7720 2024-03-14T12:16:41.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9799659640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97940cfba0 con 0x7f97940d0760 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f97937fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f97940b7720 0x7f97940b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f97927fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f97940cf370 0x7f97940d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41162/0 (socket says 172.21.15.82:41162) 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f97927fc640 1 -- 172.21.15.82:0/276101978 learned_addr learned my addr 172.21.15.82:0/276101978 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f97937fe640 1 -- 172.21.15.82:0/276101978 >> v2:172.21.15.164:3300/0 conn(0x7f97940d0760 msgr2=0x7f97940d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9792ffd640 1 --2- 172.21.15.82:0/276101978 >> v2:172.21.15.164:3300/0 conn(0x7f97940d0760 0x7f97940d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9791ffb640 1 -- 172.21.15.82:0/276101978 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158291185 0 0) 0x7f97940d1080 con 0x7f97940cf370 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f97937fe640 1 --2- 172.21.15.82:0/276101978 >> v2:172.21.15.164:3300/0 conn(0x7f97940d0760 0x7f97940d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f97937fe640 1 -- 172.21.15.82:0/276101978 >> v1:172.21.15.82:6789/0 conn(0x7f97940cf370 legacy=0x7f97940d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f97937fe640 1 -- 172.21.15.82:0/276101978 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940b6c40 con 0x7f97940b7720 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.033+0000 7f9792ffd640 1 --2- 172.21.15.82:0/276101978 >> v2:172.21.15.164:3300/0 conn(0x7f97940d0760 0x7f97940d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.034+0000 7f97937fe640 1 --2- 172.21.15.82:0/276101978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940b7720 0x7f97940b7b50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f979408ca90 tx=0x7f978c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b97816da128fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.034+0000 7f9791ffb640 1 -- 172.21.15.82:0/276101978 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c039070 con 0x7f97940b7720 2024-03-14T12:16:41.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.034+0000 7f9791ffb640 1 -- 172.21.15.82:0/276101978 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c02cd00 con 0x7f97940b7720 2024-03-14T12:16:41.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.034+0000 7f9791ffb640 1 -- 172.21.15.82:0/276101978 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c034660 con 0x7f97940b7720 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 -- 172.21.15.82:0/276101978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940b7720 msgr2=0x7f97940b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 --2- 172.21.15.82:0/276101978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940b7720 0x7f97940b7b50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f979408ca90 tx=0x7f978c02bf50 comp rx=0 tx=0).stop 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 -- 172.21.15.82:0/276101978 shutdown_connections 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 --2- 172.21.15.82:0/276101978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940b7720 0x7f97940b7b50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 --2- 172.21.15.82:0/276101978 >> v2:172.21.15.164:3300/0 conn(0x7f97940d0760 0x7f97940d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 -- 172.21.15.82:0/276101978 >> 172.21.15.82:0/276101978 conn(0x7f979409d030 msgr2=0x7f97940d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 -- 172.21.15.82:0/276101978 shutdown_connections 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.035+0000 7f9799659640 1 -- 172.21.15.82:0/276101978 wait complete. 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.036+0000 7f9799659640 1 Processor -- start 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.036+0000 7f9799659640 1 -- start start 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97940b7720 0x7f97940cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f97940cf370 0x7f979415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940d0760 0x7f9794146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97941617a0 con 0x7f97940d0760 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97940b7200 con 0x7f97940b7720 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9794161920 con 0x7f97940cf370 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940d0760 0x7f9794146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940d0760 0x7f9794146f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50726/0 (socket says 172.21.15.82:50726) 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 -- 172.21.15.82:0/1147111321 learned_addr learned my addr 172.21.15.82:0/1147111321 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9792ffd640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97940b7720 0x7f97940cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97927fc640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f97940cf370 0x7f979415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 -- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f97940cf370 msgr2=0x7f979415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f97940cf370 0x7f979415cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 -- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97940b7720 msgr2=0x7f97940cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97940b7720 0x7f97940cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 -- 172.21.15.82:0/1147111321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97941547e0 con 0x7f97940d0760 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f97937fe640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940d0760 0x7f9794146f10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f978c009e80 tx=0x7f978c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f977b7fe640 1 -- 172.21.15.82:0/1147111321 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c039050 con 0x7f97940d0760 2024-03-14T12:16:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f977b7fe640 1 -- 172.21.15.82:0/1147111321 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c045070 con 0x7f97940d0760 2024-03-14T12:16:41.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9794147af0 con 0x7f97940d0760 2024-03-14T12:16:41.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f977b7fe640 1 -- 172.21.15.82:0/1147111321 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c03e830 con 0x7f97940d0760 2024-03-14T12:16:41.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.037+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97940cb1a0 con 0x7f97940d0760 2024-03-14T12:16:41.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.038+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979404f050 con 0x7f97940d0760 2024-03-14T12:16:41.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.039+0000 7f977b7fe640 1 -- 172.21.15.82:0/1147111321 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f978c03e9d0 con 0x7f97940d0760 2024-03-14T12:16:41.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.039+0000 7f977b7fe640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f978006bbd0 0x7f978006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.039+0000 7f977b7fe640 1 -- 172.21.15.82:0/1147111321 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f978c0ad9e0 con 0x7f97940d0760 2024-03-14T12:16:41.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.039+0000 7f9792ffd640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f978006bbd0 0x7f978006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.040+0000 7f9792ffd640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f978006bbd0 0x7f978006e090 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f978400a950 tx=0x7f9784008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.044+0000 7f977b7fe640 1 -- 172.21.15.82:0/1147111321 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f97941547e0 con 0x7f97940d0760 2024-03-14T12:16:41.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.228+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f97940d0540 con 0x7f97940d0760 2024-03-14T12:16:41.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.228+0000 7f977b7fe640 1 -- 172.21.15.82:0/1147111321 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f978c07c5f0 con 0x7f97940d0760 2024-03-14T12:16:41.229 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f978006bbd0 msgr2=0x7f978006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f978006bbd0 0x7f978006e090 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f978400a950 tx=0x7f9784008040 comp rx=0 tx=0).stop 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940d0760 msgr2=0x7f9794146f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940d0760 0x7f9794146f10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f978c009e80 tx=0x7f978c004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 shutdown_connections 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97940d0760 0x7f9794146f10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f978006bbd0 0x7f978006e090 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f97940cf370 0x7f979415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 --2- 172.21.15.82:0/1147111321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97940b7720 0x7f97940cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 >> 172.21.15.82:0/1147111321 conn(0x7f979409d030 msgr2=0x7f97940d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 shutdown_connections 2024-03-14T12:16:41.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.230+0000 7f9799659640 1 -- 172.21.15.82:0/1147111321 wait complete. 2024-03-14T12:16:41.241 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T12:16:41.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:41.242 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T12:16:41.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:16:41.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:41.271 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T12:16:41.326 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:16:41.326 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 counter dump 2024-03-14T12:16:41.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.467+0000 7fbd4f861640 1 Processor -- start 2024-03-14T12:16:41.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.468+0000 7fbd4f861640 1 -- start start 2024-03-14T12:16:41.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.468+0000 7fbd4f861640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbd480b08c0 0x7fbd4806c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.468+0000 7fbd4f861640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 0x7fbd48165a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.468+0000 7fbd4f861640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4815c270 con 0x7fbd480afdd0 2024-03-14T12:16:41.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.468+0000 7fbd4f861640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd48066de0 con 0x7fbd480b08c0 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.468+0000 7fbd4f861640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480af380 con 0x7fbd480cc670 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4d5fd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbd480afdd0 0x7fbd480b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51638/0 (socket says 172.21.15.82:51638) 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/2453124298 learned_addr learned my addr 172.21.15.82:0/2453124298 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389614313 0 0) 0x7fbd4815c270 con 0x7fbd480afdd0 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd34003520 con 0x7fbd480afdd0 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 0x7fbd48165a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/2453124298 >> v2:172.21.15.164:3300/0 conn(0x7fbd480b08c0 0x7fbd4806c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213251307 0 0) 0x7fbd34003520 con 0x7fbd480afdd0 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 msgr2=0x7fbd48165a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 0x7fbd48165a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 >> v2:172.21.15.164:3300/0 conn(0x7fbd480b08c0 msgr2=0x7fbd4806c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 --2- 172.21.15.82:0/2453124298 >> v2:172.21.15.164:3300/0 conn(0x7fbd480b08c0 0x7fbd4806c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480af660 con 0x7fbd480afdd0 2024-03-14T12:16:41.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fbd440019f0 con 0x7fbd480afdd0 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 0x7fbd48165a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 >> v1:172.21.15.82:6789/0 conn(0x7fbd480afdd0 legacy=0x7fbd480b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 0x7fbd34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd34004d40 0x7fbd340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48066de0 con 0x7fbd340039b0 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480af380 con 0x7fbd34004d40 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3400ba70 con 0x7fbd480cc670 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 0x7fbd34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.469+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd34004d40 0x7fbd340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 msgr2=0x7fbd34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd480cc670 0x7fbd34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd34004d40 msgr2=0x7fbd340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd34004d40 0x7fbd340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/2453124298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34003520 con 0x7fbd340039b0 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd34004d40 0x7fbd340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd34003dd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400a920 tx=0x7fbd34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbec7c64d60b518 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400a4c0 con 0x7fbd340039b0 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44034650 con 0x7fbd340039b0 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.470+0000 7fbd3ffff640 1 -- 172.21.15.82:0/2453124298 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44034a40 con 0x7fbd340039b0 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.471+0000 7fbd4f861640 1 -- 172.21.15.82:0/2453124298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 msgr2=0x7fbd34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.471+0000 7fbd4f861640 1 --2- 172.21.15.82:0/2453124298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd34003dd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400a920 tx=0x7fbd34005830 comp rx=0 tx=0).stop 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.471+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/2453124298 reap_dead start 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.471+0000 7fbd4f861640 1 -- 172.21.15.82:0/2453124298 shutdown_connections 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.471+0000 7fbd4f861640 1 -- 172.21.15.82:0/2453124298 >> 172.21.15.82:0/2453124298 conn(0x7fbd480b4e50 msgr2=0x7fbd480b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.471+0000 7fbd4f861640 1 -- 172.21.15.82:0/2453124298 shutdown_connections 2024-03-14T12:16:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.471+0000 7fbd4f861640 1 -- 172.21.15.82:0/2453124298 wait complete. 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.472+0000 7fbd4f861640 1 Processor -- start 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.472+0000 7fbd4f861640 1 -- start start 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4f861640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd48154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4f861640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4815b910 0x7fbd4814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4f861640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd48159f20 0x7fbd4809c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4f861640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480b2150 con 0x7fbd340039b0 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4f861640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480b1fd0 con 0x7fbd4815b910 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4f861640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480b22d0 con 0x7fbd48159f20 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd48154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd48154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54006/0 (socket says 172.21.15.82:54006) 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/88310480 learned_addr learned my addr 172.21.15.82:0/88310480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd48159f20 0x7fbd4809c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4815b910 0x7fbd4814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd48159f20 msgr2=0x7fbd4809c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd48159f20 0x7fbd4809c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4815b910 msgr2=0x7fbd4814e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4815b910 0x7fbd4814e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/88310480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4809b790 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4815b910 0x7fbd4814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd48154c10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbd4402ccb0 tx=0x7fbd480af830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.473+0000 7fbd3dffb640 1 -- 172.21.15.82:0/88310480 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44045070 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.474+0000 7fbd3dffb640 1 -- 172.21.15.82:0/88310480 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4403a040 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.474+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4814e9d0 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.474+0000 7fbd3dffb640 1 -- 172.21.15.82:0/88310480 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44034800 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.474+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd48159720 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.475+0000 7fbd3dffb640 1 -- 172.21.15.82:0/88310480 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbd44048070 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.475+0000 7fbd3dffb640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd2c06bbd0 0x7fbd2c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.476+0000 7fbd3dffb640 1 -- 172.21.15.82:0/88310480 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd440ac510 con 0x7fbd340039b0 2024-03-14T12:16:41.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.476+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd2c06bbd0 0x7fbd2c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.476+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd2c06bbd0 0x7fbd2c06e090 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbd4815a990 tx=0x7fbd38008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.476+0000 7fbd52444740 1 Processor -- start 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.476+0000 7fbd52444740 1 -- start start 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x56329412e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x56329412e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd52444740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294129210 0x563294158d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd52444740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632940f6360 con 0x563294129210 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563294092ec0 con 0x56329412cae0 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56329403bd30 con 0x563294176a60 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294129210 0x563294158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294129210 0x563294158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54010/0 (socket says 172.21.15.82:54010) 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/1486432823 learned_addr learned my addr 172.21.15.82:0/1486432823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x56329412e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x56329412e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 msgr2=0x56329412e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x56329412e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 msgr2=0x56329412e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x56329412e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.477+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/1486432823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56329403abc0 con 0x563294129210 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294129210 0x563294158d70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbd480cca90 tx=0x7fbd40010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7c2e187d7e479d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd32ffd640 1 -- 172.21.15.82:0/1486432823 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4001b070 con 0x563294129210 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd32ffd640 1 -- 172.21.15.82:0/1486432823 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40004220 con 0x563294129210 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd32ffd640 1 -- 172.21.15.82:0/1486432823 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000f040 con 0x563294129210 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 -- 172.21.15.82:0/1486432823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294129210 msgr2=0x563294158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294129210 0x563294158d70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbd480cca90 tx=0x7fbd40010a80 comp rx=0 tx=0).stop 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 -- 172.21.15.82:0/1486432823 shutdown_connections 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294129210 0x563294158d70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x56329412e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 --2- 172.21.15.82:0/1486432823 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x56329412e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 -- 172.21.15.82:0/1486432823 >> 172.21.15.82:0/1486432823 conn(0x56329412b480 msgr2=0x563294173d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 -- 172.21.15.82:0/1486432823 shutdown_connections 2024-03-14T12:16:41.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.478+0000 7fbd52444740 1 -- 172.21.15.82:0/1486432823 wait complete. 2024-03-14T12:16:41.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.479+0000 7fbd52444740 1 -- start start 2024-03-14T12:16:41.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.479+0000 7fbd52444740 1 Processor -- start 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd52444740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56329412cae0 0x5632941432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x563294178440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563294156c50 0x5632941579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd52444740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632940314a0 con 0x56329412cae0 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56329408ef60 con 0x563294156c50 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56329412cff0 con 0x563294176a60 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd52444740 10 client.0 _ll_register_callbacks cb 0x56329408cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56329412cae0 0x5632941432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56329412cae0 0x5632941432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54018/0 (socket says 172.21.15.82:54018) 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/3887177263 learned_addr learned my addr 172.21.15.82:0/3887177263 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:41.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x563294178440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563294156c50 0x5632941579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 msgr2=0x563294178440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x563294178440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563294156c50 msgr2=0x5632941579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563294156c50 0x5632941579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 -- 172.21.15.82:0/3887177263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563293da73f0 con 0x56329412cae0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x563294178440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56329412cae0 0x5632941432f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbd4407a800 tx=0x7fbd4407b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67115033a70d3b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.480+0000 7fbd27fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd27fff640 1 -- 172.21.15.82:0/3887177263 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440aae60 con 0x56329412cae0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd27fff640 1 -- 172.21.15.82:0/3887177263 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd4407bba0 con 0x56329412cae0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd40004220 con 0x56329412cae0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd27fff640 1 -- 172.21.15.82:0/3887177263 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd44083010 con 0x56329412cae0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 10 client.6261 fetch_fsmap learned FSMap version 27 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x563293e717d0 con 0x56329412cae0 2024-03-14T12:16:41.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd27fff640 1 -- 172.21.15.82:0/3887177263 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40004220 con 0x56329412cae0 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd27fff640 1 -- 172.21.15.82:0/3887177263 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd440ac0b0 con 0x56329412cae0 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 10 client.6261 fetch_fsmap finished waiting for FSMap version 27 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 10 client.6261 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 20 client.6261 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x563294185060 0x563294185460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 4 client.6261 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 --> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632940a0400 con 0x563294185060 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x563294185060 0x563294185460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd27fff640 10 client.6261 ms_handle_connect on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.481+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x563294185060 0x563294185460 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cf87f48092d5fb server_cookie=16b7f35132af6304 in_seq=0 out_seq=0 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd27fff640 1 -- 172.21.15.82:0/3887177263 <== mds.0 v2:172.21.15.82:6836/1422543802 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632940a0400 con 0x563294185060 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd27fff640 10 client.6261 handle_command_reply: tid=0 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 client.6261 shutdown 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56329412cae0 msgr2=0x5632941432f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56329412cae0 0x5632941432f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbd4407a800 tx=0x7fbd4407b0a0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 shutdown_connections 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563294156c50 0x5632941579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294176a60 0x563294178440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x563294185060 0x563294185460 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 --2- 172.21.15.82:0/3887177263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56329412cae0 0x5632941432f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd27fff640 0 client.6261 ms_handle_reset on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 >> 172.21.15.82:0/3887177263 conn(0x563294129210 msgr2=0x56329412de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 shutdown_connections 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 1 -- 172.21.15.82:0/3887177263 wait complete. 2024-03-14T12:16:41.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.482+0000 7fbd52444740 20 client.6261 trim_cache size 0 max 16384 2024-03-14T12:16:41.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.494+0000 7fbd52444740 1 Processor -- start 2024-03-14T12:16:41.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.494+0000 7fbd52444740 1 -- start start 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.494+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x563294177e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.494+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x5632941784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.494+0000 7fbd52444740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd52444740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd481fe850 con 0x563294176a60 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd481fd9d0 con 0x56329412cae0 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd481fce20 con 0x563294156c50 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54022/0 (socket says 172.21.15.82:54022) 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/367345435 learned_addr learned my addr 172.21.15.82:0/367345435 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x5632941784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x563294177e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 msgr2=0x5632941784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x5632941784a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 msgr2=0x563294177e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x563294177e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/367345435 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563294173570 con 0x563294176a60 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412eba0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006fd0 tx=0x7fbd40018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efb5b875480962a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd327fc640 1 -- 172.21.15.82:0/367345435 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000f040 con 0x563294176a60 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd327fc640 1 -- 172.21.15.82:0/367345435 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40025070 con 0x563294176a60 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.495+0000 7fbd327fc640 1 -- 172.21.15.82:0/367345435 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4001b040 con 0x563294176a60 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 -- 172.21.15.82:0/367345435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 msgr2=0x56329412eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412eba0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006fd0 tx=0x7fbd40018000 comp rx=0 tx=0).stop 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 -- 172.21.15.82:0/367345435 shutdown_connections 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412eba0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x5632941784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 --2- 172.21.15.82:0/367345435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x563294177e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 -- 172.21.15.82:0/367345435 >> 172.21.15.82:0/367345435 conn(0x563294129210 msgr2=0x563294172b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 -- 172.21.15.82:0/367345435 shutdown_connections 2024-03-14T12:16:41.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 -- 172.21.15.82:0/367345435 wait complete. 2024-03-14T12:16:41.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 -- start start 2024-03-14T12:16:41.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.496+0000 7fbd52444740 1 Processor -- start 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x5632941c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd52444740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x56329412cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd52444740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd52444740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd481fd3d0 con 0x563294176a60 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632941733a0 con 0x56329412cae0 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd52444740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56329415c120 con 0x563294156c50 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd52444740 10 client.0 _ll_register_callbacks cb 0x7fbd40010210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd4ddfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd4ddfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54028/0 (socket says 172.21.15.82:54028) 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.497+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/4277836886 learned_addr learned my addr 172.21.15.82:0/4277836886 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4cdfc640 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x56329412cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x5632941c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 msgr2=0x56329412cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x56329412cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 msgr2=0x5632941c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x5632941c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4ddfe640 1 -- 172.21.15.82:0/4277836886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632941e7f90 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd4ddfe640 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412d400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbd400105c0 tx=0x7fbd400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a842c603cea3c14b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd27fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd27fff640 1 -- 172.21.15.82:0/4277836886 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4002d030 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd27fff640 1 -- 172.21.15.82:0/4277836886 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd40012950 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd40025070 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd27fff640 1 -- 172.21.15.82:0/4277836886 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd4001b040 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 10 client.6267 fetch_fsmap learned FSMap version 27 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x56329418ed30 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd27fff640 1 -- 172.21.15.82:0/4277836886 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40025070 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd27fff640 1 -- 172.21.15.82:0/4277836886 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd40012b50 con 0x563294176a60 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 10 client.6267 fetch_fsmap finished waiting for FSMap version 27 2024-03-14T12:16:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 10 client.6267 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 20 client.6267 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x56329412e610 0x56329418de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 4 client.6267 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.498+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 --> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x56329409a750 con 0x56329412e610 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.499+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x56329412e610 0x56329418de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.499+0000 7fbd27fff640 10 client.6267 ms_handle_connect on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.499+0000 7fbd4d5fd640 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x56329412e610 0x56329418de80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a9f4b1475a033b server_cookie=9a30e8a61932dd56 in_seq=0 out_seq=0 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.499+0000 7fbd27fff640 1 -- 172.21.15.82:0/4277836886 <== mds.0 v2:172.21.15.82:6836/1422543802 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26753 (crc 0 0 0) 0x56329409a750 con 0x56329412e610 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.499+0000 7fbd27fff640 10 client.6267 handle_command_reply: tid=0 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.499+0000 7fbd52444740 1 client.6267 shutdown 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 msgr2=0x56329412d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412d400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbd400105c0 tx=0x7fbd400046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 shutdown_connections 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x56329412e610 0x56329418de80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563294176a60 0x56329412d400 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563294156c50 0x56329412cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd27fff640 0 client.6267 ms_handle_reset on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 --2- 172.21.15.82:0/4277836886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56329412cae0 0x5632941c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 >> 172.21.15.82:0/4277836886 conn(0x563294129210 msgr2=0x56329412eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 shutdown_connections 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 1 -- 172.21.15.82:0/4277836886 wait complete. 2024-03-14T12:16:41.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd52444740 20 client.6267 trim_cache size 0 max 16384 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 59, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 56, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 15589, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 64813, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 11, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.016344358, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001362194, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.013462353, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000498599, 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 55, 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.196418646, 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003571248 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 60, 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000008246, 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000137 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 68, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 64, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 114708, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 21651, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008389368, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001776939, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.011564802, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000374329, 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 63, 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.070789889, 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001123649 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8, 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000774, 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000096 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 110944, 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 10992 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:16:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 14, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3603, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 97100, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.003947764, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000589343, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002986632, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000289688, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 18, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.116770253, 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.006487236 2024-03-14T12:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 90560 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 48, 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003760639, 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000078346 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000018003, 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000009001 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 28, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 28, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 28, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003572602, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000127592 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:16:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 10, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 55, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 55, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:16:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:16:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 1 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:16:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "6048" 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 2 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.5198", 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 5, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 18, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:16:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.6210", 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 0, 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:16:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:16:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 7, 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 7, 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:16:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 2, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4196884, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.007709456, 2024-03-14T12:16:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001927364 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:16:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 2, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 2, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 42468, 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 28, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 28, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 4, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 27, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003310085, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000122595 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:16:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:16:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000262517, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000262517 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:16:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 27, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 13, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 0, 2024-03-14T12:16:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:16:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:16:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:16:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 212839, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 57, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:16:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 51472, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 205, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 42, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 13700, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 42, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 42, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.873983758, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.020809137 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 42, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 66 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 42, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 42, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 4, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 7, 2024-03-14T12:16:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:16:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:16:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 58, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 6, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 6, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 15, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-03-14T12:16:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:16:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 136, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 117429, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 136, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 136, 2024-03-14T12:16:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 117429, 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 42, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 13700, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 42, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 13700, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 42, 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 42, 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 42, 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 42, 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x5581581aa6e0": [ 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 7, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2580, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 7, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 4, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2580, 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:16:41.565 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:41.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd2c06bbd0 msgr2=0x7fbd2c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd2c06bbd0 0x7fbd2c06e090 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbd4815a990 tx=0x7fbd38008040 comp rx=0 tx=0).stop 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 msgr2=0x7fbd48154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd48154c10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbd4402ccb0 tx=0x7fbd480af830 comp rx=0 tx=0).stop 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 shutdown_connections 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd2c06bbd0 0x7fbd2c06e090 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd48159f20 0x7fbd4809c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4815b910 0x7fbd4814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 --2- 172.21.15.82:0/88310480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd340039b0 0x7fbd48154c10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.500+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 >> 172.21.15.82:0/88310480 conn(0x7fbd480b4e50 msgr2=0x7fbd480ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.501+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 shutdown_connections 2024-03-14T12:16:41.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:41.501+0000 7fbd4f861640 1 -- 172.21.15.82:0/88310480 wait complete. 2024-03-14T12:16:42.569 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:16:42.569 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 counter dump 2024-03-14T12:16:42.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.683+0000 7f5a4d044640 1 Processor -- start 2024-03-14T12:16:42.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.683+0000 7f5a4d044640 1 -- start start 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4d044640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5a480cd270 0x7f5a480d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4d044640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480d26e0 0x7f5a480d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4d044640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a480cf830 con 0x7f5a4815bed0 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4d044640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a480cd7f0 con 0x7f5a480cd270 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4d044640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480cfad0 con 0x7f5a480d26e0 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5a480cd270 0x7f5a480d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5a480cd270 0x7f5a480d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54034/0 (socket says 172.21.15.82:54034) 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.684+0000 7f5a4659c640 1 -- 172.21.15.82:0/3762781044 learned_addr learned my addr 172.21.15.82:0/3762781044 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.685+0000 7f5a4759e640 1 --2- 172.21.15.82:0/3762781044 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480d26e0 0x7f5a480d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.685+0000 7f5a4659c640 1 -- 172.21.15.82:0/3762781044 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480d26e0 msgr2=0x7f5a480d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.685+0000 7f5a4659c640 1 --2- 172.21.15.82:0/3762781044 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480d26e0 0x7f5a480d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.685+0000 7f5a4659c640 1 -- 172.21.15.82:0/3762781044 >> v1:172.21.15.164:6789/0 conn(0x7f5a4815bed0 legacy=0x7f5a480c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.685+0000 7f5a4659c640 1 -- 172.21.15.82:0/3762781044 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480cfdb0 con 0x7f5a480cd270 2024-03-14T12:16:42.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.685+0000 7f5a4659c640 1 --2- 172.21.15.82:0/3762781044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480cd270 0x7f5a480d02d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009870 tx=0x7f5a3c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58df24366009bd03 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:42.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a45d9b640 1 -- 172.21.15.82:0/3762781044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c0317c0 con 0x7f5a480cd270 2024-03-14T12:16:42.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a45d9b640 1 -- 172.21.15.82:0/3762781044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c037070 con 0x7f5a480cd270 2024-03-14T12:16:42.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a45d9b640 1 -- 172.21.15.82:0/3762781044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c031b80 con 0x7f5a480cd270 2024-03-14T12:16:42.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 -- 172.21.15.82:0/3762781044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480cd270 msgr2=0x7f5a480d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3762781044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480cd270 0x7f5a480d02d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009870 tx=0x7f5a3c02b860 comp rx=0 tx=0).stop 2024-03-14T12:16:42.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 -- 172.21.15.82:0/3762781044 shutdown_connections 2024-03-14T12:16:42.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3762781044 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480d26e0 0x7f5a480d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3762781044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480cd270 0x7f5a480d02d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 -- 172.21.15.82:0/3762781044 >> 172.21.15.82:0/3762781044 conn(0x7f5a480d1bc0 msgr2=0x7f5a480d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 -- 172.21.15.82:0/3762781044 shutdown_connections 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.686+0000 7f5a4d044640 1 -- 172.21.15.82:0/3762781044 wait complete. 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.688+0000 7f5a4d044640 1 Processor -- start 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.688+0000 7f5a4d044640 1 -- start start 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.688+0000 7f5a4d044640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480cd270 0x7f5a481f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.688+0000 7f5a4d044640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480d26e0 0x7f5a481f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4d044640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4815bed0 0x7f5a480b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4d044640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815c2b0 con 0x7f5a480d26e0 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4d044640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4806b770 con 0x7f5a4815bed0 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4d044640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480c5e20 con 0x7f5a480cd270 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480d26e0 0x7f5a481f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480d26e0 0x7f5a481f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54046/0 (socket says 172.21.15.82:54046) 2024-03-14T12:16:42.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 -- 172.21.15.82:0/3890695431 learned_addr learned my addr 172.21.15.82:0/3890695431 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480cd270 0x7f5a481f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4759e640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4815bed0 0x7f5a480b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 -- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480cd270 msgr2=0x7f5a481f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480cd270 0x7f5a481f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 -- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4815bed0 msgr2=0x7f5a480b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4815bed0 0x7f5a480b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 -- 172.21.15.82:0/3890695431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a481f2510 con 0x7f5a480d26e0 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4659c640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480d26e0 0x7f5a481f3100 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:42.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a2f7fe640 1 -- 172.21.15.82:0/3890695431 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c032650 con 0x7f5a480d26e0 2024-03-14T12:16:42.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a2f7fe640 1 -- 172.21.15.82:0/3890695431 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c0327f0 con 0x7f5a480d26e0 2024-03-14T12:16:42.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a480b6630 con 0x7f5a480d26e0 2024-03-14T12:16:42.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a2f7fe640 1 -- 172.21.15.82:0/3890695431 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c032be0 con 0x7f5a480d26e0 2024-03-14T12:16:42.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.689+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a480b6bf0 con 0x7f5a480d26e0 2024-03-14T12:16:42.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.691+0000 7f5a2f7fe640 1 -- 172.21.15.82:0/3890695431 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5a3c04c060 con 0x7f5a480d26e0 2024-03-14T12:16:42.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.692+0000 7f5a2f7fe640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5a3406bbd0 0x7f5a3406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.692+0000 7f5a2f7fe640 1 -- 172.21.15.82:0/3890695431 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a3c0ac3e0 con 0x7f5a480d26e0 2024-03-14T12:16:42.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.692+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5a3406bbd0 0x7f5a3406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.692+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5a3406bbd0 0x7f5a3406e090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800b450 tx=0x7f5a3800d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:42.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.693+0000 7f5a4fc25740 1 Processor -- start 2024-03-14T12:16:42.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.693+0000 7f5a4fc25740 1 -- start start 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.693+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee104ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104c9210 0x55ee104f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee10496360 con 0x55ee104ccae0 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee10432ec0 con 0x55ee104c9210 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee103dbd30 con 0x55ee10516a60 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54050/0 (socket says 172.21.15.82:54050) 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 -- 172.21.15.82:0/2994393328 learned_addr learned my addr 172.21.15.82:0/2994393328 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee104ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 -- 172.21.15.82:0/2994393328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 msgr2=0x55ee104ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee104ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 -- 172.21.15.82:0/2994393328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104c9210 msgr2=0x55ee104f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104c9210 0x55ee104f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 -- 172.21.15.82:0/2994393328 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee103dabc0 con 0x55ee104ccae0 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a4659c640 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce5d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0aa9c0 tx=0x7f5a3c07a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930d1f0a38afbaeb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a2cff9640 1 -- 172.21.15.82:0/2994393328 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c083090 con 0x55ee104ccae0 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a2cff9640 1 -- 172.21.15.82:0/2994393328 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c037070 con 0x55ee104ccae0 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.694+0000 7f5a2cff9640 1 -- 172.21.15.82:0/2994393328 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c032d80 con 0x55ee104ccae0 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2994393328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 msgr2=0x55ee104ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce5d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0aa9c0 tx=0x7f5a3c07a450 comp rx=0 tx=0).stop 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2994393328 shutdown_connections 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104c9210 0x55ee104f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce5d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2994393328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee104ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2994393328 >> 172.21.15.82:0/2994393328 conn(0x55ee104cb480 msgr2=0x55ee10513d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2994393328 shutdown_connections 2024-03-14T12:16:42.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.695+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2994393328 wait complete. 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 -- start start 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 Processor -- start 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104ccae0 0x55ee104f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee1051a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10518e00 0x55ee104f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee103d14a0 con 0x55ee10518e00 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1042ef60 con 0x55ee104ccae0 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.696+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee103dbd30 con 0x55ee10516a60 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4fc25740 10 client.0 _ll_register_callbacks cb 0x55ee1042cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10518e00 0x55ee104f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10518e00 0x55ee104f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54060/0 (socket says 172.21.15.82:54060) 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 -- 172.21.15.82:0/1870215961 learned_addr learned my addr 172.21.15.82:0/1870215961 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:42.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4659c640 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee1051a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104ccae0 0x55ee104f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 -- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 msgr2=0x55ee1051a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee1051a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 -- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104ccae0 msgr2=0x55ee104f04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104ccae0 0x55ee104f04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 -- 172.21.15.82:0/1870215961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee101473f0 con 0x55ee10518e00 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4759e640 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10518e00 0x55ee104f5800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5a40010530 tx=0x7f5a400109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9e74f28fa698b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a25ffb640 1 -- 172.21.15.82:0/1870215961 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40018070 con 0x55ee10518e00 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a25ffb640 1 -- 172.21.15.82:0/1870215961 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a40004020 con 0x55ee10518e00 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a3c037070 con 0x55ee10518e00 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a25ffb640 1 -- 172.21.15.82:0/1870215961 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a40022050 con 0x55ee10518e00 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4fc25740 10 client.6279 fetch_fsmap learned FSMap version 27 2024-03-14T12:16:42.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55ee102117d0 con 0x55ee10518e00 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.697+0000 7f5a25ffb640 1 -- 172.21.15.82:0/1870215961 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c037070 con 0x55ee10518e00 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a25ffb640 1 -- 172.21.15.82:0/1870215961 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a40025020 con 0x55ee10518e00 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a4fc25740 10 client.6279 fetch_fsmap finished waiting for FSMap version 27 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a4fc25740 10 client.6279 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a4fc25740 20 client.6279 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee104f1480 0x55ee10527680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a4fc25740 4 client.6279 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 --> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee10440400 con 0x55ee104f1480 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee104f1480 0x55ee10527680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a25ffb640 10 client.6279 ms_handle_connect on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.698+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee104f1480 0x55ee10527680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166e1550d0be4a9 server_cookie=d5cf60b76c734759 in_seq=0 out_seq=0 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a25ffb640 1 -- 172.21.15.82:0/1870215961 <== mds.0 v2:172.21.15.82:6836/1422543802 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee10440400 con 0x55ee104f1480 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a25ffb640 10 client.6279 handle_command_reply: tid=0 2024-03-14T12:16:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 client.6279 shutdown 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10518e00 msgr2=0x55ee104f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10518e00 0x55ee104f5800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5a40010530 tx=0x7f5a400109f0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 shutdown_connections 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee104f1480 0x55ee10527680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10518e00 0x55ee104f5800 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10516a60 0x55ee1051a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/1870215961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee104ccae0 0x55ee104f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a25ffb640 0 client.6279 ms_handle_reset on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 >> 172.21.15.82:0/1870215961 conn(0x55ee104c9210 msgr2=0x55ee104cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 shutdown_connections 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 1 -- 172.21.15.82:0/1870215961 wait complete. 2024-03-14T12:16:42.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.699+0000 7f5a4fc25740 20 client.6279 trim_cache size 0 max 16384 2024-03-14T12:16:42.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 Processor -- start 2024-03-14T12:16:42.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 -- start start 2024-03-14T12:16:42.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10516a60 0x55ee104c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10518e00 0x55ee104cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481fe0f0 con 0x55ee104ccae0 2024-03-14T12:16:42.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815ed20 con 0x55ee10516a60 2024-03-14T12:16:42.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481f6550 con 0x55ee10518e00 2024-03-14T12:16:42.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a46d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a46d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54074/0 (socket says 172.21.15.82:54074) 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.711+0000 7f5a46d9d640 1 -- 172.21.15.82:0/960728120 learned_addr learned my addr 172.21.15.82:0/960728120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a4759e640 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10518e00 0x55ee104cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a4659c640 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10516a60 0x55ee104c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a46d9d640 1 -- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10518e00 msgr2=0x55ee104cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10518e00 0x55ee104cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a46d9d640 1 -- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10516a60 msgr2=0x55ee104c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10516a60 0x55ee104c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a46d9d640 1 -- 172.21.15.82:0/960728120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee104c4f60 con 0x55ee104ccae0 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a4659c640 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10516a60 0x55ee104c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce460 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800ab10 tx=0x7f5a38011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56926e4bf4c6255e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a27fff640 1 -- 172.21.15.82:0/960728120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38003b70 con 0x55ee104ccae0 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a27fff640 1 -- 172.21.15.82:0/960728120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38003d10 con 0x55ee104ccae0 2024-03-14T12:16:42.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.712+0000 7f5a27fff640 1 -- 172.21.15.82:0/960728120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38016b40 con 0x55ee104ccae0 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 -- 172.21.15.82:0/960728120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 msgr2=0x55ee104ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce460 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800ab10 tx=0x7f5a38011000 comp rx=0 tx=0).stop 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 -- 172.21.15.82:0/960728120 shutdown_connections 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee10518e00 0x55ee104cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10516a60 0x55ee104c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/960728120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee104ccae0 0x55ee104ce460 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 -- 172.21.15.82:0/960728120 >> 172.21.15.82:0/960728120 conn(0x55ee104c9210 msgr2=0x55ee104f6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 -- 172.21.15.82:0/960728120 shutdown_connections 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 -- 172.21.15.82:0/960728120 wait complete. 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 -- start start 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.713+0000 7f5a4fc25740 1 Processor -- start 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee104ccae0 0x55ee1057f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10516a60 0x55ee10523280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4fc25740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10518e00 0x55ee10509700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee10496360 con 0x55ee10516a60 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815e720 con 0x55ee10518e00 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4fc25740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee10432ec0 con 0x55ee104ccae0 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4fc25740 10 client.0 _ll_register_callbacks cb 0x7f5a3c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10516a60 0x55ee10523280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10516a60 0x55ee10523280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54088/0 (socket says 172.21.15.82:54088) 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 -- 172.21.15.82:0/2883576623 learned_addr learned my addr 172.21.15.82:0/2883576623 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee104ccae0 0x55ee1057f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4759e640 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10518e00 0x55ee10509700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 -- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee104ccae0 msgr2=0x55ee1057f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee104ccae0 0x55ee1057f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 -- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10518e00 msgr2=0x55ee10509700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10518e00 0x55ee10509700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.714+0000 7f5a4659c640 1 -- 172.21.15.82:0/2883576623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee105237c0 con 0x55ee10516a60 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4659c640 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10516a60 0x55ee10523280 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c02b8a0 tx=0x7f5a3c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3138edaf5c163905 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a25ffb640 1 -- 172.21.15.82:0/2883576623 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c08a060 con 0x55ee10516a60 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a25ffb640 1 -- 172.21.15.82:0/2883576623 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a3c0770a0 con 0x55ee10516a60 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a38003d10 con 0x55ee10516a60 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a25ffb640 1 -- 172.21.15.82:0/2883576623 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a3c07a800 con 0x55ee10516a60 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 10 client.6285 fetch_fsmap learned FSMap version 27 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55ee104f7800 con 0x55ee10516a60 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a25ffb640 1 -- 172.21.15.82:0/2883576623 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38003d10 con 0x55ee10516a60 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a25ffb640 1 -- 172.21.15.82:0/2883576623 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a3c082c00 con 0x55ee10516a60 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 10 client.6285 fetch_fsmap finished waiting for FSMap version 27 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 10 client.6285 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 20 client.6285 populate_metadata read hostname 'smithi082' 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee10509c40 0x55ee104f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 4 client.6285 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:16:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 --> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x55ee1043a750 con 0x55ee10509c40 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.715+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee10509c40 0x55ee104f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a25ffb640 10 client.6285 ms_handle_connect on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a46d9d640 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee10509c40 0x55ee104f5110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851d43e27434db0f server_cookie=a157d8be9753a77 in_seq=0 out_seq=0 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a25ffb640 1 -- 172.21.15.82:0/2883576623 <== mds.0 v2:172.21.15.82:6836/1422543802 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26000 (crc 0 0 0) 0x55ee1043a750 con 0x55ee10509c40 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a25ffb640 10 client.6285 handle_command_reply: tid=0 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 client.6285 shutdown 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10516a60 msgr2=0x55ee10523280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10516a60 0x55ee10523280 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c02b8a0 tx=0x7f5a3c009240 comp rx=0 tx=0).stop 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 shutdown_connections 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee10518e00 0x55ee10509700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:6836/1422543802,v1:172.21.15.82:6837/1422543802] conn(0x55ee10509c40 0x55ee104f5110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee10516a60 0x55ee10523280 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 --2- 172.21.15.82:0/2883576623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55ee104ccae0 0x55ee1057f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a25ffb640 0 client.6285 ms_handle_reset on v2:172.21.15.82:6836/1422543802 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 >> 172.21.15.82:0/2883576623 conn(0x55ee104c9210 msgr2=0x55ee10519fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 shutdown_connections 2024-03-14T12:16:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 1 -- 172.21.15.82:0/2883576623 wait complete. 2024-03-14T12:16:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.716+0000 7f5a4fc25740 20 client.6285 trim_cache size 0 max 16384 2024-03-14T12:16:42.724 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:42.724 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:16:42.724 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.724 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:42.724 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.726 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 61, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 58, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 15881, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 110835, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 13, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.017099998, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001430604, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.014015168, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000498599, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 57, 2024-03-14T12:16:42.727 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.196443840, 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003446383 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 62, 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000008425, 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000135 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.728 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 70, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 64, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 115544, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 21651, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008666320, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001823566, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.011649202, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000513259, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 63, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.070789889, 2024-03-14T12:16:42.729 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001123649 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000001051, 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000105 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 110944, 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 10992 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.730 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 14, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3603, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 97100, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.003947764, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000589343, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002986632, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000289688, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 18, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.116770253, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.006487236 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:16:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 90560 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:16:42.732 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 48, 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003760639, 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000078346 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.733 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000018003, 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000009001 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.734 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 28, 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 28, 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 28, 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003572602, 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000127592 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:16:42.735 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 12, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 10, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:16:42.736 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 55, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 55, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:16:42.737 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:16:42.738 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 1 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:16:42.739 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:16:42.740 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "6048" 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 1 2024-03-14T12:16:42.741 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.6210", 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 23, 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:16:42.742 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:16:42.743 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 7, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 7, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 2, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:16:42.744 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4196884, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.007709456, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001927364 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.745 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 2, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 42852, 2024-03-14T12:16:42.746 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215392 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 28, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 28, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 4, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.747 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 27, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003310085, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000122595 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.748 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.749 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.750 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:16:42.751 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.752 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000262517, 2024-03-14T12:16:42.754 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000262517 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:16:42.756 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-03-14T12:16:42.757 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 27, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 13, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 0, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:16:42.758 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:16:42.759 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:16:42.760 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 212839, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 57, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5312, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 75, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:16:42.761 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 51472, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 205, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.762 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 42, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 13700, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 42, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 42, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.873983758, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.020809137 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:16:42.763 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 42, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 66 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 42, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 0, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 42, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 11, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 0, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 4, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 7, 2024-03-14T12:16:42.764 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 22, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:16:42.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:16:42.766 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 58, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 1, 2024-03-14T12:16:42.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 6, 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 6, 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 15, 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 1 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.768 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.769 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.770 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 140, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 118197, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 140, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:42.771 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 140, 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 118197, 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.772 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 42, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 13700, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 42, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 13700, 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:42.773 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:42.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 42, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 42, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 42, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 42, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:16:42.776 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x5581581aa6e0": [ 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:16:42.777 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 7, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 2580, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 7, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 4, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2580, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.778 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.779 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:42.779 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:16:42.779 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:42.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5a3406bbd0 msgr2=0x7f5a3406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5a3406bbd0 0x7f5a3406e090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800b450 tx=0x7f5a3800d040 comp rx=0 tx=0).stop 2024-03-14T12:16:42.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480d26e0 msgr2=0x7f5a481f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:42.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480d26e0 0x7f5a481f3100 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 shutdown_connections 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a4815bed0 0x7f5a480b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a480d26e0 0x7f5a481f3100 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5a3406bbd0 0x7f5a3406e090 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 --2- 172.21.15.82:0/3890695431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5a480cd270 0x7f5a481f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 >> 172.21.15.82:0/3890695431 conn(0x7f5a480d1bc0 msgr2=0x7f5a480c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 shutdown_connections 2024-03-14T12:16:42.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:42.717+0000 7f5a4d044640 1 -- 172.21.15.82:0/3890695431 wait complete. 2024-03-14T12:16:42.782 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:16:42.782 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:42.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:42.884 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:16:42.884 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:42.884 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:42.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:16:42.899 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:42.899 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:42.916 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:42.916 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:42.974 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:43.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.074+0000 7fe69c5f3640 1 Processor -- start 2024-03-14T12:16:43.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.074+0000 7fe69c5f3640 1 -- start start 2024-03-14T12:16:43.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.074+0000 7fe69c5f3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe694151230 0x7fe6940cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.074+0000 7fe69c5f3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6940b4b70 0x7fe694070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.074+0000 7fe69c5f3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe69415dde0 0x7fe69415e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.074+0000 7fe69c5f3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69415e790 con 0x7fe6940b4b70 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.074+0000 7fe69c5f3640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69406c3b0 con 0x7fe69415dde0 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe69c5f3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69406c690 con 0x7fe694151230 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6940b4b70 0x7fe694070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6940b4b70 0x7fe694070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54104/0 (socket says 172.21.15.82:54104) 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 -- 172.21.15.82:0/368816717 learned_addr learned my addr 172.21.15.82:0/368816717 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe69a38f640 1 --2- 172.21.15.82:0/368816717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe694151230 0x7fe6940cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe69ab90640 1 --2- 172.21.15.82:0/368816717 >> v2:172.21.15.164:3300/0 conn(0x7fe69415dde0 0x7fe69415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 -- 172.21.15.82:0/368816717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe694151230 msgr2=0x7fe6940cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 --2- 172.21.15.82:0/368816717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe694151230 0x7fe6940cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 -- 172.21.15.82:0/368816717 >> v2:172.21.15.164:3300/0 conn(0x7fe69415dde0 msgr2=0x7fe69415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:43.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 --2- 172.21.15.82:0/368816717 >> v2:172.21.15.164:3300/0 conn(0x7fe69415dde0 0x7fe69415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.075+0000 7fe699b8e640 1 -- 172.21.15.82:0/368816717 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6940b3da0 con 0x7fe6940b4b70 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe699b8e640 1 --2- 172.21.15.82:0/368816717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6940b4b70 0x7fe694070630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe694072410 tx=0x7fe68402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ef6d68503535a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69938d640 1 -- 172.21.15.82:0/368816717 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68402e070 con 0x7fe6940b4b70 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69938d640 1 -- 172.21.15.82:0/368816717 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684037070 con 0x7fe6940b4b70 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69938d640 1 -- 172.21.15.82:0/368816717 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684031a00 con 0x7fe6940b4b70 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 -- 172.21.15.82:0/368816717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6940b4b70 msgr2=0x7fe694070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/368816717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6940b4b70 0x7fe694070630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe694072410 tx=0x7fe68402b880 comp rx=0 tx=0).stop 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 -- 172.21.15.82:0/368816717 shutdown_connections 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/368816717 >> v2:172.21.15.164:3300/0 conn(0x7fe69415dde0 0x7fe69415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/368816717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6940b4b70 0x7fe694070630 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/368816717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe694151230 0x7fe6940cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 -- 172.21.15.82:0/368816717 >> 172.21.15.82:0/368816717 conn(0x7fe6940b1c70 msgr2=0x7fe6940cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:43.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 -- 172.21.15.82:0/368816717 shutdown_connections 2024-03-14T12:16:43.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.076+0000 7fe69c5f3640 1 -- 172.21.15.82:0/368816717 wait complete. 2024-03-14T12:16:43.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 Processor -- start 2024-03-14T12:16:43.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 -- start start 2024-03-14T12:16:43.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6940b4b70 0x7fe6940c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe694151230 0x7fe6941f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe69415dde0 0x7fe694148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe694151610 con 0x7fe694151230 2024-03-14T12:16:43.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940b4360 con 0x7fe6940b4b70 2024-03-14T12:16:43.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe69c5f3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940cf5e0 con 0x7fe69415dde0 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe699b8e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe694151230 0x7fe6941f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe699b8e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe694151230 0x7fe6941f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54110/0 (socket says 172.21.15.82:54110) 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.078+0000 7fe699b8e640 1 -- 172.21.15.82:0/4178186195 learned_addr learned my addr 172.21.15.82:0/4178186195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe69ab90640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe69415dde0 0x7fe694148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe69a38f640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6940b4b70 0x7fe6940c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe699b8e640 1 -- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe69415dde0 msgr2=0x7fe694148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe699b8e640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe69415dde0 0x7fe694148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe699b8e640 1 -- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6940b4b70 msgr2=0x7fe6940c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe699b8e640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6940b4b70 0x7fe6940c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe699b8e640 1 -- 172.21.15.82:0/4178186195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6941f6200 con 0x7fe694151230 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe699b8e640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe694151230 0x7fe6941f5cc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe68402b8c0 tx=0x7fe6840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe68affd640 1 -- 172.21.15.82:0/4178186195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684032880 con 0x7fe694151230 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe68affd640 1 -- 172.21.15.82:0/4178186195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684032a40 con 0x7fe694151230 2024-03-14T12:16:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6940c4a50 con 0x7fe694151230 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe68affd640 1 -- 172.21.15.82:0/4178186195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684032e30 con 0x7fe694151230 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.079+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6940c5010 con 0x7fe694151230 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.080+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69406dad0 con 0x7fe694151230 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.080+0000 7fe68affd640 1 -- 172.21.15.82:0/4178186195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe6840453f0 con 0x7fe694151230 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.081+0000 7fe68affd640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe67806bbd0 0x7fe67806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.081+0000 7fe68affd640 1 -- 172.21.15.82:0/4178186195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6840ac820 con 0x7fe694151230 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.081+0000 7fe69a38f640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe67806bbd0 0x7fe67806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.081+0000 7fe69a38f640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe67806bbd0 0x7fe67806e090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006fd0 tx=0x7fe68c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.085+0000 7fe68affd640 1 -- 172.21.15.82:0/4178186195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe69406dad0 con 0x7fe694151230 2024-03-14T12:16:43.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.259+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe69404f050 con 0x7fe694151230 2024-03-14T12:16:43.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.260+0000 7fe68affd640 1 -- 172.21.15.82:0/4178186195 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16653 (secure 0 0 0) 0x7fe684040070 con 0x7fe694151230 2024-03-14T12:16:43.261 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:43.261 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":58,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:33.876540+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe67806bbd0 msgr2=0x7fe67806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe67806bbd0 0x7fe67806e090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006fd0 tx=0x7fe68c008040 comp rx=0 tx=0).stop 2024-03-14T12:16:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe694151230 msgr2=0x7fe6941f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe694151230 0x7fe6941f5cc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe68402b8c0 tx=0x7fe6840046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 shutdown_connections 2024-03-14T12:16:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe67806bbd0 0x7fe67806e090 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe69415dde0 0x7fe694148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe694151230 0x7fe6941f5cc0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 --2- 172.21.15.82:0/4178186195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe6940b4b70 0x7fe6940c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 >> 172.21.15.82:0/4178186195 conn(0x7fe6940b1c70 msgr2=0x7fe6940ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 shutdown_connections 2024-03-14T12:16:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.263+0000 7fe69c5f3640 1 -- 172.21.15.82:0/4178186195 wait complete. 2024-03-14T12:16:43.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:30.434900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:31.188753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:43.274 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:43.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.377+0000 7f618364b640 1 Processor -- start 2024-03-14T12:16:43.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.377+0000 7f618364b640 1 -- start start 2024-03-14T12:16:43.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f618364b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f617c15bde0 0x7f617c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f618364b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f617c0d08c0 0x7f617c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f618364b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617c0ce380 con 0x7f617c0cd250 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f618364b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617c0cef90 con 0x7f617c15bde0 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f618364b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617c0b5a30 con 0x7f617c0d08c0 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f61813e7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f617c15bde0 0x7f617c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f6180be6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f617c0cd250 0x7f617c0d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51640/0 (socket says 172.21.15.82:51640) 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.378+0000 7f6180be6640 1 -- 172.21.15.82:0/3837111915 learned_addr learned my addr 172.21.15.82:0/3837111915 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f61813e7640 1 -- 172.21.15.82:0/3837111915 >> v2:172.21.15.164:3300/0 conn(0x7f617c0d08c0 msgr2=0x7f617c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f61813e7640 1 --2- 172.21.15.82:0/3837111915 >> v2:172.21.15.164:3300/0 conn(0x7f617c0d08c0 0x7f617c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f61813e7640 1 -- 172.21.15.82:0/3837111915 >> v1:172.21.15.82:6789/0 conn(0x7f617c0cd250 legacy=0x7f617c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f6173fff640 1 -- 172.21.15.82:0/3837111915 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826751643 0 0) 0x7f617c0ce380 con 0x7f617c0cd250 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f61813e7640 1 -- 172.21.15.82:0/3837111915 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0ce660 con 0x7f617c15bde0 2024-03-14T12:16:43.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f61813e7640 1 --2- 172.21.15.82:0/3837111915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c15bde0 0x7f617c15c1c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f617c071d70 tx=0x7f617802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2488718a63effd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f6173fff640 1 -- 172.21.15.82:0/3837111915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178034070 con 0x7f617c15bde0 2024-03-14T12:16:43.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f6173fff640 1 -- 172.21.15.82:0/3837111915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178030840 con 0x7f617c15bde0 2024-03-14T12:16:43.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.379+0000 7f6173fff640 1 -- 172.21.15.82:0/3837111915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178030c10 con 0x7f617c15bde0 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 -- 172.21.15.82:0/3837111915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c15bde0 msgr2=0x7f617c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 --2- 172.21.15.82:0/3837111915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c15bde0 0x7f617c15c1c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f617c071d70 tx=0x7f617802b980 comp rx=0 tx=0).stop 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 -- 172.21.15.82:0/3837111915 shutdown_connections 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 --2- 172.21.15.82:0/3837111915 >> v2:172.21.15.164:3300/0 conn(0x7f617c0d08c0 0x7f617c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 --2- 172.21.15.82:0/3837111915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c15bde0 0x7f617c15c1c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 -- 172.21.15.82:0/3837111915 >> 172.21.15.82:0/3837111915 conn(0x7f617c0d1bd0 msgr2=0x7f617c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 -- 172.21.15.82:0/3837111915 shutdown_connections 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.380+0000 7f618364b640 1 -- 172.21.15.82:0/3837111915 wait complete. 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.381+0000 7f618364b640 1 Processor -- start 2024-03-14T12:16:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.381+0000 7f618364b640 1 -- start start 2024-03-14T12:16:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f618364b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f617c0cd250 0x7f617c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f618364b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c0d08c0 0x7f617c14ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f618364b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f617c15bde0 0x7f617c0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f618364b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c0cf420 con 0x7f617c0d08c0 2024-03-14T12:16:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f618364b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c0cec20 con 0x7f617c0cd250 2024-03-14T12:16:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f618364b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c0b6c20 con 0x7f617c15bde0 2024-03-14T12:16:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c0d08c0 0x7f617c14ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c0d08c0 0x7f617c14ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54126/0 (socket says 172.21.15.82:54126) 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 -- 172.21.15.82:0/3741540997 learned_addr learned my addr 172.21.15.82:0/3741540997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6181be8640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f617c15bde0 0x7f617c0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f61813e7640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f617c0cd250 0x7f617c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 -- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f617c15bde0 msgr2=0x7f617c0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f617c15bde0 0x7f617c0c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 -- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f617c0cd250 msgr2=0x7f617c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f617c0cd250 0x7f617c0cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.382+0000 7f6180be6640 1 -- 172.21.15.82:0/3741540997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0ca110 con 0x7f617c0d08c0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.383+0000 7f6180be6640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c0d08c0 0x7f617c14ef00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f616c00c4d0 tx=0x7f616c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.383+0000 7f6171ffb640 1 -- 172.21.15.82:0/3741540997 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c018070 con 0x7f617c0d08c0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.383+0000 7f6171ffb640 1 -- 172.21.15.82:0/3741540997 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c007500 con 0x7f617c0d08c0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.383+0000 7f6171ffb640 1 -- 172.21.15.82:0/3741540997 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c004a20 con 0x7f617c0d08c0 2024-03-14T12:16:43.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.383+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617c15c6c0 con 0x7f617c0d08c0 2024-03-14T12:16:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.383+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c15cc80 con 0x7f617c0d08c0 2024-03-14T12:16:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.384+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617c04f050 con 0x7f617c0d08c0 2024-03-14T12:16:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.384+0000 7f6171ffb640 1 -- 172.21.15.82:0/3741540997 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f616c01b070 con 0x7f617c0d08c0 2024-03-14T12:16:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.384+0000 7f6171ffb640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f616006bc00 0x7f616006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.384+0000 7f6171ffb640 1 -- 172.21.15.82:0/3741540997 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f616c089d10 con 0x7f617c0d08c0 2024-03-14T12:16:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.384+0000 7f61813e7640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f616006bc00 0x7f616006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.385+0000 7f61813e7640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f616006bc00 0x7f616006e0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6178036000 tx=0x7f6178008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.389+0000 7f6171ffb640 1 -- 172.21.15.82:0/3741540997 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f616c058890 con 0x7f617c0d08c0 2024-03-14T12:16:43.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.561+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f617c0b7b50 con 0x7f617c0d08c0 2024-03-14T12:16:43.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.561+0000 7f6171ffb640 1 -- 172.21.15.82:0/3741540997 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f616c058190 con 0x7f617c0d08c0 2024-03-14T12:16:43.562 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f616006bc00 msgr2=0x7f616006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f616006bc00 0x7f616006e0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6178036000 tx=0x7f6178008030 comp rx=0 tx=0).stop 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c0d08c0 msgr2=0x7f617c14ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c0d08c0 0x7f617c14ef00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f616c00c4d0 tx=0x7f616c00c990 comp rx=0 tx=0).stop 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 shutdown_connections 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f616006bc00 0x7f616006e0c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f617c15bde0 0x7f617c0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f617c0d08c0 0x7f617c14ef00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 --2- 172.21.15.82:0/3741540997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f617c0cd250 0x7f617c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 >> 172.21.15.82:0/3741540997 conn(0x7f617c0d1bd0 msgr2=0x7f617c15f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:43.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 shutdown_connections 2024-03-14T12:16:43.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.563+0000 7f618364b640 1 -- 172.21.15.82:0/3741540997 wait complete. 2024-03-14T12:16:43.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:16:43.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.678+0000 7f03eabcd640 1 Processor -- start 2024-03-14T12:16:43.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.678+0000 7f03eabcd640 1 -- start start 2024-03-14T12:16:43.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.678+0000 7f03eabcd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f03e415e0c0 0x7f03e40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.678+0000 7f03eabcd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.678+0000 7f03eabcd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e40cf440 con 0x7f03e40d01b0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.678+0000 7f03eabcd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e40cf1e0 con 0x7f03e415e0c0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.678+0000 7f03eabcd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40ccb60 con 0x7f03e40b0bc0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e8969640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f03e415e0c0 0x7f03e40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e916a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f03e40d01b0 0x7f03e406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51656/0 (socket says 172.21.15.82:51656) 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e916a640 1 -- 172.21.15.82:0/2609864845 learned_addr learned my addr 172.21.15.82:0/2609864845 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e8969640 1 -- 172.21.15.82:0/2609864845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 msgr2=0x7f03e40ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e3fff640 1 --2- 172.21.15.82:0/2609864845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e8969640 1 --2- 172.21.15.82:0/2609864845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e37fe640 1 -- 172.21.15.82:0/2609864845 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771796614 0 0) 0x7f03e40cf440 con 0x7f03e40d01b0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e8969640 1 -- 172.21.15.82:0/2609864845 >> v1:172.21.15.82:6789/0 conn(0x7f03e40d01b0 legacy=0x7f03e406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.679+0000 7f03e8969640 1 -- 172.21.15.82:0/2609864845 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40cce40 con 0x7f03e415e0c0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03e8969640 1 --2- 172.21.15.82:0/2609864845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e415e0c0 0x7f03e40cd920 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f03d80097a0 tx=0x7f03d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57303a0961a50347 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03e37fe640 1 -- 172.21.15.82:0/2609864845 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8035020 con 0x7f03e415e0c0 2024-03-14T12:16:43.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03e37fe640 1 -- 172.21.15.82:0/2609864845 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d8031830 con 0x7f03e415e0c0 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03e37fe640 1 -- 172.21.15.82:0/2609864845 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8031c00 con 0x7f03e415e0c0 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 -- 172.21.15.82:0/2609864845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e415e0c0 msgr2=0x7f03e40cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 --2- 172.21.15.82:0/2609864845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e415e0c0 0x7f03e40cd920 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f03d80097a0 tx=0x7f03d802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 -- 172.21.15.82:0/2609864845 shutdown_connections 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 --2- 172.21.15.82:0/2609864845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 --2- 172.21.15.82:0/2609864845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e415e0c0 0x7f03e40cd920 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 -- 172.21.15.82:0/2609864845 >> 172.21.15.82:0/2609864845 conn(0x7f03e40d1bd0 msgr2=0x7f03e40d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 -- 172.21.15.82:0/2609864845 shutdown_connections 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.680+0000 7f03eabcd640 1 -- 172.21.15.82:0/2609864845 wait complete. 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.681+0000 7f03eabcd640 1 Processor -- start 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.681+0000 7f03eabcd640 1 -- start start 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03eabcd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03eabcd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e40d01b0 0x7f03e41f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03eabcd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e415e0c0 0x7f03e41f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03eabcd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e415e4a0 con 0x7f03e40d01b0 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03eabcd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cd400 con 0x7f03e415e0c0 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03eabcd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40d0640 con 0x7f03e40b0bc0 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e40d01b0 0x7f03e41f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e40d01b0 0x7f03e41f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54154/0 (socket says 172.21.15.82:54154) 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.682+0000 7f03e3fff640 1 -- 172.21.15.82:0/1675159469 learned_addr learned my addr 172.21.15.82:0/1675159469 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e8969640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e916a640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e415e0c0 0x7f03e41f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e3fff640 1 -- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 msgr2=0x7f03e40cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e3fff640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e3fff640 1 -- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e415e0c0 msgr2=0x7f03e41f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e3fff640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e415e0c0 0x7f03e41f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e3fff640 1 -- 172.21.15.82:0/1675159469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40caa10 con 0x7f03e40d01b0 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e8969640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e3fff640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e40d01b0 0x7f03e41f7ba0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c7f0 tx=0x7f03d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e17fa640 1 -- 172.21.15.82:0/1675159469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0015070 con 0x7f03e40d01b0 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e17fa640 1 -- 172.21.15.82:0/1675159469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0004240 con 0x7f03e40d01b0 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03e17fa640 1 -- 172.21.15.82:0/1675159469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0007620 con 0x7f03e40d01b0 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e40c96b0 con 0x7f03e40d01b0 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.683+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e40c9f50 con 0x7f03e40d01b0 2024-03-14T12:16:43.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.684+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e4075300 con 0x7f03e40d01b0 2024-03-14T12:16:43.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.684+0000 7f03e17fa640 1 -- 172.21.15.82:0/1675159469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f03d00079d0 con 0x7f03e40d01b0 2024-03-14T12:16:43.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.685+0000 7f03e17fa640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f03c806bbd0 0x7f03c806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.685+0000 7f03e17fa640 1 -- 172.21.15.82:0/1675159469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03d008ae70 con 0x7f03e40d01b0 2024-03-14T12:16:43.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.685+0000 7f03e8969640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f03c806bbd0 0x7f03c806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.685+0000 7f03e8969640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f03c806bbd0 0x7f03c806e090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f03d8036000 tx=0x7f03d803c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.690+0000 7f03e17fa640 1 -- 172.21.15.82:0/1675159469 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f03e4075300 con 0x7f03e40d01b0 2024-03-14T12:16:43.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.860+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03e40cc5a0 con 0x7f03e40d01b0 2024-03-14T12:16:43.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.860+0000 7f03e17fa640 1 -- 172.21.15.82:0/1675159469 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03e40cc5a0 con 0x7f03e40d01b0 2024-03-14T12:16:43.861 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:16:43.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f03c806bbd0 msgr2=0x7f03c806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f03c806bbd0 0x7f03c806e090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f03d8036000 tx=0x7f03d803c040 comp rx=0 tx=0).stop 2024-03-14T12:16:43.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e40d01b0 msgr2=0x7f03e41f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e40d01b0 0x7f03e41f7ba0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f03d000c7f0 tx=0x7f03d000ccb0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 shutdown_connections 2024-03-14T12:16:43.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03e415e0c0 0x7f03e41f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03e40d01b0 0x7f03e41f7ba0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f03c806bbd0 0x7f03c806e090 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 --2- 172.21.15.82:0/1675159469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03e40b0bc0 0x7f03e40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 >> 172.21.15.82:0/1675159469 conn(0x7f03e40d1bd0 msgr2=0x7f03e40cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:43.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.862+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 shutdown_connections 2024-03-14T12:16:43.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.863+0000 7f03eabcd640 1 -- 172.21.15.82:0/1675159469 wait complete. 2024-03-14T12:16:43.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:16:43.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.976+0000 7f37241df640 1 Processor -- start 2024-03-14T12:16:43.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f37241df640 1 -- start start 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f37241df640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f371c1608c0 0x7f371c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f37241df640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f371c15fd00 0x7f371c160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f37241df640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c0cf8a0 con 0x7f371c15c0e0 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f37241df640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c0d0ba0 con 0x7f371c1608c0 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f37241df640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c1606b0 con 0x7f371c15fd00 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f3721f7b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f371c15c0e0 0x7f371c0cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51670/0 (socket says 172.21.15.82:51670) 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f372177a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f371c1608c0 0x7f371c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.977+0000 7f3721f7b640 1 -- 172.21.15.82:0/3762374281 learned_addr learned my addr 172.21.15.82:0/3762374281 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.978+0000 7f372177a640 1 -- 172.21.15.82:0/3762374281 >> v2:172.21.15.164:3300/0 conn(0x7f371c15fd00 msgr2=0x7f371c160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.978+0000 7f372177a640 1 --2- 172.21.15.82:0/3762374281 >> v2:172.21.15.164:3300/0 conn(0x7f371c15fd00 0x7f371c160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.978+0000 7f372177a640 1 -- 172.21.15.82:0/3762374281 >> v1:172.21.15.82:6789/0 conn(0x7f371c15c0e0 legacy=0x7f371c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.978+0000 7f3720f79640 1 -- 172.21.15.82:0/3762374281 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200570143 0 0) 0x7f371c0cf8a0 con 0x7f371c15c0e0 2024-03-14T12:16:43.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.978+0000 7f372177a640 1 -- 172.21.15.82:0/3762374281 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0cfb80 con 0x7f371c1608c0 2024-03-14T12:16:43.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.978+0000 7f372177a640 1 --2- 172.21.15.82:0/3762374281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c1608c0 0x7f371c0d0660 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f371c071b80 tx=0x7f370c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb5c4e72450323b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f3720f79640 1 -- 172.21.15.82:0/3762374281 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c035020 con 0x7f371c1608c0 2024-03-14T12:16:43.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f3720f79640 1 -- 172.21.15.82:0/3762374281 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c031830 con 0x7f371c1608c0 2024-03-14T12:16:43.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f3720f79640 1 -- 172.21.15.82:0/3762374281 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c031c00 con 0x7f371c1608c0 2024-03-14T12:16:43.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 -- 172.21.15.82:0/3762374281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c1608c0 msgr2=0x7f371c0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 --2- 172.21.15.82:0/3762374281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c1608c0 0x7f371c0d0660 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f371c071b80 tx=0x7f370c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 -- 172.21.15.82:0/3762374281 shutdown_connections 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 --2- 172.21.15.82:0/3762374281 >> v2:172.21.15.164:3300/0 conn(0x7f371c15fd00 0x7f371c160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 --2- 172.21.15.82:0/3762374281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c1608c0 0x7f371c0d0660 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 -- 172.21.15.82:0/3762374281 >> 172.21.15.82:0/3762374281 conn(0x7f371c0b3680 msgr2=0x7f371c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 -- 172.21.15.82:0/3762374281 shutdown_connections 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.979+0000 7f37241df640 1 -- 172.21.15.82:0/3762374281 wait complete. 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.980+0000 7f37241df640 1 Processor -- start 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.980+0000 7f37241df640 1 -- start start 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f37241df640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f371c15c0e0 0x7f371c158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f37241df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c15fd00 0x7f371c09ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f37241df640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f371c1608c0 0x7f371c0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f37241df640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0cd960 con 0x7f371c15fd00 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f37241df640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0d01d0 con 0x7f371c1608c0 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f37241df640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c15c4c0 con 0x7f371c15c0e0 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f372177a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c15fd00 0x7f371c09ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f372177a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c15fd00 0x7f371c09ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54178/0 (socket says 172.21.15.82:54178) 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f372177a640 1 -- 172.21.15.82:0/1077291482 learned_addr learned my addr 172.21.15.82:0/1077291482 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f3721f7b640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f371c15c0e0 0x7f371c158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f372277c640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f371c1608c0 0x7f371c0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.981+0000 7f372177a640 1 -- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f371c15c0e0 msgr2=0x7f371c158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f372177a640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f371c15c0e0 0x7f371c158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f372177a640 1 -- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f371c1608c0 msgr2=0x7f371c0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f372177a640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f371c1608c0 0x7f371c0cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:43.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f372177a640 1 -- 172.21.15.82:0/1077291482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0c6ac0 con 0x7f371c15fd00 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f372277c640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f371c1608c0 0x7f371c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f372177a640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c15fd00 0x7f371c09ba70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f370c009890 tx=0x7f370c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f37127fc640 1 -- 172.21.15.82:0/1077291482 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c049070 con 0x7f371c15fd00 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f37127fc640 1 -- 172.21.15.82:0/1077291482 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c03c9d0 con 0x7f371c15fd00 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c0c6da0 con 0x7f371c15fd00 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f37127fc640 1 -- 172.21.15.82:0/1077291482 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c03cda0 con 0x7f371c15fd00 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.982+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c0c7360 con 0x7f371c15fd00 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.983+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c0b3e00 con 0x7f371c15fd00 2024-03-14T12:16:43.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.983+0000 7f37127fc640 1 -- 172.21.15.82:0/1077291482 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f370c04c070 con 0x7f371c15fd00 2024-03-14T12:16:43.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.984+0000 7f37127fc640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f370006bbd0 0x7f370006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:43.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.984+0000 7f37127fc640 1 -- 172.21.15.82:0/1077291482 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f370c0ac920 con 0x7f371c15fd00 2024-03-14T12:16:43.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.984+0000 7f3721f7b640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f370006bbd0 0x7f370006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:43.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.984+0000 7f3721f7b640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f370006bbd0 0x7f370006e090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f37180012e0 tx=0x7f3718001630 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:43.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:43.988+0000 7f37127fc640 1 -- 172.21.15.82:0/1077291482 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f371c0b3e00 con 0x7f371c15fd00 2024-03-14T12:16:44.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.160+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f371c0c7560 con 0x7f371c15fd00 2024-03-14T12:16:44.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.160+0000 7f37127fc640 1 -- 172.21.15.82:0/1077291482 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f371c0c7560 con 0x7f371c15fd00 2024-03-14T12:16:44.160 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f370006bbd0 msgr2=0x7f370006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f370006bbd0 0x7f370006e090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f37180012e0 tx=0x7f3718001630 comp rx=0 tx=0).stop 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c15fd00 msgr2=0x7f371c09ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c15fd00 0x7f371c09ba70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f370c009890 tx=0x7f370c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 shutdown_connections 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f371c1608c0 0x7f371c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f371c15fd00 0x7f371c09ba70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f370006bbd0 0x7f370006e090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 --2- 172.21.15.82:0/1077291482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f371c15c0e0 0x7f371c158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 >> 172.21.15.82:0/1077291482 conn(0x7f371c0b3680 msgr2=0x7f371c0b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:44.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 shutdown_connections 2024-03-14T12:16:44.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.162+0000 7f37241df640 1 -- 172.21.15.82:0/1077291482 wait complete. 2024-03-14T12:16:44.173 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:16:44.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.273+0000 7f92c4576640 1 Processor -- start 2024-03-14T12:16:44.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.273+0000 7f92c4576640 1 -- start start 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.274+0000 7f92c4576640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92bc0cf370 0x7f92bc0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.274+0000 7f92c4576640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0b7720 0x7f92bc0b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.274+0000 7f92c4576640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0cf8c0 con 0x7f92bc0d0760 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.274+0000 7f92c4576640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc0cfba0 con 0x7f92bc0cf370 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.274+0000 7f92c4576640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0b6c40 con 0x7f92bc0b7720 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c1b11640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92bc0cf370 0x7f92bc0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c2312640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f92bc0d0760 0x7f92bc0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51672/0 (socket says 172.21.15.82:51672) 2024-03-14T12:16:44.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c2312640 1 -- 172.21.15.82:0/700600718 learned_addr learned my addr 172.21.15.82:0/700600718 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:44.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c1b11640 1 -- 172.21.15.82:0/700600718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0b7720 msgr2=0x7f92bc0b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:44.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c1b11640 1 --2- 172.21.15.82:0/700600718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0b7720 0x7f92bc0b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c1b11640 1 -- 172.21.15.82:0/700600718 >> v1:172.21.15.82:6789/0 conn(0x7f92bc0d0760 legacy=0x7f92bc0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c1310640 1 -- 172.21.15.82:0/700600718 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44042171 0 0) 0x7f92bc0cf8c0 con 0x7f92bc0d0760 2024-03-14T12:16:44.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.275+0000 7f92c1b11640 1 -- 172.21.15.82:0/700600718 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0b6f20 con 0x7f92bc0cf370 2024-03-14T12:16:44.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c1b11640 1 --2- 172.21.15.82:0/700600718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc0d4660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f92b80097d0 tx=0x7f92b802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c58549244d07d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:44.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c1310640 1 -- 172.21.15.82:0/700600718 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8035020 con 0x7f92bc0cf370 2024-03-14T12:16:44.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c1310640 1 -- 172.21.15.82:0/700600718 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8031890 con 0x7f92bc0cf370 2024-03-14T12:16:44.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c1310640 1 -- 172.21.15.82:0/700600718 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8031c60 con 0x7f92bc0cf370 2024-03-14T12:16:44.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 -- 172.21.15.82:0/700600718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 msgr2=0x7f92bc0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 --2- 172.21.15.82:0/700600718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc0d4660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f92b80097d0 tx=0x7f92b802b930 comp rx=0 tx=0).stop 2024-03-14T12:16:44.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 -- 172.21.15.82:0/700600718 shutdown_connections 2024-03-14T12:16:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 --2- 172.21.15.82:0/700600718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0b7720 0x7f92bc0b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 --2- 172.21.15.82:0/700600718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc0d4660 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 -- 172.21.15.82:0/700600718 >> 172.21.15.82:0/700600718 conn(0x7f92bc09d030 msgr2=0x7f92bc0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 -- 172.21.15.82:0/700600718 shutdown_connections 2024-03-14T12:16:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.276+0000 7f92c4576640 1 -- 172.21.15.82:0/700600718 wait complete. 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.277+0000 7f92c4576640 1 Processor -- start 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c4576640 1 -- start start 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c4576640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92bc0b7720 0x7f92bc0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c4576640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c4576640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0d0760 0x7f92bc1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c4576640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc1617a0 con 0x7f92bc0cf370 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c4576640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0d4ba0 con 0x7f92bc0b7720 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c4576640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc161920 con 0x7f92bc0d0760 2024-03-14T12:16:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c1b11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c1b11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc1f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54194/0 (socket says 172.21.15.82:54194) 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.278+0000 7f92c1b11640 1 -- 172.21.15.82:0/283184669 learned_addr learned my addr 172.21.15.82:0/283184669 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c1b11640 1 -- 172.21.15.82:0/283184669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0d0760 msgr2=0x7f92bc1f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c2b13640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0d0760 0x7f92bc1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c1b11640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0d0760 0x7f92bc1f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c1b11640 1 -- 172.21.15.82:0/283184669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92bc0b7720 msgr2=0x7f92bc0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c1b11640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92bc0b7720 0x7f92bc0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c1b11640 1 -- 172.21.15.82:0/283184669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0cc500 con 0x7f92bc0cf370 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c1b11640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc1f4390 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f92b802bf40 tx=0x7f92b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92b2ffd640 1 -- 172.21.15.82:0/283184669 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80453f0 con 0x7f92bc0cf370 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92b2ffd640 1 -- 172.21.15.82:0/283184669 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8045590 con 0x7f92bc0cf370 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92b2ffd640 1 -- 172.21.15.82:0/283184669 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8045980 con 0x7f92bc0cf370 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc0cc7e0 con 0x7f92bc0cf370 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.279+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc0ccda0 con 0x7f92bc0cf370 2024-03-14T12:16:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.280+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc04f050 con 0x7f92bc0cf370 2024-03-14T12:16:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.280+0000 7f92b2ffd640 1 -- 172.21.15.82:0/283184669 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f92b8038020 con 0x7f92bc0cf370 2024-03-14T12:16:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.281+0000 7f92b2ffd640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92a006bbd0 0x7f92a006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.281+0000 7f92b2ffd640 1 -- 172.21.15.82:0/283184669 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92b80ace50 con 0x7f92bc0cf370 2024-03-14T12:16:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.281+0000 7f92c2312640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92a006bbd0 0x7f92a006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.281+0000 7f92c2312640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92a006bbd0 0x7f92a006e090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f92ac007fd0 tx=0x7f92ac007960 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:44.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.285+0000 7f92b2ffd640 1 -- 172.21.15.82:0/283184669 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f92b8040070 con 0x7f92bc0cf370 2024-03-14T12:16:44.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.470+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92bc0ce110 con 0x7f92bc0cf370 2024-03-14T12:16:44.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.471+0000 7f92b2ffd640 1 -- 172.21.15.82:0/283184669 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f92b802d040 con 0x7f92bc0cf370 2024-03-14T12:16:44.470 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:16:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92a006bbd0 msgr2=0x7f92a006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92a006bbd0 0x7f92a006e090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f92ac007fd0 tx=0x7f92ac007960 comp rx=0 tx=0).stop 2024-03-14T12:16:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 msgr2=0x7f92bc1f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc1f4390 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f92b802bf40 tx=0x7f92b80046c0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 shutdown_connections 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92a006bbd0 0x7f92a006e090 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92bc0d0760 0x7f92bc1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc0cf370 0x7f92bc1f4390 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 --2- 172.21.15.82:0/283184669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92bc0b7720 0x7f92bc0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 >> 172.21.15.82:0/283184669 conn(0x7f92bc09d030 msgr2=0x7f92bc15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 shutdown_connections 2024-03-14T12:16:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.473+0000 7f92c4576640 1 -- 172.21.15.82:0/283184669 wait complete. 2024-03-14T12:16:44.483 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T12:16:44.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:44.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T12:16:44.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:16:44.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:16:44.520 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T12:16:44.576 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:44.577 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:44.592 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:16:44.592 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:44.592 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:44.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:16:44.634 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.735+0000 7f53d189a640 1 Processor -- start 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.735+0000 7f53d189a640 1 -- start start 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53d189a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f53cc0b8730 0x7f53cc168b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53d189a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc0cf8c0 con 0x7f53cc0d0760 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53d189a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc0cfba0 con 0x7f53cc0cf370 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53d189a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53cc0b6c40 con 0x7f53cc0b8730 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53caffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f53cc0d0760 0x7f53cc0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51680/0 (socket says 172.21.15.82:51680) 2024-03-14T12:16:44.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53caffd640 1 -- 172.21.15.82:0/3764186362 learned_addr learned my addr 172.21.15.82:0/3764186362 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055065316 0 0) 0x7f53cc0cf8c0 con 0x7f53cc0d0760 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b4003520 con 0x7f53cc0d0760 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53cb7fe640 1 --2- 172.21.15.82:0/3764186362 >> v2:172.21.15.164:3300/0 conn(0x7f53cc0b8730 0x7f53cc168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3434525826 0 0) 0x7f53b4003520 con 0x7f53cc0d0760 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 >> v2:172.21.15.164:3300/0 conn(0x7f53cc0b8730 msgr2=0x7f53cc168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 --2- 172.21.15.82:0/3764186362 >> v2:172.21.15.164:3300/0 conn(0x7f53cc0b8730 0x7f53cc168b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 >> v1:172.21.15.164:6789/0 conn(0x7f53cc0cf370 legacy=0x7f53cc0d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0b6f20 con 0x7f53cc0d0760 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f53bc001670 con 0x7f53cc0d0760 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 >> v1:172.21.15.82:6789/0 conn(0x7f53cc0d0760 legacy=0x7f53cc0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b4004800 0x7f53b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.736+0000 7f53c9ffb640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b40051c0 0x7f53b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0b6c40 con 0x7f53b40039b0 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b400c0f0 con 0x7f53b4004800 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b400c2f0 con 0x7f53b40051c0 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53caffd640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53ca7fc640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b4004800 0x7f53b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53cb7fe640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b40051c0 0x7f53b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53caffd640 1 -- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b40051c0 msgr2=0x7f53b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53caffd640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b40051c0 0x7f53b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53caffd640 1 -- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b4004800 msgr2=0x7f53b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53caffd640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b4004800 0x7f53b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53cb7fe640 1 -- 172.21.15.82:0/3764186362 reap_dead start 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53caffd640 1 -- 172.21.15.82:0/3764186362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4003520 con 0x7f53b40039b0 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.737+0000 7f53ca7fc640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b4004800 0x7f53b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53caffd640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53b4003dd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c240 tx=0x7f53b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f4ec0e559a1128 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc007d10 con 0x7f53b40039b0 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc030070 con 0x7f53b40039b0 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53c9ffb640 1 -- 172.21.15.82:0/3764186362 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc005d70 con 0x7f53b40039b0 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53d189a640 1 -- 172.21.15.82:0/3764186362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 msgr2=0x7f53b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53d189a640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53b4003dd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c240 tx=0x7f53b4005830 comp rx=0 tx=0).stop 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53d189a640 1 -- 172.21.15.82:0/3764186362 shutdown_connections 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53d189a640 1 --2- 172.21.15.82:0/3764186362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53b4003dd0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53d189a640 1 -- 172.21.15.82:0/3764186362 >> 172.21.15.82:0/3764186362 conn(0x7f53cc09d030 msgr2=0x7f53cc0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53d189a640 1 -- 172.21.15.82:0/3764186362 shutdown_connections 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.738+0000 7f53d189a640 1 -- 172.21.15.82:0/3764186362 wait complete. 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.739+0000 7f53d189a640 1 Processor -- start 2024-03-14T12:16:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53d189a640 1 -- start start 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53d189a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53cc1454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53d189a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53cc146c00 0x7f53cc0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53d189a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53cc147080 0x7f53cc0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53d189a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc161920 con 0x7f53b40039b0 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53d189a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc1617a0 con 0x7f53cc146c00 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53d189a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0d0b40 con 0x7f53cc147080 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53cc1454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53cc1454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54214/0 (socket says 172.21.15.82:54214) 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53caffd640 1 -- 172.21.15.82:0/1131895999 learned_addr learned my addr 172.21.15.82:0/1131895999 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53cb7fe640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53cc147080 0x7f53cc0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.740+0000 7f53ca7fc640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53cc146c00 0x7f53cc0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53caffd640 1 -- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53cc147080 msgr2=0x7f53cc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53caffd640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53cc147080 0x7f53cc0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53caffd640 1 -- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53cc146c00 msgr2=0x7f53cc0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53caffd640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53cc146c00 0x7f53cc0c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53caffd640 1 -- 172.21.15.82:0/1131895999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc145b60 con 0x7f53b40039b0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53ca7fc640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53cc146c00 0x7f53cc0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53cb7fe640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53cc147080 0x7f53cc0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53caffd640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53cc1454d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c360 tx=0x7f53bc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53b3fff640 1 -- 172.21.15.82:0/1131895999 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc046070 con 0x7f53b40039b0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53b3fff640 1 -- 172.21.15.82:0/1131895999 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc004bf0 con 0x7f53b40039b0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53b3fff640 1 -- 172.21.15.82:0/1131895999 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc007d10 con 0x7f53b40039b0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc145e40 con 0x7f53b40039b0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.741+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc146400 con 0x7f53b40039b0 2024-03-14T12:16:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.742+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc04f050 con 0x7f53b40039b0 2024-03-14T12:16:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.742+0000 7f53b3fff640 1 -- 172.21.15.82:0/1131895999 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f53bc007070 con 0x7f53b40039b0 2024-03-14T12:16:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.743+0000 7f53b3fff640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f53b806bbd0 0x7f53b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.743+0000 7f53b3fff640 1 -- 172.21.15.82:0/1131895999 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53bc0abe70 con 0x7f53b40039b0 2024-03-14T12:16:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.743+0000 7f53ca7fc640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f53b806bbd0 0x7f53b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.743+0000 7f53ca7fc640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f53b806bbd0 0x7f53b806e090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000c00 tx=0x7f53c0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.747+0000 7f53b3fff640 1 -- 172.21.15.82:0/1131895999 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f53bc07a280 con 0x7f53b40039b0 2024-03-14T12:16:44.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.959+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53cc0d0540 con 0x7f53b40039b0 2024-03-14T12:16:44.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.959+0000 7f53b3fff640 1 -- 172.21.15.82:0/1131895999 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f53bc07e320 con 0x7f53b40039b0 2024-03-14T12:16:44.959 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:44.959 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5111,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/4078909817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4078909817},{"type":"v1","addr":"172.21.15.164:6834","nonce":4078909817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2349002857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2349002857},{"type":"v1","addr":"172.21.15.164:6835","nonce":2349002857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/285826310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":285826310},{"type":"v1","addr":"172.21.15.82:6835","nonce":285826310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:32.222787+0000","modified":"2024-03-14T12:16:34.220226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1422543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1422543802},{"type":"v1","addr":"172.21.15.82:6837","nonce":1422543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T12:16:44.959 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f53b806bbd0 msgr2=0x7f53b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f53b806bbd0 0x7f53b806e090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000c00 tx=0x7f53c0009040 comp rx=0 tx=0).stop 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 msgr2=0x7f53cc1454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53cc1454d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c360 tx=0x7f53bc02e070 comp rx=0 tx=0).stop 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 shutdown_connections 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f53b806bbd0 0x7f53b806e090 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53cc147080 0x7f53cc0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53cc146c00 0x7f53cc0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 --2- 172.21.15.82:0/1131895999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40039b0 0x7f53cc1454d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 >> 172.21.15.82:0/1131895999 conn(0x7f53cc09d030 msgr2=0x7f53cc0b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 shutdown_connections 2024-03-14T12:16:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:44.961+0000 7f53d189a640 1 -- 172.21.15.82:0/1131895999 wait complete. 2024-03-14T12:16:44.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:45.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.072+0000 7fd45bd7e640 1 Processor -- start 2024-03-14T12:16:45.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.072+0000 7fd45bd7e640 1 -- start start 2024-03-14T12:16:45.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd45bd7e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd45406b670 0x7fd454072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd45bd7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406eb20 0x7fd45406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd45bd7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4540727a0 con 0x7fd4540b0200 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd45bd7e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd454071810 con 0x7fd45406b670 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd45bd7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd454071dd0 con 0x7fd45406eb20 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd459b1a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd45406b670 0x7fd454072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd45a31b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406eb20 0x7fd45406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd459319640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd4540b0200 0x7fd4540ce730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:33504/0 (socket says 172.21.15.82:33504) 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd45a31b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406eb20 0x7fd45406efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:48872/0 (socket says 172.21.15.82:48872) 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd459319640 1 -- 172.21.15.82:0/3812545725 learned_addr learned my addr 172.21.15.82:0/3812545725 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.073+0000 7fd459b1a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd45406b670 0x7fd454072260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:33738/0 (socket says 172.21.15.82:33738) 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.074+0000 7fd45a31b640 1 -- 172.21.15.82:0/3812545725 >> v2:172.21.15.164:3300/0 conn(0x7fd45406b670 msgr2=0x7fd454072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.074+0000 7fd45a31b640 1 --2- 172.21.15.82:0/3812545725 >> v2:172.21.15.164:3300/0 conn(0x7fd45406b670 0x7fd454072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.074+0000 7fd45a31b640 1 -- 172.21.15.82:0/3812545725 >> v1:172.21.15.164:6789/0 conn(0x7fd4540b0200 legacy=0x7fd4540ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.074+0000 7fd45a31b640 1 -- 172.21.15.82:0/3812545725 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540d0410 con 0x7fd45406eb20 2024-03-14T12:16:45.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.074+0000 7fd459b1a640 1 --2- 172.21.15.82:0/3812545725 >> v2:172.21.15.164:3300/0 conn(0x7fd45406b670 0x7fd454072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:45.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45a31b640 1 --2- 172.21.15.82:0/3812545725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406eb20 0x7fd45406efb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f4a5dadb970970d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd458b18640 1 -- 172.21.15.82:0/3812545725 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c039070 con 0x7fd45406eb20 2024-03-14T12:16:45.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd458b18640 1 -- 172.21.15.82:0/3812545725 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c02cc50 con 0x7fd45406eb20 2024-03-14T12:16:45.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd458b18640 1 -- 172.21.15.82:0/3812545725 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c034440 con 0x7fd45406eb20 2024-03-14T12:16:45.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3812545725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406eb20 msgr2=0x7fd45406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3812545725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406eb20 0x7fd45406efb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c02bea0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3812545725 shutdown_connections 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3812545725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406eb20 0x7fd45406efb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3812545725 >> v2:172.21.15.164:3300/0 conn(0x7fd45406b670 0x7fd454072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3812545725 >> 172.21.15.82:0/3812545725 conn(0x7fd4540d0b60 msgr2=0x7fd4540d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3812545725 shutdown_connections 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.075+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3812545725 wait complete. 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 Processor -- start 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 -- start start 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406b670 0x7fd4540b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd45406eb20 0x7fd454155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4540b0200 0x7fd4540c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540d0100 con 0x7fd4540b0200 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd454071a60 con 0x7fd45406eb20 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45bd7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540cb1b0 con 0x7fd45406b670 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45a31b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4540b0200 0x7fd4540c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45a31b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4540b0200 0x7fd4540c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54228/0 (socket says 172.21.15.82:54228) 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.077+0000 7fd45a31b640 1 -- 172.21.15.82:0/3409864126 learned_addr learned my addr 172.21.15.82:0/3409864126 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd459b1a640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406b670 0x7fd4540b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd459319640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd45406eb20 0x7fd454155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45a31b640 1 -- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406b670 msgr2=0x7fd4540b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45a31b640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406b670 0x7fd4540b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45a31b640 1 -- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd45406eb20 msgr2=0x7fd454155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45a31b640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd45406eb20 0x7fd454155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45a31b640 1 -- 172.21.15.82:0/3409864126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454155580 con 0x7fd4540b0200 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd459b1a640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406b670 0x7fd4540b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45a31b640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4540b0200 0x7fd4540c6f70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd44a7fc640 1 -- 172.21.15.82:0/3409864126 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c039050 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd44a7fc640 1 -- 172.21.15.82:0/3409864126 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c045070 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4540c74b0 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd44a7fc640 1 -- 172.21.15.82:0/3409864126 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c03e610 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.078+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4540b3ba0 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.079+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45404f050 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.079+0000 7fd44a7fc640 1 -- 172.21.15.82:0/3409864126 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd44c03e7b0 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.080+0000 7fd44a7fc640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd43806bbd0 0x7fd43806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.080+0000 7fd44a7fc640 1 -- 172.21.15.82:0/3409864126 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd44c0ad560 con 0x7fd4540b0200 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.080+0000 7fd459b1a640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd43806bbd0 0x7fd43806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.080+0000 7fd459b1a640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd43806bbd0 0x7fd43806e090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd450009ac0 tx=0x7fd450008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.084+0000 7fd44a7fc640 1 -- 172.21.15.82:0/3409864126 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd454155580 con 0x7fd4540b0200 2024-03-14T12:16:45.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.257+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4540ce710 con 0x7fd4540b0200 2024-03-14T12:16:45.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.258+0000 7fd44a7fc640 1 -- 172.21.15.82:0/3409864126 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16653 (secure 0 0 0) 0x7fd44c07c0c0 con 0x7fd4540b0200 2024-03-14T12:16:45.258 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:45.258 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":58,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:33.876540+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd43806bbd0 msgr2=0x7fd43806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd43806bbd0 0x7fd43806e090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd450009ac0 tx=0x7fd450008040 comp rx=0 tx=0).stop 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4540b0200 msgr2=0x7fd4540c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4540b0200 0x7fd4540c6f70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009d90 tx=0x7fd44c004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 shutdown_connections 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4540b0200 0x7fd4540c6f70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd45406eb20 0x7fd454155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd43806bbd0 0x7fd43806e090 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 --2- 172.21.15.82:0/3409864126 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd45406b670 0x7fd4540b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.260+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 >> 172.21.15.82:0/3409864126 conn(0x7fd4540d0b60 msgr2=0x7fd4540d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:45.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.261+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 shutdown_connections 2024-03-14T12:16:45.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.261+0000 7fd45bd7e640 1 -- 172.21.15.82:0/3409864126 wait complete. 2024-03-14T12:16:45.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:30.434900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:31.188753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:45.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:45.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.373+0000 7f28cddc4640 1 Processor -- start 2024-03-14T12:16:45.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.373+0000 7f28cddc4640 1 -- start start 2024-03-14T12:16:45.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.374+0000 7f28cddc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 0x7f28c80b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.374+0000 7f28cddc4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28c815c2a0 0x7f28c80cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.374+0000 7f28cddc4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80b2a50 con 0x7f28c80d0100 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.374+0000 7f28cddc4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80ce8e0 con 0x7f28c815c2a0 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.374+0000 7f28cddc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80ceb30 con 0x7f28c8145790 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.374+0000 7f28c7fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f28c80d0100 0x7f28c80707a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51688/0 (socket says 172.21.15.82:51688) 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.374+0000 7f28c7fff640 1 -- 172.21.15.82:0/2301990164 learned_addr learned my addr 172.21.15.82:0/2301990164 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c77fe640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 0x7f28c80b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654662032 0 0) 0x7f28c80b2a50 con 0x7f28c80d0100 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/2301990164 >> v2:172.21.15.164:3300/0 conn(0x7f28c815c2a0 0x7f28c80cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b0003520 con 0x7f28c80d0100 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3508644821 0 0) 0x7f28b0003520 con 0x7f28c80d0100 2024-03-14T12:16:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 msgr2=0x7f28c80b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 0x7f28c80b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 >> v2:172.21.15.164:3300/0 conn(0x7f28c815c2a0 msgr2=0x7f28c80cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 --2- 172.21.15.82:0/2301990164 >> v2:172.21.15.164:3300/0 conn(0x7f28c815c2a0 0x7f28c80cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80cee10 con 0x7f28c80d0100 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f28c00019f0 con 0x7f28c80d0100 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 >> v1:172.21.15.82:6789/0 conn(0x7f28c80d0100 legacy=0x7f28c80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b00039b0 0x7f28b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b0004800 0x7f28b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 0x7f28b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80ce8e0 con 0x7f28b0004800 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80ceb30 con 0x7f28b00039b0 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b000b9c0 con 0x7f28c8145790 2024-03-14T12:16:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b0004800 0x7f28b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c77fe640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b00039b0 0x7f28b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c7fff640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 0x7f28b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c6ffd640 1 -- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 msgr2=0x7f28b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c8145790 0x7f28b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c6ffd640 1 -- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b00039b0 msgr2=0x7f28b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.375+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b00039b0 0x7f28b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.376+0000 7f28c6ffd640 1 -- 172.21.15.82:0/2301990164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b0003520 con 0x7f28b0004800 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.376+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b0004800 0x7f28b0004c80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009af0 tx=0x7f28b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13310d73a6105339 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.376+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc039070 con 0x7f28b0004800 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.376+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc02c710 con 0x7f28b0004800 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.376+0000 7f28c67fc640 1 -- 172.21.15.82:0/2301990164 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc009690 con 0x7f28b0004800 2024-03-14T12:16:45.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.377+0000 7f28cddc4640 1 -- 172.21.15.82:0/2301990164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b0004800 msgr2=0x7f28b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.377+0000 7f28cddc4640 1 --2- 172.21.15.82:0/2301990164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b0004800 0x7f28b0004c80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009af0 tx=0x7f28b0005830 comp rx=0 tx=0).stop 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.377+0000 7f28c7fff640 1 -- 172.21.15.82:0/2301990164 reap_dead start 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.377+0000 7f28cddc4640 1 -- 172.21.15.82:0/2301990164 shutdown_connections 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.377+0000 7f28cddc4640 1 -- 172.21.15.82:0/2301990164 >> 172.21.15.82:0/2301990164 conn(0x7f28c806aef0 msgr2=0x7f28c806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.377+0000 7f28cddc4640 1 -- 172.21.15.82:0/2301990164 shutdown_connections 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.377+0000 7f28cddc4640 1 -- 172.21.15.82:0/2301990164 wait complete. 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.378+0000 7f28cddc4640 1 Processor -- start 2024-03-14T12:16:45.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.378+0000 7f28cddc4640 1 -- start start 2024-03-14T12:16:45.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28cddc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b0004800 0x7f28c8159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28cddc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c8142d70 0x7f28c81431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28cddc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c81f5180 0x7f28c81f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28cddc4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80cf0f0 con 0x7f28c8142d70 2024-03-14T12:16:45.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28cddc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80cc600 con 0x7f28b0004800 2024-03-14T12:16:45.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28cddc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80b4820 con 0x7f28c81f5180 2024-03-14T12:16:45.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c8142d70 0x7f28c81431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c8142d70 0x7f28c81431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54234/0 (socket says 172.21.15.82:54234) 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 -- 172.21.15.82:0/579348405 learned_addr learned my addr 172.21.15.82:0/579348405 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c77fe640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b0004800 0x7f28c8159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c7fff640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c81f5180 0x7f28c81f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 -- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c81f5180 msgr2=0x7f28c81f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c81f5180 0x7f28c81f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 -- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b0004800 msgr2=0x7f28c8159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b0004800 0x7f28c8159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 -- 172.21.15.82:0/579348405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c81f5eb0 con 0x7f28c8142d70 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.379+0000 7f28c6ffd640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c8142d70 0x7f28c81431f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009af0 tx=0x7f28bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.380+0000 7f28affff640 1 -- 172.21.15.82:0/579348405 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc042020 con 0x7f28c8142d70 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.380+0000 7f28affff640 1 -- 172.21.15.82:0/579348405 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc03a040 con 0x7f28c8142d70 2024-03-14T12:16:45.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.380+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c813f5f0 con 0x7f28c8142d70 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.380+0000 7f28affff640 1 -- 172.21.15.82:0/579348405 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc039070 con 0x7f28c8142d70 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.380+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c813fbb0 con 0x7f28c8142d70 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.381+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c804f050 con 0x7f28c8142d70 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.381+0000 7f28affff640 1 -- 172.21.15.82:0/579348405 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f28bc045020 con 0x7f28c8142d70 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.381+0000 7f28affff640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f28b406bc00 0x7f28b406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.381+0000 7f28affff640 1 -- 172.21.15.82:0/579348405 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28bc0abf90 con 0x7f28c8142d70 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.382+0000 7f28c77fe640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f28b406bc00 0x7f28b406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.382+0000 7f28c77fe640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f28b406bc00 0x7f28b406e0c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28c81e7d90 tx=0x7f28b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.386+0000 7f28affff640 1 -- 172.21.15.82:0/579348405 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f28bc04c070 con 0x7f28c8142d70 2024-03-14T12:16:45.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.554+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28c80cd490 con 0x7f28c8142d70 2024-03-14T12:16:45.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.555+0000 7f28affff640 1 -- 172.21.15.82:0/579348405 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28bc07aa80 con 0x7f28c8142d70 2024-03-14T12:16:45.554 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:45.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f28b406bc00 msgr2=0x7f28b406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f28b406bc00 0x7f28b406e0c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28c81e7d90 tx=0x7f28b8008040 comp rx=0 tx=0).stop 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c8142d70 msgr2=0x7f28c81431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c8142d70 0x7f28c81431f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009af0 tx=0x7f28bc004730 comp rx=0 tx=0).stop 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 shutdown_connections 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f28b406bc00 0x7f28b406e0c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28c81f5180 0x7f28c81f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28c8142d70 0x7f28c81431f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 --2- 172.21.15.82:0/579348405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b0004800 0x7f28c8159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 >> 172.21.15.82:0/579348405 conn(0x7f28c806aef0 msgr2=0x7f28c8144910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 shutdown_connections 2024-03-14T12:16:45.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.557+0000 7f28cddc4640 1 -- 172.21.15.82:0/579348405 wait complete. 2024-03-14T12:16:45.567 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:16:45.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.671+0000 7fc3b8473640 1 Processor -- start 2024-03-14T12:16:45.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.671+0000 7fc3b8473640 1 -- start start 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b8473640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc3b00cfe30 0x7fc3b0143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b8473640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3b015ee80 0x7fc3b0144520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b8473640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b00b34e0 con 0x7fc3b00d0210 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b8473640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b015f4a0 con 0x7fc3b015ee80 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b8473640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b00b3780 con 0x7fc3b00cfe30 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b6a10640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3b015ee80 0x7fc3b0144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b6a10640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3b015ee80 0x7fc3b0144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54236/0 (socket says 172.21.15.82:54236) 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.672+0000 7fc3b6a10640 1 -- 172.21.15.82:0/480446568 learned_addr learned my addr 172.21.15.82:0/480446568 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b620f640 1 --2- 172.21.15.82:0/480446568 >> v2:172.21.15.164:3300/0 conn(0x7fc3b00cfe30 0x7fc3b0143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b6a10640 1 -- 172.21.15.82:0/480446568 >> v2:172.21.15.164:3300/0 conn(0x7fc3b00cfe30 msgr2=0x7fc3b0143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b6a10640 1 --2- 172.21.15.82:0/480446568 >> v2:172.21.15.164:3300/0 conn(0x7fc3b00cfe30 0x7fc3b0143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b6a10640 1 -- 172.21.15.82:0/480446568 >> v1:172.21.15.164:6789/0 conn(0x7fc3b00d0210 legacy=0x7fc3b015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b6a10640 1 -- 172.21.15.82:0/480446568 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b00b3a60 con 0x7fc3b015ee80 2024-03-14T12:16:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b6a10640 1 --2- 172.21.15.82:0/480446568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b015ee80 0x7fc3b0144520 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00d0a60 tx=0x7fc3a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450b7bc455a86013 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b520d640 1 -- 172.21.15.82:0/480446568 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8035020 con 0x7fc3b015ee80 2024-03-14T12:16:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b520d640 1 -- 172.21.15.82:0/480446568 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8031870 con 0x7fc3b015ee80 2024-03-14T12:16:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.673+0000 7fc3b520d640 1 -- 172.21.15.82:0/480446568 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8031c40 con 0x7fc3b015ee80 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 -- 172.21.15.82:0/480446568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b015ee80 msgr2=0x7fc3b0144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 --2- 172.21.15.82:0/480446568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b015ee80 0x7fc3b0144520 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00d0a60 tx=0x7fc3a802b910 comp rx=0 tx=0).stop 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 -- 172.21.15.82:0/480446568 shutdown_connections 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 --2- 172.21.15.82:0/480446568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b015ee80 0x7fc3b0144520 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 --2- 172.21.15.82:0/480446568 >> v2:172.21.15.164:3300/0 conn(0x7fc3b00cfe30 0x7fc3b0143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 -- 172.21.15.82:0/480446568 >> 172.21.15.82:0/480446568 conn(0x7fc3b006aef0 msgr2=0x7fc3b006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 -- 172.21.15.82:0/480446568 shutdown_connections 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.674+0000 7fc3b8473640 1 -- 172.21.15.82:0/480446568 wait complete. 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.675+0000 7fc3b8473640 1 Processor -- start 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.675+0000 7fc3b8473640 1 -- start start 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b8473640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3b00cfe30 0x7fc3b01531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b8473640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b00d0210 0x7fc3b0154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b8473640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3b015ee80 0x7fc3b013a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b8473640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00b43e0 con 0x7fc3b00d0210 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b8473640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00b4260 con 0x7fc3b00cfe30 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b8473640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b006d870 con 0x7fc3b015ee80 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b00d0210 0x7fc3b0154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b00d0210 0x7fc3b0154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54238/0 (socket says 172.21.15.82:54238) 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 -- 172.21.15.82:0/547790265 learned_addr learned my addr 172.21.15.82:0/547790265 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b620f640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3b00cfe30 0x7fc3b01531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b6a10640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3b015ee80 0x7fc3b013a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 -- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3b015ee80 msgr2=0x7fc3b013a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3b015ee80 0x7fc3b013a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 -- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3b00cfe30 msgr2=0x7fc3b01531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3b00cfe30 0x7fc3b01531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 -- 172.21.15.82:0/547790265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b0154150 con 0x7fc3b00d0210 2024-03-14T12:16:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b620f640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3b00cfe30 0x7fc3b01531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.676+0000 7fc3b5a0e640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b00d0210 0x7fc3b0154d60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400c410 tx=0x7fc3a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.677+0000 7fc3aeffd640 1 -- 172.21.15.82:0/547790265 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4017070 con 0x7fc3b00d0210 2024-03-14T12:16:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.677+0000 7fc3aeffd640 1 -- 172.21.15.82:0/547790265 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a40035f0 con 0x7fc3b00d0210 2024-03-14T12:16:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.677+0000 7fc3aeffd640 1 -- 172.21.15.82:0/547790265 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40039e0 con 0x7fc3b00d0210 2024-03-14T12:16:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.677+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b01538b0 con 0x7fc3b00d0210 2024-03-14T12:16:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.677+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b0153e70 con 0x7fc3b00d0210 2024-03-14T12:16:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.678+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b013b460 con 0x7fc3b00d0210 2024-03-14T12:16:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.678+0000 7fc3aeffd640 1 -- 172.21.15.82:0/547790265 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc3a401a070 con 0x7fc3b00d0210 2024-03-14T12:16:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.678+0000 7fc3aeffd640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc39c06bbd0 0x7fc39c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.678+0000 7fc3aeffd640 1 -- 172.21.15.82:0/547790265 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a4089eb0 con 0x7fc3b00d0210 2024-03-14T12:16:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.678+0000 7fc3b620f640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc39c06bbd0 0x7fc39c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.679+0000 7fc3b620f640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc39c06bbd0 0x7fc39c06e090 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007f50 tx=0x7fc3a00074a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.683+0000 7fc3aeffd640 1 -- 172.21.15.82:0/547790265 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc3a4058a30 con 0x7fc3b00d0210 2024-03-14T12:16:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.853+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3b004f050 con 0x7fc3b00d0210 2024-03-14T12:16:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.853+0000 7fc3aeffd640 1 -- 172.21.15.82:0/547790265 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3a40010c0 con 0x7fc3b00d0210 2024-03-14T12:16:45.853 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc39c06bbd0 msgr2=0x7fc39c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc39c06bbd0 0x7fc39c06e090 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007f50 tx=0x7fc3a00074a0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b00d0210 msgr2=0x7fc3b0154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b00d0210 0x7fc3b0154d60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400c410 tx=0x7fc3a400c8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 shutdown_connections 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc39c06bbd0 0x7fc39c06e090 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3b015ee80 0x7fc3b013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3b00d0210 0x7fc3b0154d60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 --2- 172.21.15.82:0/547790265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3b00cfe30 0x7fc3b01531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 >> 172.21.15.82:0/547790265 conn(0x7fc3b006aef0 msgr2=0x7fc3b0070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 shutdown_connections 2024-03-14T12:16:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.855+0000 7fc3b8473640 1 -- 172.21.15.82:0/547790265 wait complete. 2024-03-14T12:16:45.865 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:16:45.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.969+0000 7fe2d423d640 1 Processor -- start 2024-03-14T12:16:45.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.969+0000 7fe2d423d640 1 -- start start 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d423d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2cc0d02d0 0x7fe2cc15e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d423d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0b41d0 0x7fe2cc15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d423d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2cc0b3450 con 0x7fe2cc0cf3f0 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d423d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2cc0cf7d0 con 0x7fe2cc0d02d0 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d423d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc15f200 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d27da640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0b41d0 0x7fe2cc15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d17d8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2cc0d02d0 0x7fe2cc15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d17d8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2cc0d02d0 0x7fe2cc15e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:33804/0 (socket says 172.21.15.82:33804) 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d27da640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0b41d0 0x7fe2cc15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:48932/0 (socket says 172.21.15.82:48932) 2024-03-14T12:16:45.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.970+0000 7fe2d17d8640 1 -- 172.21.15.82:0/892422486 learned_addr learned my addr 172.21.15.82:0/892422486 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:45.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.971+0000 7fe2d17d8640 1 -- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0b41d0 msgr2=0x7fe2cc15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:45.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.971+0000 7fe2d17d8640 1 --2- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0b41d0 0x7fe2cc15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.971+0000 7fe2d17d8640 1 -- 172.21.15.82:0/892422486 >> v1:172.21.15.164:6789/0 conn(0x7fe2cc0cf3f0 legacy=0x7fe2cc0cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.971+0000 7fe2d17d8640 1 -- 172.21.15.82:0/892422486 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc15f4b0 con 0x7fe2cc0d02d0 2024-03-14T12:16:45.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.971+0000 7fe2d27da640 1 --2- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0b41d0 0x7fe2cc15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d17d8640 1 --2- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 0x7fe2cc15e780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0cf950 tx=0x7fe2bc02e0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eee3454f5a35302 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d0fd7640 1 -- 172.21.15.82:0/892422486 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc03b070 con 0x7fe2cc0d02d0 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d0fd7640 1 -- 172.21.15.82:0/892422486 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2bc036440 con 0x7fe2cc0d02d0 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d0fd7640 1 -- 172.21.15.82:0/892422486 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc02ee60 con 0x7fe2cc0d02d0 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 -- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 msgr2=0x7fe2cc15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 --2- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 0x7fe2cc15e780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0cf950 tx=0x7fe2bc02e0b0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 -- 172.21.15.82:0/892422486 shutdown_connections 2024-03-14T12:16:45.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 --2- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0b41d0 0x7fe2cc15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 --2- 172.21.15.82:0/892422486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 0x7fe2cc15e780 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 -- 172.21.15.82:0/892422486 >> 172.21.15.82:0/892422486 conn(0x7fe2cc06aef0 msgr2=0x7fe2cc06b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:45.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 -- 172.21.15.82:0/892422486 shutdown_connections 2024-03-14T12:16:45.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.972+0000 7fe2d423d640 1 -- 172.21.15.82:0/892422486 wait complete. 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 Processor -- start 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 -- start start 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2cc0b41d0 0x7fe2cc0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0cf3f0 0x7fe2cc14e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 0x7fe2cc0ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0b3cb0 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0b39d0 con 0x7fe2cc0d02d0 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d423d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc06d890 con 0x7fe2cc0cf3f0 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d1fd9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2cc0b41d0 0x7fe2cc0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d1fd9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2cc0b41d0 0x7fe2cc0aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54246/0 (socket says 172.21.15.82:54246) 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d1fd9640 1 -- 172.21.15.82:0/1576344688 learned_addr learned my addr 172.21.15.82:0/1576344688 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d17d8640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0cf3f0 0x7fe2cc14e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.974+0000 7fe2d27da640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 0x7fe2cc0ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d1fd9640 1 -- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0cf3f0 msgr2=0x7fe2cc14e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d1fd9640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0cf3f0 0x7fe2cc14e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d1fd9640 1 -- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 msgr2=0x7fe2cc0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d1fd9640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 0x7fe2cc0ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d1fd9640 1 -- 172.21.15.82:0/1576344688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc145ad0 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d1fd9640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2cc0b41d0 0x7fe2cc0aef40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c790 tx=0x7fe2c800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2c27fc640 1 -- 172.21.15.82:0/1576344688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8016020 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2c27fc640 1 -- 172.21.15.82:0/1576344688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c8006d50 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc14d320 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2c27fc640 1 -- 172.21.15.82:0/1576344688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8003bb0 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.975+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc0ae460 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.976+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2cc06d060 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.976+0000 7fe2c27fc640 1 -- 172.21.15.82:0/1576344688 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe2c80050d0 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.977+0000 7fe2c27fc640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2b006bbd0 0x7fe2b006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.977+0000 7fe2d17d8640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2b006bbd0 0x7fe2b006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:45.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.977+0000 7fe2c27fc640 1 -- 172.21.15.82:0/1576344688 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2c808a7a0 con 0x7fe2cc0b41d0 2024-03-14T12:16:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.977+0000 7fe2d17d8640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2b006bbd0 0x7fe2b006e090 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc038000 tx=0x7fe2bc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:45.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:45.981+0000 7fe2c27fc640 1 -- 172.21.15.82:0/1576344688 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe2cc06d060 con 0x7fe2cc0b41d0 2024-03-14T12:16:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.154+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2cc04f050 con 0x7fe2cc0b41d0 2024-03-14T12:16:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.155+0000 7fe2c27fc640 1 -- 172.21.15.82:0/1576344688 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2c8020020 con 0x7fe2cc0b41d0 2024-03-14T12:16:46.154 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:16:46.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2b006bbd0 msgr2=0x7fe2b006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2b006bbd0 0x7fe2b006e090 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc038000 tx=0x7fe2bc008040 comp rx=0 tx=0).stop 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2cc0b41d0 msgr2=0x7fe2cc0aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2cc0b41d0 0x7fe2cc0aef40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c790 tx=0x7fe2c800cb90 comp rx=0 tx=0).stop 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 shutdown_connections 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2cc0d02d0 0x7fe2cc0ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe2b006bbd0 0x7fe2b006e090 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2cc0cf3f0 0x7fe2cc14e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 --2- 172.21.15.82:0/1576344688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2cc0b41d0 0x7fe2cc0aef40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 >> 172.21.15.82:0/1576344688 conn(0x7fe2cc06aef0 msgr2=0x7fe2cc0cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 shutdown_connections 2024-03-14T12:16:46.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.157+0000 7fe2d423d640 1 -- 172.21.15.82:0/1576344688 wait complete. 2024-03-14T12:16:46.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:46.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.268+0000 7f5eb6c0c640 1 Processor -- start 2024-03-14T12:16:46.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.268+0000 7f5eb6c0c640 1 -- start start 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb6c0c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5eb006b420 0x7f5eb00cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb6c0c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5eb00afff0 0x7f5eb00d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb6c0c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00b1340 con 0x7f5eb00b0f60 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb6c0c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00d0ee0 con 0x7f5eb00afff0 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb6c0c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00b4ee0 con 0x7f5eb006b420 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb51a9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5eb00afff0 0x7f5eb00d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb49a8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5eb00b0f60 0x7f5eb006ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51700/0 (socket says 172.21.15.82:51700) 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb49a8640 1 -- 172.21.15.82:0/3905971299 learned_addr learned my addr 172.21.15.82:0/3905971299 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.269+0000 7f5eb51a9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5eb00afff0 0x7f5eb00d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54254/0 (socket says 172.21.15.82:54254) 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eb51a9640 1 -- 172.21.15.82:0/3905971299 >> v2:172.21.15.164:3300/0 conn(0x7f5eb006b420 msgr2=0x7f5eb00cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eaffff640 1 --2- 172.21.15.82:0/3905971299 >> v2:172.21.15.164:3300/0 conn(0x7f5eb006b420 0x7f5eb00cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eaf7fe640 1 -- 172.21.15.82:0/3905971299 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235762570 0 0) 0x7f5eb00b1340 con 0x7f5eb00b0f60 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eb51a9640 1 --2- 172.21.15.82:0/3905971299 >> v2:172.21.15.164:3300/0 conn(0x7f5eb006b420 0x7f5eb00cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eb51a9640 1 -- 172.21.15.82:0/3905971299 >> v1:172.21.15.82:6789/0 conn(0x7f5eb00b0f60 legacy=0x7f5eb006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eb51a9640 1 -- 172.21.15.82:0/3905971299 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00b0480 con 0x7f5eb00afff0 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eaffff640 1 --2- 172.21.15.82:0/3905971299 >> v2:172.21.15.164:3300/0 conn(0x7f5eb006b420 0x7f5eb00cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eb51a9640 1 --2- 172.21.15.82:0/3905971299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00afff0 0x7f5eb00d09a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5eb009e320 tx=0x7f5ea802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f84937913cd958 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.270+0000 7f5eaf7fe640 1 -- 172.21.15.82:0/3905971299 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8039070 con 0x7f5eb00afff0 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eaf7fe640 1 -- 172.21.15.82:0/3905971299 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea802cc80 con 0x7f5eb00afff0 2024-03-14T12:16:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eaf7fe640 1 -- 172.21.15.82:0/3905971299 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8034660 con 0x7f5eb00afff0 2024-03-14T12:16:46.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/3905971299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00afff0 msgr2=0x7f5eb00d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/3905971299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00afff0 0x7f5eb00d09a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5eb009e320 tx=0x7f5ea802bed0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/3905971299 shutdown_connections 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/3905971299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00afff0 0x7f5eb00d09a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/3905971299 >> v2:172.21.15.164:3300/0 conn(0x7f5eb006b420 0x7f5eb00cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/3905971299 >> 172.21.15.82:0/3905971299 conn(0x7f5eb0072310 msgr2=0x7f5eb0072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/3905971299 shutdown_connections 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.271+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/3905971299 wait complete. 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.272+0000 7f5eb6c0c640 1 Processor -- start 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.272+0000 7f5eb6c0c640 1 -- start start 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb6c0c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb006b420 0x7f5eb00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb6c0c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00afff0 0x7f5eb00b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb6c0c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00b0f60 0x7f5eb00b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb6c0c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cd5f0 con 0x7f5eb00b0f60 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb6c0c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cd310 con 0x7f5eb00afff0 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb6c0c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cd8d0 con 0x7f5eb006b420 2024-03-14T12:16:46.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb51a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00b0f60 0x7f5eb00b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb51a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00b0f60 0x7f5eb00b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54256/0 (socket says 172.21.15.82:54256) 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb51a9640 1 -- 172.21.15.82:0/760717787 learned_addr learned my addr 172.21.15.82:0/760717787 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eb49a8640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb006b420 0x7f5eb00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.273+0000 7f5eaffff640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00afff0 0x7f5eb00b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb51a9640 1 -- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb006b420 msgr2=0x7f5eb00c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb51a9640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb006b420 0x7f5eb00c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb51a9640 1 -- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00afff0 msgr2=0x7f5eb00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb51a9640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00afff0 0x7f5eb00b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb51a9640 1 -- 172.21.15.82:0/760717787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00c3c50 con 0x7f5eb00b0f60 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb49a8640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb006b420 0x7f5eb00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eaffff640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00afff0 0x7f5eb00b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb51a9640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00b0f60 0x7f5eb00b1a90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5ea802c4e0 tx=0x7f5ea8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5ead7fa640 1 -- 172.21.15.82:0/760717787 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8039050 con 0x7f5eb00b0f60 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5ead7fa640 1 -- 172.21.15.82:0/760717787 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8045070 con 0x7f5eb00b0f60 2024-03-14T12:16:46.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb00c3f30 con 0x7f5eb00b0f60 2024-03-14T12:16:46.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5ead7fa640 1 -- 172.21.15.82:0/760717787 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea803e830 con 0x7f5eb00b0f60 2024-03-14T12:16:46.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.274+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb00c44f0 con 0x7f5eb00b0f60 2024-03-14T12:16:46.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.275+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb004f050 con 0x7f5eb00b0f60 2024-03-14T12:16:46.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.275+0000 7f5ead7fa640 1 -- 172.21.15.82:0/760717787 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5ea803e9d0 con 0x7f5eb00b0f60 2024-03-14T12:16:46.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.276+0000 7f5ead7fa640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5e9406bbd0 0x7f5e9406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.276+0000 7f5ead7fa640 1 -- 172.21.15.82:0/760717787 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ea80ad870 con 0x7f5eb00b0f60 2024-03-14T12:16:46.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.276+0000 7f5eb49a8640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5e9406bbd0 0x7f5e9406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.276+0000 7f5eb49a8640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5e9406bbd0 0x7f5e9406e090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000f60 tx=0x7f5ea4007ca0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.280+0000 7f5ead7fa640 1 -- 172.21.15.82:0/760717787 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5ea807c360 con 0x7f5eb00b0f60 2024-03-14T12:16:46.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.490+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eb006adb0 con 0x7f5eb00b0f60 2024-03-14T12:16:46.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.491+0000 7f5ead7fa640 1 -- 172.21.15.82:0/760717787 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f5ea807bc60 con 0x7f5eb00b0f60 2024-03-14T12:16:46.490 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:46.490 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5111,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/4078909817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4078909817},{"type":"v1","addr":"172.21.15.164:6834","nonce":4078909817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2349002857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2349002857},{"type":"v1","addr":"172.21.15.164:6835","nonce":2349002857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/285826310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":285826310},{"type":"v1","addr":"172.21.15.82:6835","nonce":285826310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:32.222787+0000","modified":"2024-03-14T12:16:34.220226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1422543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1422543802},{"type":"v1","addr":"172.21.15.82:6837","nonce":1422543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T12:16:46.491 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-03-14T12:16:46.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5e9406bbd0 msgr2=0x7f5e9406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5e9406bbd0 0x7f5e9406e090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000f60 tx=0x7f5ea4007ca0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00b0f60 msgr2=0x7f5eb00b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00b0f60 0x7f5eb00b1a90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5ea802c4e0 tx=0x7f5ea8004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 shutdown_connections 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5eb00b0f60 0x7f5eb00b1a90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00afff0 0x7f5eb00b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f5e9406bbd0 0x7f5e9406e090 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 --2- 172.21.15.82:0/760717787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb006b420 0x7f5eb00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 >> 172.21.15.82:0/760717787 conn(0x7f5eb0072310 msgr2=0x7f5eb00acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 shutdown_connections 2024-03-14T12:16:46.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.493+0000 7f5eb6c0c640 1 -- 172.21.15.82:0/760717787 wait complete. 2024-03-14T12:16:46.503 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:46.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.604+0000 7f178804a640 1 Processor -- start 2024-03-14T12:16:46.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.604+0000 7f178804a640 1 -- start start 2024-03-14T12:16:46.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f178804a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1780071740 0x7f1780071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f178804a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f178015e670 0x7f17800b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f178804a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1780072060 con 0x7f1780143f60 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f178804a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178015ebe0 con 0x7f1780071740 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f178804a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178015eec0 con 0x7f178015e670 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f1785de6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1780071740 0x7f1780071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f1785de6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1780071740 0x7f1780071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54260/0 (socket says 172.21.15.82:54260) 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f1785de6640 1 -- 172.21.15.82:0/1312438544 learned_addr learned my addr 172.21.15.82:0/1312438544 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f17855e5640 1 --2- 172.21.15.82:0/1312438544 >> v2:172.21.15.164:3300/0 conn(0x7f178015e670 0x7f17800b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.605+0000 7f1785de6640 1 -- 172.21.15.82:0/1312438544 >> v2:172.21.15.164:3300/0 conn(0x7f178015e670 msgr2=0x7f17800b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1785de6640 1 --2- 172.21.15.82:0/1312438544 >> v2:172.21.15.164:3300/0 conn(0x7f178015e670 0x7f17800b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1785de6640 1 -- 172.21.15.82:0/1312438544 >> v1:172.21.15.82:6789/0 conn(0x7f1780143f60 legacy=0x7f178015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1785de6640 1 -- 172.21.15.82:0/1312438544 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178015f1a0 con 0x7f1780071740 2024-03-14T12:16:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1784de4640 1 -- 172.21.15.82:0/1312438544 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502124022 0 0) 0x7f1780072060 con 0x7f1780143f60 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1785de6640 1 --2- 172.21.15.82:0/1312438544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780071740 0x7f1780071b20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1780072ac0 tx=0x7f177c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f132811092ef2c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1784de4640 1 -- 172.21.15.82:0/1312438544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c02cb20 con 0x7f1780071740 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1784de4640 1 -- 172.21.15.82:0/1312438544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c02ccc0 con 0x7f1780071740 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.606+0000 7f1784de4640 1 -- 172.21.15.82:0/1312438544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c034610 con 0x7f1780071740 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 -- 172.21.15.82:0/1312438544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780071740 msgr2=0x7f1780071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 --2- 172.21.15.82:0/1312438544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780071740 0x7f1780071b20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1780072ac0 tx=0x7f177c02bf80 comp rx=0 tx=0).stop 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 -- 172.21.15.82:0/1312438544 shutdown_connections 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 --2- 172.21.15.82:0/1312438544 >> v2:172.21.15.164:3300/0 conn(0x7f178015e670 0x7f17800b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 --2- 172.21.15.82:0/1312438544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780071740 0x7f1780071b20 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 -- 172.21.15.82:0/1312438544 >> 172.21.15.82:0/1312438544 conn(0x7f17800b1c70 msgr2=0x7f17800cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 -- 172.21.15.82:0/1312438544 shutdown_connections 2024-03-14T12:16:46.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.607+0000 7f178804a640 1 -- 172.21.15.82:0/1312438544 wait complete. 2024-03-14T12:16:46.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.608+0000 7f178804a640 1 Processor -- start 2024-03-14T12:16:46.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.608+0000 7f178804a640 1 -- start start 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f178804a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1780071740 0x7f17801f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f178804a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1780143f60 0x7f17801f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f178804a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178015e670 0x7f178013cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f178804a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178015b080 con 0x7f178015e670 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f178804a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1780143ce0 con 0x7f1780071740 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f178804a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1780144670 con 0x7f1780143f60 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178015e670 0x7f178013cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178015e670 0x7f178013cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54266/0 (socket says 172.21.15.82:54266) 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 -- 172.21.15.82:0/2934897968 learned_addr learned my addr 172.21.15.82:0/2934897968 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f1785de6640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1780071740 0x7f17801f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17855e5640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1780143f60 0x7f17801f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 -- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1780143f60 msgr2=0x7f17801f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1780143f60 0x7f17801f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 -- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1780071740 msgr2=0x7f17801f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1780071740 0x7f17801f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 -- 172.21.15.82:0/2934897968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17801f5000 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.609+0000 7f17865e7640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178015e670 0x7f178013cc30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f177800c480 tx=0x7f177800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.610+0000 7f17767fc640 1 -- 172.21.15.82:0/2934897968 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778005480 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.610+0000 7f17767fc640 1 -- 172.21.15.82:0/2934897968 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17780035f0 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.610+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178015bfa0 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.610+0000 7f17767fc640 1 -- 172.21.15.82:0/2934897968 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778003a00 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.610+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178015c520 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.611+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178004f050 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.611+0000 7f17767fc640 1 -- 172.21.15.82:0/2934897968 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1778003ba0 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.611+0000 7f17767fc640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f176406bd70 0x7f176406e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.611+0000 7f17767fc640 1 -- 172.21.15.82:0/2934897968 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f177808a680 con 0x7f178015e670 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.611+0000 7f1785de6640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f176406bd70 0x7f176406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.612+0000 7f1785de6640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f176406bd70 0x7f176406e230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f177c035000 tx=0x7f177c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.616+0000 7f17767fc640 1 -- 172.21.15.82:0/2934897968 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1778059200 con 0x7f178015e670 2024-03-14T12:16:46.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.788+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17800d07b0 con 0x7f178015e670 2024-03-14T12:16:46.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.789+0000 7f17767fc640 1 -- 172.21.15.82:0/2934897968 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f1778058b00 con 0x7f178015e670 2024-03-14T12:16:46.789 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:46.789 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":58,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:33.876540+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f176406bd70 msgr2=0x7f176406e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f176406bd70 0x7f176406e230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f177c035000 tx=0x7f177c030000 comp rx=0 tx=0).stop 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178015e670 msgr2=0x7f178013cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178015e670 0x7f178013cc30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f177800c480 tx=0x7f177800c940 comp rx=0 tx=0).stop 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 shutdown_connections 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f178015e670 0x7f178013cc30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f176406bd70 0x7f176406e230 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1780143f60 0x7f17801f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 --2- 172.21.15.82:0/2934897968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1780071740 0x7f17801f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 >> 172.21.15.82:0/2934897968 conn(0x7f17800b1c70 msgr2=0x7f1780071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:46.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 shutdown_connections 2024-03-14T12:16:46.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.791+0000 7f178804a640 1 -- 172.21.15.82:0/2934897968 wait complete. 2024-03-14T12:16:46.802 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T12:16:46.802 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:46.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.903+0000 7fc592f0a640 1 Processor -- start 2024-03-14T12:16:46.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.903+0000 7fc592f0a640 1 -- start start 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc592f0a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc592f0a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c15ebe0 con 0x7fc58c143f60 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc592f0a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c15eec0 con 0x7fc58c071740 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc592f0a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c15f1a0 con 0x7fc58c15e670 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc5914a7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc58c143f60 0x7fc58c15b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51720/0 (socket says 172.21.15.82:51720) 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc5914a7640 1 -- 172.21.15.82:0/1210640667 learned_addr learned my addr 172.21.15.82:0/1210640667 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc58bfff640 1 --2- 172.21.15.82:0/1210640667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc58b7fe640 1 -- 172.21.15.82:0/1210640667 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786270592 0 0) 0x7fc58c15ebe0 con 0x7fc58c143f60 2024-03-14T12:16:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc58b7fe640 1 -- 172.21.15.82:0/1210640667 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc57c003520 con 0x7fc58c143f60 2024-03-14T12:16:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc58b7fe640 1 -- 172.21.15.82:0/1210640667 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895561746 0 0) 0x7fc58c15eec0 con 0x7fc58c071740 2024-03-14T12:16:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc58bfff640 1 -- 172.21.15.82:0/1210640667 >> v1:172.21.15.164:6789/0 conn(0x7fc58c071740 legacy=0x7fc58c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc58bfff640 1 -- 172.21.15.82:0/1210640667 >> v1:172.21.15.82:6789/0 conn(0x7fc58c143f60 legacy=0x7fc58c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.904+0000 7fc58bfff640 1 -- 172.21.15.82:0/1210640667 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c15f480 con 0x7fc58c15e670 2024-03-14T12:16:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.905+0000 7fc58bfff640 1 --2- 172.21.15.82:0/1210640667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c0b2940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5780097d0 tx=0x7fc57802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b87c3bf2e6b5ad0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.905+0000 7fc58b7fe640 1 -- 172.21.15.82:0/1210640667 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578035020 con 0x7fc58c15e670 2024-03-14T12:16:46.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.905+0000 7fc58b7fe640 1 -- 172.21.15.82:0/1210640667 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc578031890 con 0x7fc58c15e670 2024-03-14T12:16:46.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.905+0000 7fc58b7fe640 1 -- 172.21.15.82:0/1210640667 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578031c40 con 0x7fc58c15e670 2024-03-14T12:16:46.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.906+0000 7fc592f0a640 1 -- 172.21.15.82:0/1210640667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 msgr2=0x7fc58c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.906+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1210640667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c0b2940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5780097d0 tx=0x7fc57802b930 comp rx=0 tx=0).stop 2024-03-14T12:16:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.906+0000 7fc592f0a640 1 -- 172.21.15.82:0/1210640667 shutdown_connections 2024-03-14T12:16:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.906+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1210640667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c0b2940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.906+0000 7fc592f0a640 1 -- 172.21.15.82:0/1210640667 >> 172.21.15.82:0/1210640667 conn(0x7fc58c0b1c70 msgr2=0x7fc58c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.906+0000 7fc592f0a640 1 -- 172.21.15.82:0/1210640667 shutdown_connections 2024-03-14T12:16:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.906+0000 7fc592f0a640 1 -- 172.21.15.82:0/1210640667 wait complete. 2024-03-14T12:16:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.907+0000 7fc592f0a640 1 Processor -- start 2024-03-14T12:16:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.907+0000 7fc592f0a640 1 -- start start 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc592f0a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c071740 0x7fc58c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc592f0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c143f60 0x7fc58c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc592f0a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc592f0a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c144670 con 0x7fc58c071740 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc592f0a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c15b080 con 0x7fc58c143f60 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc592f0a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c1447f0 con 0x7fc58c15e670 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c071740 0x7fc58c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c071740 0x7fc58c1596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54276/0 (socket says 172.21.15.82:54276) 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 -- 172.21.15.82:0/1540417937 learned_addr learned my addr 172.21.15.82:0/1540417937 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc5914a7640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc58bfff640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c143f60 0x7fc58c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 -- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 msgr2=0x7fc58c1f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c1f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 -- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c143f60 msgr2=0x7fc58c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c143f60 0x7fc58c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 -- 172.21.15.82:0/1540417937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c15d160 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.908+0000 7fc590ca6640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c071740 0x7fc58c1596d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c410 tx=0x7fc58000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.909+0000 7fc5897fa640 1 -- 172.21.15.82:0/1540417937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580017070 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.909+0000 7fc5897fa640 1 -- 172.21.15.82:0/1540417937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5800035f0 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.909+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c15d440 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.909+0000 7fc5897fa640 1 -- 172.21.15.82:0/1540417937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800039e0 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.909+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c15da00 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.910+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c04f050 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.910+0000 7fc5897fa640 1 -- 172.21.15.82:0/1540417937 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc58001a070 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.910+0000 7fc5897fa640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc57006bbd0 0x7fc57006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.911+0000 7fc5897fa640 1 -- 172.21.15.82:0/1540417937 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc580089e20 con 0x7fc58c071740 2024-03-14T12:16:46.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.911+0000 7fc58bfff640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc57006bbd0 0x7fc57006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.911+0000 7fc58bfff640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc57006bbd0 0x7fc57006e090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc578036000 tx=0x7fc57803d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:46.915+0000 7fc5897fa640 1 -- 172.21.15.82:0/1540417937 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc5800582c0 con 0x7fc58c071740 2024-03-14T12:16:47.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.120+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc58c15a7c0 con 0x7fc58c071740 2024-03-14T12:16:47.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.121+0000 7fc5897fa640 1 -- 172.21.15.82:0/1540417937 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fc58005c360 con 0x7fc58c071740 2024-03-14T12:16:47.121 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:47.121 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5111,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/4078909817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4078909817},{"type":"v1","addr":"172.21.15.164:6834","nonce":4078909817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2349002857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2349002857},{"type":"v1","addr":"172.21.15.164:6835","nonce":2349002857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/285826310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":285826310},{"type":"v1","addr":"172.21.15.82:6835","nonce":285826310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:32.222787+0000","modified":"2024-03-14T12:16:34.220226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1422543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1422543802},{"type":"v1","addr":"172.21.15.82:6837","nonce":1422543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T12:16:47.121 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc57006bbd0 msgr2=0x7fc57006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc57006bbd0 0x7fc57006e090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc578036000 tx=0x7fc57803d040 comp rx=0 tx=0).stop 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c071740 msgr2=0x7fc58c1596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c071740 0x7fc58c1596d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc58000c410 tx=0x7fc58000c8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 shutdown_connections 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc57006bbd0 0x7fc57006e090 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc58c15e670 0x7fc58c1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c143f60 0x7fc58c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 --2- 172.21.15.82:0/1540417937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58c071740 0x7fc58c1596d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 >> 172.21.15.82:0/1540417937 conn(0x7fc58c0b1c70 msgr2=0x7fc58c0b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 shutdown_connections 2024-03-14T12:16:47.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.123+0000 7fc592f0a640 1 -- 172.21.15.82:0/1540417937 wait complete. 2024-03-14T12:16:47.134 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:47.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.234+0000 7f66477cd640 1 Processor -- start 2024-03-14T12:16:47.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.235+0000 7f66477cd640 1 -- start start 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.235+0000 7f66477cd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f66400b4220 0x7f6640161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.235+0000 7f66477cd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6640162010 0x7f664006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.235+0000 7f66477cd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664006fa90 con 0x7f664015e150 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.235+0000 7f66477cd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664006c750 con 0x7f6640162010 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.235+0000 7f66477cd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664006be20 con 0x7f66400b4220 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.235+0000 7f6645d6a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6640162010 0x7f664006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.236+0000 7f6645d6a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6640162010 0x7f664006cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54284/0 (socket says 172.21.15.82:54284) 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.236+0000 7f6645d6a640 1 -- 172.21.15.82:0/1173041467 learned_addr learned my addr 172.21.15.82:0/1173041467 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:47.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.236+0000 7f6644d68640 1 --2- 172.21.15.82:0/1173041467 >> v2:172.21.15.164:3300/0 conn(0x7f66400b4220 0x7f6640161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.236+0000 7f6645d6a640 1 -- 172.21.15.82:0/1173041467 >> v2:172.21.15.164:3300/0 conn(0x7f66400b4220 msgr2=0x7f6640161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.236+0000 7f6645d6a640 1 --2- 172.21.15.82:0/1173041467 >> v2:172.21.15.164:3300/0 conn(0x7f66400b4220 0x7f6640161ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.236+0000 7f6645d6a640 1 -- 172.21.15.82:0/1173041467 >> v1:172.21.15.164:6789/0 conn(0x7f664015e150 legacy=0x7f664006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.236+0000 7f6645d6a640 1 -- 172.21.15.82:0/1173041467 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664006c100 con 0x7f6640162010 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f6645d6a640 1 --2- 172.21.15.82:0/1173041467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6640162010 0x7f664006cbe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6638009e50 tx=0x7f663802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863812effde89e98 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f6637fff640 1 -- 172.21.15.82:0/1173041467 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663802cb20 con 0x7f6640162010 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f6637fff640 1 -- 172.21.15.82:0/1173041467 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663802ccc0 con 0x7f6640162010 2024-03-14T12:16:47.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f6637fff640 1 -- 172.21.15.82:0/1173041467 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638034610 con 0x7f6640162010 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 -- 172.21.15.82:0/1173041467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6640162010 msgr2=0x7f664006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 --2- 172.21.15.82:0/1173041467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6640162010 0x7f664006cbe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6638009e50 tx=0x7f663802bf80 comp rx=0 tx=0).stop 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 -- 172.21.15.82:0/1173041467 shutdown_connections 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 --2- 172.21.15.82:0/1173041467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6640162010 0x7f664006cbe0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 --2- 172.21.15.82:0/1173041467 >> v2:172.21.15.164:3300/0 conn(0x7f66400b4220 0x7f6640161ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 -- 172.21.15.82:0/1173041467 >> 172.21.15.82:0/1173041467 conn(0x7f66400b3670 msgr2=0x7f66400b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 -- 172.21.15.82:0/1173041467 shutdown_connections 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.237+0000 7f66477cd640 1 -- 172.21.15.82:0/1173041467 wait complete. 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.238+0000 7f66477cd640 1 Processor -- start 2024-03-14T12:16:47.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.238+0000 7f66477cd640 1 -- start start 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f66477cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66400b4220 0x7f66400cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f66477cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f664015e150 0x7f66400b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f66477cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6640162010 0x7f66400b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f66477cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664006d2a0 con 0x7f66400b4220 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f66477cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664006d120 con 0x7f664015e150 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f66477cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664006d420 con 0x7f6640162010 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f6645569640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66400b4220 0x7f66400cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f6645569640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66400b4220 0x7f66400cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54286/0 (socket says 172.21.15.82:54286) 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f6645569640 1 -- 172.21.15.82:0/2811079497 learned_addr learned my addr 172.21.15.82:0/2811079497 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f6645d6a640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6640162010 0x7f66400b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.239+0000 7f6644d68640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f664015e150 0x7f66400b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6645569640 1 -- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6640162010 msgr2=0x7f66400b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6645569640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6640162010 0x7f66400b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6645569640 1 -- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f664015e150 msgr2=0x7f66400b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6645569640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f664015e150 0x7f66400b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6645569640 1 -- 172.21.15.82:0/2811079497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66400cc430 con 0x7f66400b4220 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6645d6a640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6640162010 0x7f66400b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6644d68640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f664015e150 0x7f66400b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6645569640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66400b4220 0x7f66400cddf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f663c00c400 tx=0x7f663c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6635ffb640 1 -- 172.21.15.82:0/2811079497 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c0052d0 con 0x7f66400b4220 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6635ffb640 1 -- 172.21.15.82:0/2811079497 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c005490 con 0x7f66400b4220 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66400b5520 con 0x7f66400b4220 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f6635ffb640 1 -- 172.21.15.82:0/2811079497 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c007260 con 0x7f66400b4220 2024-03-14T12:16:47.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.240+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66400b5ae0 con 0x7f66400b4220 2024-03-14T12:16:47.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.241+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66400aebf0 con 0x7f66400b4220 2024-03-14T12:16:47.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.241+0000 7f6635ffb640 1 -- 172.21.15.82:0/2811079497 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f663c007400 con 0x7f66400b4220 2024-03-14T12:16:47.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.242+0000 7f6635ffb640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f662406bbd0 0x7f662406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.242+0000 7f6644d68640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f662406bbd0 0x7f662406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.242+0000 7f6635ffb640 1 -- 172.21.15.82:0/2811079497 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f663c08a070 con 0x7f66400b4220 2024-03-14T12:16:47.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.242+0000 7f6644d68640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f662406bbd0 0x7f662406e090 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66400cc050 tx=0x7f6630008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.246+0000 7f6635ffb640 1 -- 172.21.15.82:0/2811079497 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f66400aebf0 con 0x7f66400b4220 2024-03-14T12:16:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.452+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66400b7ae0 con 0x7f66400b4220 2024-03-14T12:16:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.452+0000 7f6635ffb640 1 -- 172.21.15.82:0/2811079497 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f663c058ad0 con 0x7f66400b4220 2024-03-14T12:16:47.452 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5111,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/4078909817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4078909817},{"type":"v1","addr":"172.21.15.164:6834","nonce":4078909817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2349002857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2349002857},{"type":"v1","addr":"172.21.15.164:6835","nonce":2349002857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/285826310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":285826310},{"type":"v1","addr":"172.21.15.82:6835","nonce":285826310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:32.222787+0000","modified":"2024-03-14T12:16:34.220226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1422543802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1422543802},{"type":"v1","addr":"172.21.15.82:6837","nonce":1422543802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T12:16:47.452 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-03-14T12:16:47.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f662406bbd0 msgr2=0x7f662406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f662406bbd0 0x7f662406e090 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66400cc050 tx=0x7f6630008040 comp rx=0 tx=0).stop 2024-03-14T12:16:47.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66400b4220 msgr2=0x7f66400cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66400b4220 0x7f66400cddf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f663c00c400 tx=0x7f663c00c8c0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 shutdown_connections 2024-03-14T12:16:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f662406bbd0 0x7f662406e090 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6640162010 0x7f66400b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f664015e150 0x7f66400b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 --2- 172.21.15.82:0/2811079497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66400b4220 0x7f66400cddf0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 >> 172.21.15.82:0/2811079497 conn(0x7f66400b3670 msgr2=0x7f66400af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 shutdown_connections 2024-03-14T12:16:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.454+0000 7f66477cd640 1 -- 172.21.15.82:0/2811079497 wait complete. 2024-03-14T12:16:47.465 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:47.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.566+0000 7feef02a0640 1 Processor -- start 2024-03-14T12:16:47.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.566+0000 7feef02a0640 1 -- start start 2024-03-14T12:16:47.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.567+0000 7feef02a0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80cf370 0x7feee80d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.567+0000 7feef02a0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80cf8c0 con 0x7feee80b7720 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.567+0000 7feef02a0640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80cfba0 con 0x7feee80d0760 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.567+0000 7feef02a0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee80b6c40 con 0x7feee80cf370 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeee83d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feee80b7720 0x7feee80b7bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51730/0 (socket says 172.21.15.82:51730) 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeee83d640 1 -- 172.21.15.82:0/858335827 learned_addr learned my addr 172.21.15.82:0/858335827 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed83b640 1 --2- 172.21.15.82:0/858335827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80cf370 0x7feee80d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210416223 0 0) 0x7feee80cf8c0 con 0x7feee80b7720 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed4003520 con 0x7feee80b7720 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749150931 0 0) 0x7feee80cfba0 con 0x7feee80d0760 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee80cf8c0 con 0x7feee80d0760 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed83b640 1 -- 172.21.15.82:0/858335827 >> v1:172.21.15.164:6789/0 conn(0x7feee80d0760 legacy=0x7feee80d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 583217021 0 0) 0x7feed4003520 con 0x7feee80b7720 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed83b640 1 -- 172.21.15.82:0/858335827 >> v1:172.21.15.82:6789/0 conn(0x7feee80b7720 legacy=0x7feee80b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.568+0000 7feeed83b640 1 -- 172.21.15.82:0/858335827 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80b6f20 con 0x7feee80cf370 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.569+0000 7feeed83b640 1 --2- 172.21.15.82:0/858335827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80cf370 0x7feee80d4660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feed80097d0 tx=0x7feed802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e37ac3a5963f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.569+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00019f0 con 0x7feee80cf370 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.569+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed8037070 con 0x7feee80cf370 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.569+0000 7feeed03a640 1 -- 172.21.15.82:0/858335827 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8031780 con 0x7feee80cf370 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.570+0000 7feef02a0640 1 -- 172.21.15.82:0/858335827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80cf370 msgr2=0x7feee80d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.570+0000 7feef02a0640 1 --2- 172.21.15.82:0/858335827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80cf370 0x7feee80d4660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feed80097d0 tx=0x7feed802b8a0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.570+0000 7feef02a0640 1 -- 172.21.15.82:0/858335827 shutdown_connections 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.570+0000 7feef02a0640 1 --2- 172.21.15.82:0/858335827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80cf370 0x7feee80d4660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.570+0000 7feef02a0640 1 -- 172.21.15.82:0/858335827 >> 172.21.15.82:0/858335827 conn(0x7feee809d030 msgr2=0x7feee80d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.570+0000 7feef02a0640 1 -- 172.21.15.82:0/858335827 shutdown_connections 2024-03-14T12:16:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.570+0000 7feef02a0640 1 -- 172.21.15.82:0/858335827 wait complete. 2024-03-14T12:16:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.571+0000 7feef02a0640 1 Processor -- start 2024-03-14T12:16:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.571+0000 7feef02a0640 1 -- start start 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.571+0000 7feef02a0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80b7720 0x7feee80c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feef02a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feee80cf370 0x7feee80cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feef02a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80d0760 0x7feee815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feef02a0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee81617a0 con 0x7feee80d0760 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feef02a0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80d4ba0 con 0x7feee80cf370 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feef02a0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee8161920 con 0x7feee80b7720 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80d0760 0x7feee815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80d0760 0x7feee815ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54288/0 (socket says 172.21.15.82:54288) 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 -- 172.21.15.82:0/3914085399 learned_addr learned my addr 172.21.15.82:0/3914085399 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee03c640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80b7720 0x7feee80c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeed83b640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feee80cf370 0x7feee80cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 -- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80b7720 msgr2=0x7feee80c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80b7720 0x7feee80c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 -- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feee80cf370 msgr2=0x7feee80cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feee80cf370 0x7feee80cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 -- 172.21.15.82:0/3914085399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80cbd70 con 0x7feee80d0760 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feeee83d640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80d0760 0x7feee815ad60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7feee0001f30 tx=0x7feee000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feedeffd640 1 -- 172.21.15.82:0/3914085399 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0017070 con 0x7feee80d0760 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feedeffd640 1 -- 172.21.15.82:0/3914085399 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0012400 con 0x7feee80d0760 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feedeffd640 1 -- 172.21.15.82:0/3914085399 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00127f0 con 0x7feee80d0760 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee80cbff0 con 0x7feee80d0760 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.572+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee80cc5b0 con 0x7feee80d0760 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.573+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee804f050 con 0x7feee80d0760 2024-03-14T12:16:47.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.574+0000 7feedeffd640 1 -- 172.21.15.82:0/3914085399 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7feee001a070 con 0x7feee80d0760 2024-03-14T12:16:47.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.574+0000 7feedeffd640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feecc06bbd0 0x7feecc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.574+0000 7feedeffd640 1 -- 172.21.15.82:0/3914085399 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feee008a950 con 0x7feee80d0760 2024-03-14T12:16:47.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.574+0000 7feeee03c640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feecc06bbd0 0x7feecc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.575+0000 7feeee03c640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feecc06bbd0 0x7feecc06e090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feee40011e0 tx=0x7feee4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.579+0000 7feedeffd640 1 -- 172.21.15.82:0/3914085399 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7feee80cbd70 con 0x7feee80d0760 2024-03-14T12:16:47.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.752+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feee80d0540 con 0x7feee80d0760 2024-03-14T12:16:47.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.753+0000 7feedeffd640 1 -- 172.21.15.82:0/3914085399 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16653 (secure 0 0 0) 0x7feee0058df0 con 0x7feee80d0760 2024-03-14T12:16:47.753 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:47.753 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":58,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:33.876540+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:30.434900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:31.188753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:47.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feecc06bbd0 msgr2=0x7feecc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feecc06bbd0 0x7feecc06e090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feee40011e0 tx=0x7feee4009040 comp rx=0 tx=0).stop 2024-03-14T12:16:47.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80d0760 msgr2=0x7feee815ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80d0760 0x7feee815ad60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7feee0001f30 tx=0x7feee000a430 comp rx=0 tx=0).stop 2024-03-14T12:16:47.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 shutdown_connections 2024-03-14T12:16:47.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80d0760 0x7feee815ad60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feee80cf370 0x7feee80cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feecc06bbd0 0x7feecc06e090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 --2- 172.21.15.82:0/3914085399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feee80b7720 0x7feee80c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 >> 172.21.15.82:0/3914085399 conn(0x7feee809d030 msgr2=0x7feee80d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:47.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 shutdown_connections 2024-03-14T12:16:47.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.755+0000 7feef02a0640 1 -- 172.21.15.82:0/3914085399 wait complete. 2024-03-14T12:16:47.766 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T12:16:47.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.868+0000 7f1e27ee9640 1 Processor -- start 2024-03-14T12:16:47.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e27ee9640 1 -- start start 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e27ee9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e20151230 0x7f1e200cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e27ee9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e2015dde0 0x7f1e2015e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e27ee9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e200b3da0 con 0x7f1e200b4b70 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e27ee9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e20070d40 con 0x7f1e2015dde0 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e27ee9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e2006c3b0 con 0x7f1e20151230 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e26486640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e2015dde0 0x7f1e2015e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e26486640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e2015dde0 0x7f1e2015e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54298/0 (socket says 172.21.15.82:54298) 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e25484640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1e200b4b70 0x7f1e20070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51744/0 (socket says 172.21.15.82:51744) 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.869+0000 7f1e26486640 1 -- 172.21.15.82:0/2855833614 learned_addr learned my addr 172.21.15.82:0/2855833614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.870+0000 7f1e25c85640 1 --2- 172.21.15.82:0/2855833614 >> v2:172.21.15.164:3300/0 conn(0x7f1e20151230 0x7f1e200cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.870+0000 7f1e26486640 1 -- 172.21.15.82:0/2855833614 >> v2:172.21.15.164:3300/0 conn(0x7f1e20151230 msgr2=0x7f1e200cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.870+0000 7f1e26486640 1 --2- 172.21.15.82:0/2855833614 >> v2:172.21.15.164:3300/0 conn(0x7f1e20151230 0x7f1e200cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.870+0000 7f1e26486640 1 -- 172.21.15.82:0/2855833614 >> v1:172.21.15.82:6789/0 conn(0x7f1e200b4b70 legacy=0x7f1e20070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.870+0000 7f1e26486640 1 -- 172.21.15.82:0/2855833614 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200b4080 con 0x7f1e2015dde0 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.870+0000 7f1e24c83640 1 -- 172.21.15.82:0/2855833614 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383119732 0 0) 0x7f1e200b3da0 con 0x7f1e200b4b70 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e26486640 1 --2- 172.21.15.82:0/2855833614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2015dde0 0x7f1e2015e210 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b2340 tx=0x7f1e1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5877d65ee55b8cf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e24c83640 1 -- 172.21.15.82:0/2855833614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18034070 con 0x7f1e2015dde0 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e24c83640 1 -- 172.21.15.82:0/2855833614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e18030860 con 0x7f1e2015dde0 2024-03-14T12:16:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e24c83640 1 -- 172.21.15.82:0/2855833614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18030c30 con 0x7f1e2015dde0 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 -- 172.21.15.82:0/2855833614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2015dde0 msgr2=0x7f1e2015e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/2855833614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2015dde0 0x7f1e2015e210 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b2340 tx=0x7f1e1802b9a0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 -- 172.21.15.82:0/2855833614 shutdown_connections 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/2855833614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2015dde0 0x7f1e2015e210 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/2855833614 >> v2:172.21.15.164:3300/0 conn(0x7f1e20151230 0x7f1e200cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 -- 172.21.15.82:0/2855833614 >> 172.21.15.82:0/2855833614 conn(0x7f1e200b1c70 msgr2=0x7f1e200cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 -- 172.21.15.82:0/2855833614 shutdown_connections 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.871+0000 7f1e27ee9640 1 -- 172.21.15.82:0/2855833614 wait complete. 2024-03-14T12:16:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.872+0000 7f1e27ee9640 1 Processor -- start 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.872+0000 7f1e27ee9640 1 -- start start 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e27ee9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e200b4b70 0x7f1e2014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e27ee9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e20151230 0x7f1e200ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e27ee9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e2015dde0 0x7f1e20153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e27ee9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2015e750 con 0x7f1e200b4b70 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e27ee9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2006c690 con 0x7f1e2015dde0 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e27ee9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20151610 con 0x7f1e20151230 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e200b4b70 0x7f1e2014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e200b4b70 0x7f1e2014f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54308/0 (socket says 172.21.15.82:54308) 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25c85640 1 -- 172.21.15.82:0/3142631105 learned_addr learned my addr 172.21.15.82:0/3142631105 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25c85640 1 -- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e20151230 msgr2=0x7f1e200ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25484640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e20151230 0x7f1e200ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e26486640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e2015dde0 0x7f1e20153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25c85640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e20151230 0x7f1e200ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25c85640 1 -- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e2015dde0 msgr2=0x7f1e20153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.873+0000 7f1e25c85640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e2015dde0 0x7f1e20153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e25c85640 1 -- 172.21.15.82:0/3142631105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200cf270 con 0x7f1e200b4b70 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e25484640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e20151230 0x7f1e200ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e26486640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e2015dde0 0x7f1e20153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e25c85640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e200b4b70 0x7f1e2014f670 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c007ef0 tx=0x7f1e1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e167fc640 1 -- 172.21.15.82:0/3142631105 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c014620 con 0x7f1e200b4b70 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e167fc640 1 -- 172.21.15.82:0/3142631105 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c019070 con 0x7f1e200b4b70 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e200c8e60 con 0x7f1e200b4b70 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e167fc640 1 -- 172.21.15.82:0/3142631105 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c014a00 con 0x7f1e200b4b70 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.874+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e200c9420 con 0x7f1e200b4b70 2024-03-14T12:16:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.875+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2004f050 con 0x7f1e200b4b70 2024-03-14T12:16:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.875+0000 7f1e167fc640 1 -- 172.21.15.82:0/3142631105 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1e1c014ba0 con 0x7f1e200b4b70 2024-03-14T12:16:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.876+0000 7f1e167fc640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bbd0 0x7f1e0406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.876+0000 7f1e167fc640 1 -- 172.21.15.82:0/3142631105 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e1c08b3a0 con 0x7f1e200b4b70 2024-03-14T12:16:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.876+0000 7f1e25484640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bbd0 0x7f1e0406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.876+0000 7f1e25484640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bbd0 0x7f1e0406e090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e2015ccc0 tx=0x7f1e10009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:47.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:47.880+0000 7f1e167fc640 1 -- 172.21.15.82:0/3142631105 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1e1c059e90 con 0x7f1e200b4b70 2024-03-14T12:16:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.090+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1e200b2f40 con 0x7f1e200b4b70 2024-03-14T12:16:48.090 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-14T12:16:48.091+0000 7f8354c85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T12:16:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.092+0000 7f1e167fc640 1 -- 172.21.15.82:0/3142631105 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1e1c059790 con 0x7f1e200b4b70 2024-03-14T12:16:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bbd0 msgr2=0x7f1e0406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bbd0 0x7f1e0406e090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e2015ccc0 tx=0x7f1e10009040 comp rx=0 tx=0).stop 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e200b4b70 msgr2=0x7f1e2014f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e200b4b70 0x7f1e2014f670 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c007ef0 tx=0x7f1e1c00e4b0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 shutdown_connections 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e2015dde0 0x7f1e20153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0406bbd0 0x7f1e0406e090 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e20151230 0x7f1e200ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 --2- 172.21.15.82:0/3142631105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e200b4b70 0x7f1e2014f670 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 >> 172.21.15.82:0/3142631105 conn(0x7f1e200b1c70 msgr2=0x7f1e2015d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 shutdown_connections 2024-03-14T12:16:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.095+0000 7f1e27ee9640 1 -- 172.21.15.82:0/3142631105 wait complete. 2024-03-14T12:16:48.096 INFO:tasks.ceph.mds.c.smithi082.stderr: -2003> 2024-03-14T12:16:38.133+0000 7f18177a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:16:48.107 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T12:16:48.125 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:16:48.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.210+0000 7fbc092fc640 1 Processor -- start 2024-03-14T12:16:48.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.210+0000 7fbc092fc640 1 -- start start 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.210+0000 7fbc092fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc0415f3a0 0x7fbc04070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.210+0000 7fbc092fc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc041608b0 con 0x7fbc04160150 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.210+0000 7fbc092fc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc04070a60 con 0x7fbc040cfc30 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.210+0000 7fbc092fc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc040d01a0 con 0x7fbc0415f3a0 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc02ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc0415f3a0 0x7fbc04070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc037fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbc04160150 0x7fbc040af090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51770/0 (socket says 172.21.15.82:51770) 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc02ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc0415f3a0 0x7fbc04070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54338/0 (socket says 172.21.15.82:54338) 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc037fe640 1 -- 172.21.15.82:0/2488701704 learned_addr learned my addr 172.21.15.82:0/2488701704 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc02ffd640 1 -- 172.21.15.82:0/2488701704 >> v1:172.21.15.164:6789/0 conn(0x7fbc040cfc30 legacy=0x7fbc0415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc02ffd640 1 -- 172.21.15.82:0/2488701704 >> v1:172.21.15.82:6789/0 conn(0x7fbc04160150 legacy=0x7fbc040af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc01ffb640 1 -- 172.21.15.82:0/2488701704 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016832781 0 0) 0x7fbc041608b0 con 0x7fbc04160150 2024-03-14T12:16:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.211+0000 7fbc02ffd640 1 -- 172.21.15.82:0/2488701704 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc040d0480 con 0x7fbc0415f3a0 2024-03-14T12:16:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc02ffd640 1 --2- 172.21.15.82:0/2488701704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0415f3a0 0x7fbc04070520 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0097a0 tx=0x7fbbfc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d416f5ed11df7f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc01ffb640 1 -- 172.21.15.82:0/2488701704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc035020 con 0x7fbc0415f3a0 2024-03-14T12:16:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc01ffb640 1 -- 172.21.15.82:0/2488701704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbfc031830 con 0x7fbc0415f3a0 2024-03-14T12:16:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc01ffb640 1 -- 172.21.15.82:0/2488701704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc031c00 con 0x7fbc0415f3a0 2024-03-14T12:16:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc092fc640 1 -- 172.21.15.82:0/2488701704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0415f3a0 msgr2=0x7fbc04070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc092fc640 1 --2- 172.21.15.82:0/2488701704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0415f3a0 0x7fbc04070520 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0097a0 tx=0x7fbbfc02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc092fc640 1 -- 172.21.15.82:0/2488701704 shutdown_connections 2024-03-14T12:16:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc092fc640 1 --2- 172.21.15.82:0/2488701704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0415f3a0 0x7fbc04070520 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc092fc640 1 -- 172.21.15.82:0/2488701704 >> 172.21.15.82:0/2488701704 conn(0x7fbc040b3680 msgr2=0x7fbc040b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc092fc640 1 -- 172.21.15.82:0/2488701704 shutdown_connections 2024-03-14T12:16:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.212+0000 7fbc092fc640 1 -- 172.21.15.82:0/2488701704 wait complete. 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 Processor -- start 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 -- start start 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc040cfc30 0x7fbc040ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc0415f3a0 0x7fbc040b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc04160150 0x7fbc040b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0415f780 con 0x7fbc04160150 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040aeb70 con 0x7fbc040cfc30 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc092fc640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040af7a0 con 0x7fbc0415f3a0 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.214+0000 7fbc037fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc04160150 0x7fbc040b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc04160150 0x7fbc040b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54342/0 (socket says 172.21.15.82:54342) 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 -- 172.21.15.82:0/734858153 learned_addr learned my addr 172.21.15.82:0/734858153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc02ffd640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc040cfc30 0x7fbc040ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc027fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc0415f3a0 0x7fbc040b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 -- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc0415f3a0 msgr2=0x7fbc040b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc0415f3a0 0x7fbc040b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 -- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc040cfc30 msgr2=0x7fbc040ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc040cfc30 0x7fbc040ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 -- 172.21.15.82:0/734858153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc040b4a50 con 0x7fbc04160150 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc02ffd640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc040cfc30 0x7fbc040ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc027fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc0415f3a0 0x7fbc040b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc037fe640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc04160150 0x7fbc040b51b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400c3e0 tx=0x7fbbf400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbbeb7fe640 1 -- 172.21.15.82:0/734858153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40074e0 con 0x7fbc04160150 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbbeb7fe640 1 -- 172.21.15.82:0/734858153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf40047e0 con 0x7fbc04160150 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbbeb7fe640 1 -- 172.21.15.82:0/734858153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4004bd0 con 0x7fbc04160150 2024-03-14T12:16:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc040cda40 con 0x7fbc04160150 2024-03-14T12:16:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.215+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc040ce000 con 0x7fbc04160150 2024-03-14T12:16:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.216+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0404f050 con 0x7fbc04160150 2024-03-14T12:16:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.217+0000 7fbbeb7fe640 1 -- 172.21.15.82:0/734858153 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbbf4011070 con 0x7fbc04160150 2024-03-14T12:16:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.217+0000 7fbbeb7fe640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbbf006bbd0 0x7fbbf006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.217+0000 7fbc02ffd640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbbf006bbd0 0x7fbbf006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.217+0000 7fbbeb7fe640 1 -- 172.21.15.82:0/734858153 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbbf408a640 con 0x7fbc04160150 2024-03-14T12:16:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.217+0000 7fbc02ffd640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbbf006bbd0 0x7fbbf006e090 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc036000 tx=0x7fbbfc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:48.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.221+0000 7fbbeb7fe640 1 -- 172.21.15.82:0/734858153 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbbf40590d0 con 0x7fbc04160150 2024-03-14T12:16:48.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:48.431+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc040b7ab0 con 0x7fbc04160150 2024-03-14T12:16:49.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.094+0000 7fbbeb7fe640 1 -- 172.21.15.82:0/734858153 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbbf40589d0 con 0x7fbc04160150 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbbf006bbd0 msgr2=0x7fbbf006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbbf006bbd0 0x7fbbf006e090 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc036000 tx=0x7fbbfc008030 comp rx=0 tx=0).stop 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc04160150 msgr2=0x7fbc040b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc04160150 0x7fbc040b51b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400c3e0 tx=0x7fbbf400c8a0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 shutdown_connections 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc04160150 0x7fbc040b51b0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbbf006bbd0 0x7fbbf006e090 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc0415f3a0 0x7fbc040b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 --2- 172.21.15.82:0/734858153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc040cfc30 0x7fbc040ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 >> 172.21.15.82:0/734858153 conn(0x7fbc040b3680 msgr2=0x7fbc0415e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:49.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 shutdown_connections 2024-03-14T12:16:49.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.096+0000 7fbc092fc640 1 -- 172.21.15.82:0/734858153 wait complete. 2024-03-14T12:16:49.108 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T12:16:49.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.255+0000 7f8eb3acd640 1 Processor -- start 2024-03-14T12:16:49.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.255+0000 7f8eb3acd640 1 -- start start 2024-03-14T12:16:49.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb3acd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 0x7f8eac06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb3acd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac072560 con 0x7f8eac06f7f0 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb3acd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac0d1590 con 0x7f8eac0b39c0 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb3acd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac161b70 con 0x7f8eac15df30 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb1068640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8eac06f7f0 0x7f8eac071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51784/0 (socket says 172.21.15.82:51784) 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb1068640 1 -- 172.21.15.82:0/2310404188 learned_addr learned my addr 172.21.15.82:0/2310404188 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785098456 0 0) 0x7f8eac072560 con 0x7f8eac06f7f0 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e98003520 con 0x7f8eac06f7f0 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb1869640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 0x7f8eac06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353489633 0 0) 0x7f8e98003520 con 0x7f8eac06f7f0 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 msgr2=0x7f8eac06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 0x7f8eac06ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 >> v1:172.21.15.164:6789/0 conn(0x7f8eac0b39c0 legacy=0x7f8eac071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac161e50 con 0x7f8eac06f7f0 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f8e9c001670 con 0x7f8eac06f7f0 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 >> v1:172.21.15.82:6789/0 conn(0x7f8eac06f7f0 legacy=0x7f8eac071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e980039e0 0x7f8e98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.256+0000 7f8eb0867640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e98004800 0x7f8e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb0867640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 0x7f8e980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac161b70 con 0x7f8e980039e0 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9800baa0 con 0x7f8e98004800 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9800bca0 con 0x7f8eac15df30 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb1869640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e980039e0 0x7f8e98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb1068640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e98004800 0x7f8e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:49.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb206a640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 0x7f8e980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb1869640 1 -- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 msgr2=0x7f8e980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb1869640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 0x7f8e980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb1869640 1 -- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e98004800 msgr2=0x7f8e98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb1869640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e98004800 0x7f8e98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb1869640 1 -- 172.21.15.82:0/2310404188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98003520 con 0x7f8e980039e0 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.257+0000 7f8eb206a640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac15df30 0x7f8e980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb1869640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e980039e0 0x7f8e98003e00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009a90 tx=0x7f8e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e31fd40cf41b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8039070 con 0x7f8e980039e0 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea802c680 con 0x7f8e980039e0 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb0867640 1 -- 172.21.15.82:0/2310404188 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8009690 con 0x7f8e980039e0 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2310404188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e980039e0 msgr2=0x7f8e98003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:49.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb3acd640 1 --2- 172.21.15.82:0/2310404188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e980039e0 0x7f8e98003e00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009a90 tx=0x7f8e98005830 comp rx=0 tx=0).stop 2024-03-14T12:16:49.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb206a640 1 -- 172.21.15.82:0/2310404188 reap_dead start 2024-03-14T12:16:49.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2310404188 shutdown_connections 2024-03-14T12:16:49.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2310404188 >> 172.21.15.82:0/2310404188 conn(0x7f8eac0d09d0 msgr2=0x7f8eac0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:49.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2310404188 shutdown_connections 2024-03-14T12:16:49.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.258+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2310404188 wait complete. 2024-03-14T12:16:49.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.259+0000 7f8eb3acd640 1 Processor -- start 2024-03-14T12:16:49.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.259+0000 7f8eb3acd640 1 -- start start 2024-03-14T12:16:49.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb3acd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e980039e0 0x7f8eac14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb3acd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac06ded0 0x7f8eac1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb3acd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac157b00 0x7f8eac0ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb3acd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac1627d0 con 0x7f8eac157b00 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb3acd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac162650 con 0x7f8e980039e0 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb3acd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac162950 con 0x7f8eac06ded0 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac157b00 0x7f8eac0ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac157b00 0x7f8eac0ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54360/0 (socket says 172.21.15.82:54360) 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 -- 172.21.15.82:0/2783180930 learned_addr learned my addr 172.21.15.82:0/2783180930 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 -- 172.21.15.82:0/2783180930 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac06ded0 msgr2=0x7f8eac1575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac06ded0 0x7f8eac1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 -- 172.21.15.82:0/2783180930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e980039e0 msgr2=0x7f8eac14fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e980039e0 0x7f8eac14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.260+0000 7f8eb206a640 1 -- 172.21.15.82:0/2783180930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac06e350 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.261+0000 7f8eb206a640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac157b00 0x7f8eac0ad850 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0b3720 tx=0x7f8ea400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.261+0000 7f8ea27fc640 1 -- 172.21.15.82:0/2783180930 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4006d50 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.261+0000 7f8ea27fc640 1 -- 172.21.15.82:0/2783180930 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4003970 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.261+0000 7f8ea27fc640 1 -- 172.21.15.82:0/2783180930 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4003d60 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.261+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac0aff60 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.261+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac0af1a0 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.262+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac157f80 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.262+0000 7f8ea27fc640 1 -- 172.21.15.82:0/2783180930 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8ea401b020 con 0x7f8eac157b00 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.263+0000 7f8ea27fc640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9006bc00 0x7f8e9006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:49.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.263+0000 7f8eb1869640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9006bc00 0x7f8e9006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:49.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.263+0000 7f8ea27fc640 1 -- 172.21.15.82:0/2783180930 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8ea408a5b0 con 0x7f8eac157b00 2024-03-14T12:16:49.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.263+0000 7f8eb1869640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9006bc00 0x7f8e9006e0c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8036000 tx=0x7f8ea802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:49.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.267+0000 7f8ea27fc640 1 -- 172.21.15.82:0/2783180930 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8ea4059100 con 0x7f8eac157b00 2024-03-14T12:16:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:49.441+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8eac04f050 con 0x7f8eac157b00 2024-03-14T12:16:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.097+0000 7f8ea27fc640 1 -- 172.21.15.82:0/2783180930 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ea4058a00 con 0x7f8eac157b00 2024-03-14T12:16:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9006bc00 msgr2=0x7f8e9006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9006bc00 0x7f8e9006e0c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8036000 tx=0x7f8ea802e000 comp rx=0 tx=0).stop 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac157b00 msgr2=0x7f8eac0ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac157b00 0x7f8eac0ad850 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0b3720 tx=0x7f8ea400dc00 comp rx=0 tx=0).stop 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 shutdown_connections 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8eac157b00 0x7f8eac0ad850 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8e9006bc00 0x7f8e9006e0c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8eac06ded0 0x7f8eac1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 --2- 172.21.15.82:0/2783180930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e980039e0 0x7f8eac14fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 >> 172.21.15.82:0/2783180930 conn(0x7f8eac0d09d0 msgr2=0x7f8eac1589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 shutdown_connections 2024-03-14T12:16:50.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.101+0000 7f8eb3acd640 1 -- 172.21.15.82:0/2783180930 wait complete. 2024-03-14T12:16:50.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T12:16:50.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.220+0000 7fe9ce814640 1 Processor -- start 2024-03-14T12:16:50.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.220+0000 7fe9ce814640 1 -- start start 2024-03-14T12:16:50.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9ce814640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:50.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9ce814640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe9c8072d00 0x7fe9c806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:50.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9ce814640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c80adfe0 con 0x7fe9c80ad420 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9ce814640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c8066c30 con 0x7fe9c8072d00 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9ce814640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c8066f10 con 0x7fe9c80ad040 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9c7fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9c77fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe9c8072d00 0x7fe9c806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9ccdb1640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe9c80ad420 0x7fe9c80ad8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:33572/0 (socket says 172.21.15.82:33572) 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.221+0000 7fe9ccdb1640 1 -- 172.21.15.82:0/87502857 learned_addr learned my addr 172.21.15.82:0/87502857 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.222+0000 7fe9c6ffd640 1 -- 172.21.15.82:0/87502857 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693327741 0 0) 0x7fe9c80adfe0 con 0x7fe9c80ad420 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.222+0000 7fe9c6ffd640 1 -- 172.21.15.82:0/87502857 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b0003520 con 0x7fe9c80ad420 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.222+0000 7fe9c77fe640 1 -- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 msgr2=0x7fe9c80aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.222+0000 7fe9c77fe640 1 --2- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.222+0000 7fe9c77fe640 1 -- 172.21.15.82:0/87502857 >> v1:172.21.15.164:6789/0 conn(0x7fe9c80ad420 legacy=0x7fe9c80ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.222+0000 7fe9c77fe640 1 -- 172.21.15.82:0/87502857 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80ae2a0 con 0x7fe9c8072d00 2024-03-14T12:16:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.222+0000 7fe9c7fff640 1 --2- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9c77fe640 1 --2- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 0x7fe9c806ad10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0097a0 tx=0x7fe9bc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23abcc013939382 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9c6ffd640 1 -- 172.21.15.82:0/87502857 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0315a0 con 0x7fe9c8072d00 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9c6ffd640 1 -- 172.21.15.82:0/87502857 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc037070 con 0x7fe9c8072d00 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9c6ffd640 1 -- 172.21.15.82:0/87502857 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc031940 con 0x7fe9c8072d00 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 -- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 msgr2=0x7fe9c806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 --2- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 0x7fe9c806ad10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0097a0 tx=0x7fe9bc02b840 comp rx=0 tx=0).stop 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 -- 172.21.15.82:0/87502857 shutdown_connections 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 --2- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 0x7fe9c806ad10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 --2- 172.21.15.82:0/87502857 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 -- 172.21.15.82:0/87502857 >> 172.21.15.82:0/87502857 conn(0x7fe9c80714a0 msgr2=0x7fe9c8072350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 -- 172.21.15.82:0/87502857 shutdown_connections 2024-03-14T12:16:50.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.223+0000 7fe9ce814640 1 -- 172.21.15.82:0/87502857 wait complete. 2024-03-14T12:16:50.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.224+0000 7fe9ce814640 1 Processor -- start 2024-03-14T12:16:50.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ce814640 1 -- start start 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ce814640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 0x7fe9c8150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ce814640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ce814640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ad420 0x7fe9c80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ce814640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80af940 con 0x7fe9c80ad420 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ce814640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80af7c0 con 0x7fe9c8072d00 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ce814640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80afac0 con 0x7fe9c80ad040 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.225+0000 7fe9ccdb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ad420 0x7fe9c80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ad420 0x7fe9c80c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54372/0 (socket says 172.21.15.82:54372) 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 -- 172.21.15.82:0/4081115941 learned_addr learned my addr 172.21.15.82:0/4081115941 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9c7fff640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 0x7fe9c8150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9c77fe640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 -- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 msgr2=0x7fe9c80c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 -- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 msgr2=0x7fe9c8150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 0x7fe9c8150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 -- 172.21.15.82:0/4081115941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80ce1d0 con 0x7fe9c80ad420 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9c77fe640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ccdb1640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ad420 0x7fe9c80c9910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000cb40 tx=0x7fe9c000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9c4ff9640 1 -- 172.21.15.82:0/4081115941 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0017070 con 0x7fe9c80ad420 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9c4ff9640 1 -- 172.21.15.82:0/4081115941 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c000aca0 con 0x7fe9c80ad420 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c80ce4b0 con 0x7fe9c80ad420 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9c4ff9640 1 -- 172.21.15.82:0/4081115941 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0012620 con 0x7fe9c80ad420 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.226+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c80cea70 con 0x7fe9c80ad420 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.227+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c804f050 con 0x7fe9c80ad420 2024-03-14T12:16:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.228+0000 7fe9c4ff9640 1 -- 172.21.15.82:0/4081115941 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe9c001a070 con 0x7fe9c80ad420 2024-03-14T12:16:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.228+0000 7fe9c4ff9640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9b406bbd0 0x7fe9b406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.228+0000 7fe9c7fff640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9b406bbd0 0x7fe9b406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.228+0000 7fe9c4ff9640 1 -- 172.21.15.82:0/4081115941 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe9c008a6b0 con 0x7fe9c80ad420 2024-03-14T12:16:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.229+0000 7fe9c7fff640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9b406bbd0 0x7fe9b406e090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009560 tx=0x7fe9b8009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:50.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.233+0000 7fe9c4ff9640 1 -- 172.21.15.82:0/4081115941 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe9c0058cc0 con 0x7fe9c80ad420 2024-03-14T12:16:50.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:50.405+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9c80aefd0 con 0x7fe9c80ad420 2024-03-14T12:16:51.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.099+0000 7fe9c4ff9640 1 -- 172.21.15.82:0/4081115941 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe9c005cd60 con 0x7fe9c80ad420 2024-03-14T12:16:51.099 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9b406bbd0 msgr2=0x7fe9b406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9b406bbd0 0x7fe9b406e090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009560 tx=0x7fe9b8009190 comp rx=0 tx=0).stop 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ad420 msgr2=0x7fe9c80c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ad420 0x7fe9c80c9910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000cb40 tx=0x7fe9c000a060 comp rx=0 tx=0).stop 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 shutdown_connections 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ad420 0x7fe9c80c9910 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe9b406bbd0 0x7fe9b406e090 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe9c80ad040 0x7fe9c80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 --2- 172.21.15.82:0/4081115941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9c8072d00 0x7fe9c8150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 >> 172.21.15.82:0/4081115941 conn(0x7fe9c80714a0 msgr2=0x7fe9c80b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 shutdown_connections 2024-03-14T12:16:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.103+0000 7fe9ce814640 1 -- 172.21.15.82:0/4081115941 wait complete. 2024-03-14T12:16:51.116 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-14T12:16:51.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.227+0000 7f13ad8a7640 1 Processor -- start 2024-03-14T12:16:51.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.227+0000 7f13ad8a7640 1 -- start start 2024-03-14T12:16:51.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.228+0000 7f13ad8a7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a80d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.228+0000 7f13ad8a7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f13a80b0030 0x7f13a80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.228+0000 7f13ad8a7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a815e500 con 0x7f13a8158670 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.228+0000 7f13ad8a7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a815e7a0 con 0x7f13a80b0030 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.228+0000 7f13ad8a7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a815ea80 con 0x7f13a8158290 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.228+0000 7f13a6ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a80d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a77fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f13a80b0030 0x7f13a80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a67fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f13a8158670 0x7f13a80af920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:33580/0 (socket says 172.21.15.82:33580) 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a6ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a80d0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:49112/0 (socket says 172.21.15.82:49112) 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a67fc640 1 -- 172.21.15.82:0/3653342999 learned_addr learned my addr 172.21.15.82:0/3653342999 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a77fe640 1 -- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 msgr2=0x7f13a80d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a77fe640 1 --2- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a80d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a77fe640 1 -- 172.21.15.82:0/3653342999 >> v1:172.21.15.164:6789/0 conn(0x7f13a8158670 legacy=0x7f13a80af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a77fe640 1 -- 172.21.15.82:0/3653342999 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a815ed60 con 0x7f13a80b0030 2024-03-14T12:16:51.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.229+0000 7f13a6ffd640 1 --2- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a80d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:51.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.230+0000 7f13a77fe640 1 --2- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 0x7f13a80d1340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009d90 tx=0x7f13a002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7b2ad07c8348a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:51.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.230+0000 7f13a5ffb640 1 -- 172.21.15.82:0/3653342999 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0039070 con 0x7f13a80b0030 2024-03-14T12:16:51.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.230+0000 7f13a5ffb640 1 -- 172.21.15.82:0/3653342999 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a002cbc0 con 0x7f13a80b0030 2024-03-14T12:16:51.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.230+0000 7f13a5ffb640 1 -- 172.21.15.82:0/3653342999 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0034440 con 0x7f13a80b0030 2024-03-14T12:16:51.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.230+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 msgr2=0x7f13a80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.230+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 0x7f13a80d1340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009d90 tx=0x7f13a002be10 comp rx=0 tx=0).stop 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.231+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3653342999 shutdown_connections 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.231+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 0x7f13a80d1340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.231+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3653342999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a80d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.231+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3653342999 >> 172.21.15.82:0/3653342999 conn(0x7f13a806c000 msgr2=0x7f13a806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.231+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3653342999 shutdown_connections 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.231+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3653342999 wait complete. 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.232+0000 7f13ad8a7640 1 Processor -- start 2024-03-14T12:16:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.232+0000 7f13ad8a7640 1 -- start start 2024-03-14T12:16:51.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.232+0000 7f13ad8a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 0x7f13a8148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13ad8a7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a81f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13ad8a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a8158670 0x7f13a81f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13ad8a7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a806d890 con 0x7f13a8158670 2024-03-14T12:16:51.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13ad8a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a806d710 con 0x7f13a80b0030 2024-03-14T12:16:51.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13ad8a7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80ae860 con 0x7f13a8158290 2024-03-14T12:16:51.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a8158670 0x7f13a81f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a8158670 0x7f13a81f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54380/0 (socket says 172.21.15.82:54380) 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 -- 172.21.15.82:0/3300049738 learned_addr learned my addr 172.21.15.82:0/3300049738 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a67fc640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a81f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a6ffd640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 0x7f13a8148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 -- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 msgr2=0x7f13a81f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a81f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 -- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 msgr2=0x7f13a8148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 0x7f13a8148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 -- 172.21.15.82:0/3300049738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a80d0270 con 0x7f13a8158670 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13a77fe640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a8158670 0x7f13a81f7540 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009eb0 tx=0x7f13a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f138ffff640 1 -- 172.21.15.82:0/3300049738 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0039050 con 0x7f13a8158670 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f138ffff640 1 -- 172.21.15.82:0/3300049738 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0045070 con 0x7f13a8158670 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a81f7bc0 con 0x7f13a8158670 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f138ffff640 1 -- 172.21.15.82:0/3300049738 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a003e830 con 0x7f13a8158670 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.233+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a815ff20 con 0x7f13a8158670 2024-03-14T12:16:51.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.234+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a804f050 con 0x7f13a8158670 2024-03-14T12:16:51.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.235+0000 7f138ffff640 1 -- 172.21.15.82:0/3300049738 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f13a003e9d0 con 0x7f13a8158670 2024-03-14T12:16:51.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.235+0000 7f138ffff640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f139406bbd0 0x7f139406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.235+0000 7f138ffff640 1 -- 172.21.15.82:0/3300049738 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f13a00ad410 con 0x7f13a8158670 2024-03-14T12:16:51.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.235+0000 7f13a6ffd640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f139406bbd0 0x7f139406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.236+0000 7f13a6ffd640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f139406bbd0 0x7f139406e090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f13a80cd530 tx=0x7f139c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.240+0000 7f138ffff640 1 -- 172.21.15.82:0/3300049738 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f13a007bd60 con 0x7f13a8158670 2024-03-14T12:16:51.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.411+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f13a8156b80 con 0x7f13a8158670 2024-03-14T12:16:51.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.414+0000 7f138ffff640 1 -- 172.21.15.82:0/3300049738 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f13a007fe00 con 0x7f13a8158670 2024-03-14T12:16:51.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f139406bbd0 msgr2=0x7f139406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f139406bbd0 0x7f139406e090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f13a80cd530 tx=0x7f139c009190 comp rx=0 tx=0).stop 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a8158670 msgr2=0x7f13a81f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a8158670 0x7f13a81f7540 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009eb0 tx=0x7f13a0004a40 comp rx=0 tx=0).stop 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 shutdown_connections 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a8158670 0x7f13a81f7540 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f139406bbd0 0x7f139406e090 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a8158290 0x7f13a81f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 --2- 172.21.15.82:0/3300049738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a80b0030 0x7f13a8148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 >> 172.21.15.82:0/3300049738 conn(0x7f13a806c000 msgr2=0x7f13a806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 shutdown_connections 2024-03-14T12:16:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.416+0000 7f13ad8a7640 1 -- 172.21.15.82:0/3300049738 wait complete. 2024-03-14T12:16:51.426 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_disconnect' 2024-03-14T12:16:51.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.527+0000 7fad3c545640 1 Processor -- start 2024-03-14T12:16:51.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.527+0000 7fad3c545640 1 -- start start 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.528+0000 7fad3c545640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 0x7fad340d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.528+0000 7fad3c545640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340cf8c0 con 0x7fad340d0760 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.528+0000 7fad3c545640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340cfba0 con 0x7fad340b7720 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.528+0000 7fad3c545640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b6c40 con 0x7fad340cf370 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.528+0000 7fad3a2e1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fad340d0760 0x7fad340d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53776/0 (socket says 172.21.15.82:53776) 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.528+0000 7fad3a2e1640 1 -- 172.21.15.82:0/2552714821 learned_addr learned my addr 172.21.15.82:0/2552714821 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad39ae0640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 0x7fad340d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305053034 0 0) 0x7fad340cf8c0 con 0x7fad340d0760 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad20003520 con 0x7fad340d0760 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831921705 0 0) 0x7fad340cfba0 con 0x7fad340b7720 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad340cf8c0 con 0x7fad340b7720 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 549886767 0 0) 0x7fad20003520 con 0x7fad340d0760 2024-03-14T12:16:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 msgr2=0x7fad340d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 0x7fad340d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 >> v1:172.21.15.164:6789/0 conn(0x7fad340b7720 legacy=0x7fad340b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340b6f20 con 0x7fad340d0760 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fad300019f0 con 0x7fad340d0760 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 >> v1:172.21.15.82:6789/0 conn(0x7fad340d0760 legacy=0x7fad340d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad20003b70 0x7fad20003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20004800 0x7fad20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 0x7fad200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b6c40 con 0x7fad20003b70 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2000ba70 con 0x7fad20004800 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2000bc70 con 0x7fad340cf370 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad20003b70 0x7fad20003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad39ae0640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20004800 0x7fad20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad3aae2640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 0x7fad200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad3a2e1640 1 -- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 msgr2=0x7fad200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad340cf370 0x7fad200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad3a2e1640 1 -- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20004800 msgr2=0x7fad20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad20004800 0x7fad20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.529+0000 7fad3a2e1640 1 -- 172.21.15.82:0/2552714821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20003520 con 0x7fad20003b70 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.530+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad20003b70 0x7fad20003f70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fad3000a7e0 tx=0x7fad20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3ee1d74dc15c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.530+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3000a430 con 0x7fad20003b70 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.530+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad30034650 con 0x7fad20003b70 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.530+0000 7fad392df640 1 -- 172.21.15.82:0/2552714821 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30034a40 con 0x7fad20003b70 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.531+0000 7fad3c545640 1 -- 172.21.15.82:0/2552714821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad20003b70 msgr2=0x7fad20003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.531+0000 7fad3c545640 1 --2- 172.21.15.82:0/2552714821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad20003b70 0x7fad20003f70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fad3000a7e0 tx=0x7fad20005830 comp rx=0 tx=0).stop 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.531+0000 7fad3aae2640 1 -- 172.21.15.82:0/2552714821 reap_dead start 2024-03-14T12:16:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.531+0000 7fad3c545640 1 -- 172.21.15.82:0/2552714821 shutdown_connections 2024-03-14T12:16:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.531+0000 7fad3c545640 1 -- 172.21.15.82:0/2552714821 >> 172.21.15.82:0/2552714821 conn(0x7fad3409d030 msgr2=0x7fad340d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.531+0000 7fad3c545640 1 -- 172.21.15.82:0/2552714821 shutdown_connections 2024-03-14T12:16:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.531+0000 7fad3c545640 1 -- 172.21.15.82:0/2552714821 wait complete. 2024-03-14T12:16:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.532+0000 7fad3c545640 1 Processor -- start 2024-03-14T12:16:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.532+0000 7fad3c545640 1 -- start start 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad20003b70 0x7fad34152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad340cae20 0x7fad340c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad340cb2a0 0x7fad340c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34161920 con 0x7fad340cb2a0 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad341617a0 con 0x7fad340cae20 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340d4660 con 0x7fad20003b70 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad340cb2a0 0x7fad340c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad340cb2a0 0x7fad340c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34174/0 (socket says 172.21.15.82:34174) 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 -- 172.21.15.82:0/1083414774 learned_addr learned my addr 172.21.15.82:0/1083414774 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad20003b70 0x7fad34152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad39ae0640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad340cae20 0x7fad340c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 -- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad20003b70 msgr2=0x7fad34152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad20003b70 0x7fad34152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 -- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad340cae20 msgr2=0x7fad340c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad340cae20 0x7fad340c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 -- 172.21.15.82:0/1083414774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340cb870 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad39ae0640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad340cae20 0x7fad340c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad20003b70 0x7fad34152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3aae2640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad340cb2a0 0x7fad340c9650 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad340d1830 tx=0x7fad2c00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad2affd640 1 -- 172.21.15.82:0/1083414774 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c005030 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad2affd640 1 -- 172.21.15.82:0/1083414774 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c0051f0 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad340cbb50 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad2affd640 1 -- 172.21.15.82:0/1083414774 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0069e0 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.533+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad340cc110 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.534+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3404f050 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.535+0000 7fad2affd640 1 -- 172.21.15.82:0/1083414774 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fad2c006b80 con 0x7fad340cb2a0 2024-03-14T12:16:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.535+0000 7fad2affd640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fad1806bbd0 0x7fad1806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.535+0000 7fad2affd640 1 -- 172.21.15.82:0/1083414774 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad2c08a040 con 0x7fad340cb2a0 2024-03-14T12:16:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.535+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fad1806bbd0 0x7fad1806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.536+0000 7fad3a2e1640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fad1806bbd0 0x7fad1806e090 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fad3000a7e0 tx=0x7fad30009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.540+0000 7fad2affd640 1 -- 172.21.15.82:0/1083414774 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fad2c01b060 con 0x7fad340cb2a0 2024-03-14T12:16:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:51.777+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_disconnect"]} v 0) v1 -- 0x7fad340d0540 con 0x7fad340cb2a0 2024-03-14T12:16:52.049 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:16:52.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:16:52.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.107+0000 7fad2affd640 1 -- 172.21.15.82:0/1083414774 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_disconnect"]}]=0 v172) v1 ==== 185+0+0 (secure 0 0 0) 0x7fad2c058e30 con 0x7fad340cb2a0 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fad1806bbd0 msgr2=0x7fad1806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fad1806bbd0 0x7fad1806e090 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fad3000a7e0 tx=0x7fad30009030 comp rx=0 tx=0).stop 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad340cb2a0 msgr2=0x7fad340c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad340cb2a0 0x7fad340c9650 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad340d1830 tx=0x7fad2c00e820 comp rx=0 tx=0).stop 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 shutdown_connections 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad340cb2a0 0x7fad340c9650 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad340cae20 0x7fad340c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fad1806bbd0 0x7fad1806e090 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 --2- 172.21.15.82:0/1083414774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad20003b70 0x7fad34152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 >> 172.21.15.82:0/1083414774 conn(0x7fad3409d030 msgr2=0x7fad3406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 shutdown_connections 2024-03-14T12:16:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.109+0000 7fad3c545640 1 -- 172.21.15.82:0/1083414774 wait complete. 2024-03-14T12:16:52.120 INFO:tasks.cephfs_test_runner:That the per-client labelled metrics are unavailable during client disconnect ... ok 2024-03-14T12:16:52.120 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters_on_client_reconnect (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled metrics are generated during client reconnect 2024-03-14T12:16:52.120 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters_on_client_reconnect (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-14T12:16:52.121 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:52.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.261+0000 7fbd952d7640 1 Processor -- start 2024-03-14T12:16:52.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.261+0000 7fbd952d7640 1 -- start start 2024-03-14T12:16:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd952d7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbd900d0760 0x7fbd900d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd952d7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd900cf370 0x7fbd900d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd952d7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d1da0 0x7fbd900d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd952d7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900d1080 con 0x7fbd900cf370 2024-03-14T12:16:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd952d7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900cf8c0 con 0x7fbd900d0760 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd952d7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900cfba0 con 0x7fbd900d1da0 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd900cf370 0x7fbd900d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd900cf370 0x7fbd900d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34190/0 (socket says 172.21.15.82:34190) 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 -- 172.21.15.82:0/1374352204 learned_addr learned my addr 172.21.15.82:0/1374352204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd8f7fe640 1 --2- 172.21.15.82:0/1374352204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d1da0 0x7fbd900d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd8effd640 1 --2- 172.21.15.82:0/1374352204 >> v2:172.21.15.164:3300/0 conn(0x7fbd900d0760 0x7fbd900d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 -- 172.21.15.82:0/1374352204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d1da0 msgr2=0x7fbd900d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 --2- 172.21.15.82:0/1374352204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d1da0 0x7fbd900d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 -- 172.21.15.82:0/1374352204 >> v2:172.21.15.164:3300/0 conn(0x7fbd900d0760 msgr2=0x7fbd900d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 --2- 172.21.15.82:0/1374352204 >> v2:172.21.15.164:3300/0 conn(0x7fbd900d0760 0x7fbd900d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.262+0000 7fbd87fff640 1 -- 172.21.15.82:0/1374352204 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900b6c40 con 0x7fbd900cf370 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.263+0000 7fbd87fff640 1 --2- 172.21.15.82:0/1374352204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd900d4660 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbd9008ca90 tx=0x7fbd7802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18eedaa56f1640c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.263+0000 7fbd8e7fc640 1 -- 172.21.15.82:0/1374352204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78034070 con 0x7fbd900cf370 2024-03-14T12:16:52.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.263+0000 7fbd8e7fc640 1 -- 172.21.15.82:0/1374352204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78030860 con 0x7fbd900cf370 2024-03-14T12:16:52.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.263+0000 7fbd8e7fc640 1 -- 172.21.15.82:0/1374352204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78030c30 con 0x7fbd900cf370 2024-03-14T12:16:52.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.263+0000 7fbd952d7640 1 -- 172.21.15.82:0/1374352204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 msgr2=0x7fbd900d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.263+0000 7fbd952d7640 1 --2- 172.21.15.82:0/1374352204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd900d4660 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbd9008ca90 tx=0x7fbd7802b9a0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.264+0000 7fbd952d7640 1 -- 172.21.15.82:0/1374352204 shutdown_connections 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.264+0000 7fbd952d7640 1 --2- 172.21.15.82:0/1374352204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d1da0 0x7fbd900d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.264+0000 7fbd952d7640 1 --2- 172.21.15.82:0/1374352204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd900d4660 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.264+0000 7fbd952d7640 1 --2- 172.21.15.82:0/1374352204 >> v2:172.21.15.164:3300/0 conn(0x7fbd900d0760 0x7fbd900d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.264+0000 7fbd952d7640 1 -- 172.21.15.82:0/1374352204 >> 172.21.15.82:0/1374352204 conn(0x7fbd9009d030 msgr2=0x7fbd900d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.264+0000 7fbd952d7640 1 -- 172.21.15.82:0/1374352204 shutdown_connections 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.264+0000 7fbd952d7640 1 -- 172.21.15.82:0/1374352204 wait complete. 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 Processor -- start 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 -- start start 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd9015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d0760 0x7fbd90150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd900d1da0 0x7fbd900b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900d4ba0 con 0x7fbd900cf370 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900b7200 con 0x7fbd900d1da0 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.265+0000 7fbd952d7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd901617a0 con 0x7fbd900d0760 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd9015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd9015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34200/0 (socket says 172.21.15.82:34200) 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 -- 172.21.15.82:0/766266868 learned_addr learned my addr 172.21.15.82:0/766266868 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 -- 172.21.15.82:0/766266868 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d0760 msgr2=0x7fbd90150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d0760 0x7fbd90150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8f7fe640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd900d1da0 0x7fbd900b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 -- 172.21.15.82:0/766266868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd900d1da0 msgr2=0x7fbd900b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd900d1da0 0x7fbd900b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 -- 172.21.15.82:0/766266868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9015aa10 con 0x7fbd900cf370 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd8effd640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd9015a4d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000d800 tx=0x7fbd8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd877fe640 1 -- 172.21.15.82:0/766266868 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80018020 con 0x7fbd900cf370 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd877fe640 1 -- 172.21.15.82:0/766266868 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd800096a0 con 0x7fbd900cf370 2024-03-14T12:16:52.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd877fe640 1 -- 172.21.15.82:0/766266868 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80009a70 con 0x7fbd900cf370 2024-03-14T12:16:52.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9015acb0 con 0x7fbd900cf370 2024-03-14T12:16:52.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.266+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd900c8ea0 con 0x7fbd900cf370 2024-03-14T12:16:52.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.267+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9004f050 con 0x7fbd900cf370 2024-03-14T12:16:52.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.268+0000 7fbd877fe640 1 -- 172.21.15.82:0/766266868 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbd8001b020 con 0x7fbd900cf370 2024-03-14T12:16:52.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.268+0000 7fbd877fe640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd7006bbd0 0x7fbd7006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.268+0000 7fbd877fe640 1 -- 172.21.15.82:0/766266868 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbd8008a9f0 con 0x7fbd900cf370 2024-03-14T12:16:52.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.268+0000 7fbd87fff640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd7006bbd0 0x7fbd7006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.268+0000 7fbd87fff640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd7006bbd0 0x7fbd7006e090 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbd78036000 tx=0x7fbd7803b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.273+0000 7fbd877fe640 1 -- 172.21.15.82:0/766266868 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbd80022030 con 0x7fbd900cf370 2024-03-14T12:16:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.444+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd900ce110 con 0x7fbd900cf370 2024-03-14T12:16:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.445+0000 7fbd877fe640 1 -- 172.21.15.82:0/766266868 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13008 (secure 0 0 0) 0x7fbd80059750 con 0x7fbd900cf370 2024-03-14T12:16:52.445 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:52.445 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":61,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:51.098477+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/1422543802":"2024-03-15T12:16:48.091555+0000","172.21.15.82:6836/1422543802":"2024-03-15T12:16:48.091555+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd7006bbd0 msgr2=0x7fbd7006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd7006bbd0 0x7fbd7006e090 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbd78036000 tx=0x7fbd7803b040 comp rx=0 tx=0).stop 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 msgr2=0x7fbd9015a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd9015a4d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000d800 tx=0x7fbd8000dcc0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 shutdown_connections 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd900d1da0 0x7fbd900b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbd7006bbd0 0x7fbd7006e090 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd900d0760 0x7fbd90150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 --2- 172.21.15.82:0/766266868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd900cf370 0x7fbd9015a4d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 >> 172.21.15.82:0/766266868 conn(0x7fbd9009d030 msgr2=0x7fbd9006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:52.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 shutdown_connections 2024-03-14T12:16:52.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.447+0000 7fbd952d7640 1 -- 172.21.15.82:0/766266868 wait complete. 2024-03-14T12:16:52.458 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:52.458 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:52.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.562+0000 7f9ca8cde640 1 Processor -- start 2024-03-14T12:16:52.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.562+0000 7f9ca8cde640 1 -- start start 2024-03-14T12:16:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9ca8cde640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 0x7f9ca40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9ca8cde640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9ca415c2f0 0x7f9ca40af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9ca8cde640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ca40b1e00 0x7f9ca40b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9ca8cde640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40b62e0 con 0x7f9ca415c2f0 2024-03-14T12:16:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9ca8cde640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca406c740 con 0x7f9ca40b1e00 2024-03-14T12:16:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9ca8cde640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40670c0 con 0x7f9ca415bf10 2024-03-14T12:16:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9ca415c2f0 0x7f9ca40af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9ca415c2f0 0x7f9ca40af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34210/0 (socket says 172.21.15.82:34210) 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 -- 172.21.15.82:0/3504493571 learned_addr learned my addr 172.21.15.82:0/3504493571 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 -- 172.21.15.82:0/3504493571 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 msgr2=0x7f9ca40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 --2- 172.21.15.82:0/3504493571 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 0x7f9ca40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 -- 172.21.15.82:0/3504493571 >> v2:172.21.15.164:3300/0 conn(0x7f9ca40b1e00 msgr2=0x7f9ca40b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 --2- 172.21.15.82:0/3504493571 >> v2:172.21.15.164:3300/0 conn(0x7f9ca40b1e00 0x7f9ca40b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.563+0000 7f9c9bfff640 1 -- 172.21.15.82:0/3504493571 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca406c130 con 0x7f9ca415c2f0 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9c9bfff640 1 --2- 172.21.15.82:0/3504493571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40af5d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9ca409e430 tx=0x7f9c8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4479ae528f61e82f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca1d9b640 1 -- 172.21.15.82:0/3504493571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c02e070 con 0x7f9ca415c2f0 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca1d9b640 1 -- 172.21.15.82:0/3504493571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c037070 con 0x7f9ca415c2f0 2024-03-14T12:16:52.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca1d9b640 1 -- 172.21.15.82:0/3504493571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c031a00 con 0x7f9ca415c2f0 2024-03-14T12:16:52.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3504493571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 msgr2=0x7f9ca40af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3504493571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40af5d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9ca409e430 tx=0x7f9c8c02b880 comp rx=0 tx=0).stop 2024-03-14T12:16:52.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3504493571 shutdown_connections 2024-03-14T12:16:52.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3504493571 >> v2:172.21.15.164:3300/0 conn(0x7f9ca40b1e00 0x7f9ca40b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3504493571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40af5d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3504493571 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 0x7f9ca40b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.564+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3504493571 >> 172.21.15.82:0/3504493571 conn(0x7f9ca40b4fd0 msgr2=0x7f9ca40b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.565+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3504493571 shutdown_connections 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.565+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3504493571 wait complete. 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.566+0000 7f9ca8cde640 1 Processor -- start 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.566+0000 7f9ca8cde640 1 -- start start 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.566+0000 7f9ca8cde640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca40b1e00 0x7f9ca415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.566+0000 7f9ca8cde640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 0x7f9ca41f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.566+0000 7f9ca8cde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca8cde640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40afb10 con 0x7f9ca415c2f0 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca8cde640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca406c3d0 con 0x7f9ca40b1e00 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca8cde640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4145b60 con 0x7f9ca415bf10 2024-03-14T12:16:52.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34212/0 (socket says 172.21.15.82:34212) 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 -- 172.21.15.82:0/3867693164 learned_addr learned my addr 172.21.15.82:0/3867693164 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9c9bfff640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 0x7f9ca41f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca259c640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca40b1e00 0x7f9ca415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 -- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 msgr2=0x7f9ca41f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 0x7f9ca41f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 -- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca40b1e00 msgr2=0x7f9ca415b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca40b1e00 0x7f9ca415b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 -- 172.21.15.82:0/3867693164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40cf530 con 0x7f9ca415c2f0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca2d9d640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40cc800 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00b970 tx=0x7f9c9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9c9affd640 1 -- 172.21.15.82:0/3867693164 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c019070 con 0x7f9ca415c2f0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9c9affd640 1 -- 172.21.15.82:0/3867693164 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00cb60 con 0x7f9ca415c2f0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca40cf7c0 con 0x7f9ca415c2f0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9c9affd640 1 -- 172.21.15.82:0/3867693164 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c014440 con 0x7f9ca415c2f0 2024-03-14T12:16:52.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.567+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca40cfd80 con 0x7f9ca415c2f0 2024-03-14T12:16:52.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.568+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca404f050 con 0x7f9ca415c2f0 2024-03-14T12:16:52.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.569+0000 7f9c9affd640 1 -- 172.21.15.82:0/3867693164 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9c9c01c070 con 0x7f9ca415c2f0 2024-03-14T12:16:52.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.569+0000 7f9c9affd640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c8406bbd0 0x7f9c8406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.569+0000 7f9c9affd640 1 -- 172.21.15.82:0/3867693164 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9c9c08a430 con 0x7f9ca415c2f0 2024-03-14T12:16:52.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.569+0000 7f9ca259c640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c8406bbd0 0x7f9c8406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.570+0000 7f9ca259c640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c8406bbd0 0x7f9c8406e090 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009890 tx=0x7f9c94008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.574+0000 7f9c9affd640 1 -- 172.21.15.82:0/3867693164 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9c9c0592b0 con 0x7f9ca415c2f0 2024-03-14T12:16:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.744+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ca40d0550 con 0x7f9ca415c2f0 2024-03-14T12:16:52.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.744+0000 7f9c9affd640 1 -- 172.21.15.82:0/3867693164 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c9c058bb0 con 0x7f9ca415c2f0 2024-03-14T12:16:52.744 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c8406bbd0 msgr2=0x7f9c8406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c8406bbd0 0x7f9c8406e090 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009890 tx=0x7f9c94008040 comp rx=0 tx=0).stop 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 msgr2=0x7f9ca40cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40cc800 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00b970 tx=0x7f9c9c00be30 comp rx=0 tx=0).stop 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 shutdown_connections 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca415c2f0 0x7f9ca40cc800 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c8406bbd0 0x7f9c8406e090 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ca415bf10 0x7f9ca41f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 --2- 172.21.15.82:0/3867693164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ca40b1e00 0x7f9ca415b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 >> 172.21.15.82:0/3867693164 conn(0x7f9ca40b4fd0 msgr2=0x7f9ca40b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 shutdown_connections 2024-03-14T12:16:52.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.746+0000 7f9ca8cde640 1 -- 172.21.15.82:0/3867693164 wait complete. 2024-03-14T12:16:52.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_reconnect' 2024-03-14T12:16:52.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.858+0000 7f287aad1640 1 Processor -- start 2024-03-14T12:16:52.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.858+0000 7f287aad1640 1 -- start start 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287aad1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 0x7f287406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287aad1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f287415c220 0x7f287406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287aad1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28740ccb40 con 0x7f287415be40 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287aad1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28740cd750 con 0x7f287415c220 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287aad1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28740cce20 con 0x7f2874066b00 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287886d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f287415be40 0x7f28740d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53778/0 (socket says 172.21.15.82:53778) 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287886d640 1 -- 172.21.15.82:0/3557883297 learned_addr learned my addr 172.21.15.82:0/3557883297 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f2873fff640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 0x7f287406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f287906e640 1 --2- 172.21.15.82:0/3557883297 >> v2:172.21.15.164:3300/0 conn(0x7f287415c220 0x7f287406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248030676 0 0) 0x7f28740ccb40 con 0x7f287415be40 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2864003520 con 0x7f287415be40 2024-03-14T12:16:52.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1917737142 0 0) 0x7f2864003520 con 0x7f287415be40 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 msgr2=0x7f287406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 0x7f287406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 >> v2:172.21.15.164:3300/0 conn(0x7f287415c220 msgr2=0x7f287406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 --2- 172.21.15.82:0/3557883297 >> v2:172.21.15.164:3300/0 conn(0x7f287415c220 0x7f287406b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740cd100 con 0x7f287415be40 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f2868001670 con 0x7f287415be40 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 >> v1:172.21.15.82:6789/0 conn(0x7f287415be40 legacy=0x7f28740d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28640039b0 0x7f2864003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 0x7f2864004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2864004d40 0x7f28640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740cd750 con 0x7f2864004d40 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740cce20 con 0x7f28640039b0 2024-03-14T12:16:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.859+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286400b9c0 con 0x7f2874066b00 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f287906e640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2864004d40 0x7f28640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f287886d640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28640039b0 0x7f2864003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f2873fff640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 0x7f2864004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f287906e640 1 -- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 msgr2=0x7f2864004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f287906e640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2874066b00 0x7f2864004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f287906e640 1 -- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28640039b0 msgr2=0x7f2864003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f287906e640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28640039b0 0x7f2864003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.860+0000 7f287906e640 1 -- 172.21.15.82:0/3557883297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864003520 con 0x7f2864004d40 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287906e640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2864004d40 0x7f28640051c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f287415c630 tx=0x7f2864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad93a180b86c8c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c039070 con 0x7f2864004d40 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c02cbd0 con 0x7f2864004d40 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f28737fe640 1 -- 172.21.15.82:0/3557883297 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c009c10 con 0x7f2864004d40 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287aad1640 1 -- 172.21.15.82:0/3557883297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2864004d40 msgr2=0x7f28640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287aad1640 1 --2- 172.21.15.82:0/3557883297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2864004d40 0x7f28640051c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f287415c630 tx=0x7f2864005830 comp rx=0 tx=0).stop 2024-03-14T12:16:52.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287906e640 1 -- 172.21.15.82:0/3557883297 reap_dead start 2024-03-14T12:16:52.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287aad1640 1 -- 172.21.15.82:0/3557883297 shutdown_connections 2024-03-14T12:16:52.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287aad1640 1 -- 172.21.15.82:0/3557883297 >> 172.21.15.82:0/3557883297 conn(0x7f2874072310 msgr2=0x7f2874072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:52.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287aad1640 1 -- 172.21.15.82:0/3557883297 shutdown_connections 2024-03-14T12:16:52.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.861+0000 7f287aad1640 1 -- 172.21.15.82:0/3557883297 wait complete. 2024-03-14T12:16:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.862+0000 7f287aad1640 1 Processor -- start 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.862+0000 7f287aad1640 1 -- start start 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f287aad1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28740c9910 0x7f287409a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f287aad1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2874150cc0 0x7f287414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f287aad1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28741504d0 0x7f28740c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f287aad1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287406ae90 con 0x7f2874150cc0 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f287aad1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287406ad10 con 0x7f28741504d0 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f287aad1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287406b010 con 0x7f28740c9910 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f2873fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2874150cc0 0x7f287414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f2873fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2874150cc0 0x7f287414ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34238/0 (socket says 172.21.15.82:34238) 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.863+0000 7f2873fff640 1 -- 172.21.15.82:0/4068940841 learned_addr learned my addr 172.21.15.82:0/4068940841 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f287886d640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28740c9910 0x7f287409a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f287906e640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28741504d0 0x7f28740c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f2873fff640 1 -- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28740c9910 msgr2=0x7f287409a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f2873fff640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28740c9910 0x7f287409a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f2873fff640 1 -- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28741504d0 msgr2=0x7f28740c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f2873fff640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28741504d0 0x7f28740c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f2873fff640 1 -- 172.21.15.82:0/4068940841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740c42e0 con 0x7f2874150cc0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f287886d640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28740c9910 0x7f287409a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f287906e640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28741504d0 0x7f28740c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f2873fff640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2874150cc0 0x7f287414ff90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f28600119a0 tx=0x7f2860011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f28717fa640 1 -- 172.21.15.82:0/4068940841 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000faf0 con 0x7f2874150cc0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f28717fa640 1 -- 172.21.15.82:0/4068940841 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286000fc90 con 0x7f2874150cc0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28740c45c0 con 0x7f2874150cc0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f28717fa640 1 -- 172.21.15.82:0/4068940841 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860014440 con 0x7f2874150cc0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.864+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287414e2c0 con 0x7f2874150cc0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.865+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287404f050 con 0x7f2874150cc0 2024-03-14T12:16:52.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.865+0000 7f28717fa640 1 -- 172.21.15.82:0/4068940841 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2860012040 con 0x7f2874150cc0 2024-03-14T12:16:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.866+0000 7f28717fa640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f285806bc00 0x7f285806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.866+0000 7f28717fa640 1 -- 172.21.15.82:0/4068940841 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f286008ad70 con 0x7f2874150cc0 2024-03-14T12:16:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.866+0000 7f287886d640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f285806bc00 0x7f285806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.866+0000 7f287886d640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f285806bc00 0x7f285806e0c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2868004840 tx=0x7f2868009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:52.870+0000 7f28717fa640 1 -- 172.21.15.82:0/4068940841 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2860019030 con 0x7f2874150cc0 2024-03-14T12:16:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:53.108+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_reconnect"]} v 0) v1 -- 0x7f287406db90 con 0x7f2874150cc0 2024-03-14T12:16:54.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.115+0000 7f28717fa640 1 -- 172.21.15.82:0/4068940841 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_reconnect"]}]=0 v174) v1 ==== 187+0+0 (secure 0 0 0) 0x7f28600595a0 con 0x7f2874150cc0 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f285806bc00 msgr2=0x7f285806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f285806bc00 0x7f285806e0c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2868004840 tx=0x7f2868009040 comp rx=0 tx=0).stop 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2874150cc0 msgr2=0x7f287414ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2874150cc0 0x7f287414ff90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f28600119a0 tx=0x7f2860011e60 comp rx=0 tx=0).stop 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 shutdown_connections 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28741504d0 0x7f28740c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2874150cc0 0x7f287414ff90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f285806bc00 0x7f285806e0c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 --2- 172.21.15.82:0/4068940841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28740c9910 0x7f287409a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 >> 172.21.15.82:0/4068940841 conn(0x7f2874072310 msgr2=0x7f28740cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:54.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 shutdown_connections 2024-03-14T12:16:54.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.118+0000 7f287aad1640 1 -- 172.21.15.82:0/4068940841 wait complete. 2024-03-14T12:16:54.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-14T12:16:54.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.241+0000 7f2a8e537640 1 Processor -- start 2024-03-14T12:16:54.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.241+0000 7f2a8e537640 1 -- start start 2024-03-14T12:16:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.241+0000 7f2a8e537640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2a8815c0e0 0x7f2a880d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.241+0000 7f2a8e537640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 0x7f2a88160190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.241+0000 7f2a8e537640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a880cf8a0 con 0x7f2a881608c0 2024-03-14T12:16:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.241+0000 7f2a8e537640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a880d0ba0 con 0x7f2a8815c0e0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.241+0000 7f2a8e537640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880d01d0 con 0x7f2a8815fd00 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a877fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2a881608c0 0x7f2a880cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53792/0 (socket says 172.21.15.82:53792) 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a877fe640 1 -- 172.21.15.82:0/2417788151 learned_addr learned my addr 172.21.15.82:0/2417788151 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a8cad4640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 0x7f2a88160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a87fff640 1 --2- 172.21.15.82:0/2417788151 >> v2:172.21.15.164:3300/0 conn(0x7f2a8815c0e0 0x7f2a880d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361217461 0 0) 0x7f2a880cf8a0 con 0x7f2a881608c0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a70003520 con 0x7f2a881608c0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4071115521 0 0) 0x7f2a70003520 con 0x7f2a881608c0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 msgr2=0x7f2a88160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 0x7f2a88160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 >> v2:172.21.15.164:3300/0 conn(0x7f2a8815c0e0 msgr2=0x7f2a880d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 --2- 172.21.15.82:0/2417788151 >> v2:172.21.15.164:3300/0 conn(0x7f2a8815c0e0 0x7f2a880d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880cfb80 con 0x7f2a881608c0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f2a78001670 con 0x7f2a881608c0 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a87fff640 1 --2- 172.21.15.82:0/2417788151 >> v2:172.21.15.164:3300/0 conn(0x7f2a8815c0e0 0x7f2a880d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 >> v1:172.21.15.82:6789/0 conn(0x7f2a881608c0 legacy=0x7f2a880cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 0x7f2a70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a70004d40 0x7f2a700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880d0ba0 con 0x7f2a700039b0 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880d01d0 con 0x7f2a70004d40 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7000fdd0 con 0x7f2a8815fd00 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a87fff640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a877fe640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 0x7f2a70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.242+0000 7f2a8cad4640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a70004d40 0x7f2a700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a87fff640 1 -- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 msgr2=0x7f2a70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a87fff640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a8815fd00 0x7f2a70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a87fff640 1 -- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a70004d40 msgr2=0x7f2a700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a87fff640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a70004d40 0x7f2a700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a87fff640 1 -- 172.21.15.82:0/2417788151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70003520 con 0x7f2a700039b0 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a87fff640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a70003dd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009c90 tx=0x7f2a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5464c77f77394685 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c039070 con 0x7f2a700039b0 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c02c980 con 0x7f2a700039b0 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.243+0000 7f2a86ffd640 1 -- 172.21.15.82:0/2417788151 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c009860 con 0x7f2a700039b0 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.244+0000 7f2a8e537640 1 -- 172.21.15.82:0/2417788151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 msgr2=0x7f2a70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.244+0000 7f2a8e537640 1 --2- 172.21.15.82:0/2417788151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a70003dd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009c90 tx=0x7f2a70005830 comp rx=0 tx=0).stop 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.244+0000 7f2a8cad4640 1 -- 172.21.15.82:0/2417788151 reap_dead start 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.244+0000 7f2a8e537640 1 -- 172.21.15.82:0/2417788151 shutdown_connections 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.244+0000 7f2a8e537640 1 -- 172.21.15.82:0/2417788151 >> 172.21.15.82:0/2417788151 conn(0x7f2a880b3680 msgr2=0x7f2a880b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.244+0000 7f2a8e537640 1 -- 172.21.15.82:0/2417788151 shutdown_connections 2024-03-14T12:16:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.244+0000 7f2a8e537640 1 -- 172.21.15.82:0/2417788151 wait complete. 2024-03-14T12:16:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.245+0000 7f2a8e537640 1 Processor -- start 2024-03-14T12:16:54.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.245+0000 7f2a8e537640 1 -- start start 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8e537640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a881eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8e537640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88144710 0x7f2a881ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8e537640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a880b5360 0x7f2a880b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8e537640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815c4c0 con 0x7f2a700039b0 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8e537640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880cd960 con 0x7f2a880b5360 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8e537640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815c640 con 0x7f2a88144710 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a881eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a881eeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34248/0 (socket says 172.21.15.82:34248) 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 -- 172.21.15.82:0/696811208 learned_addr learned my addr 172.21.15.82:0/696811208 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8cad4640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a880b5360 0x7f2a880b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a877fe640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88144710 0x7f2a881ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 -- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88144710 msgr2=0x7f2a881ef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88144710 0x7f2a881ef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 -- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a880b5360 msgr2=0x7f2a880b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a880b5360 0x7f2a880b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a87fff640 1 -- 172.21.15.82:0/696811208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8815f360 con 0x7f2a700039b0 2024-03-14T12:16:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a8cad4640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a880b5360 0x7f2a880b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.246+0000 7f2a877fe640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88144710 0x7f2a881ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.247+0000 7f2a87fff640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a881eeb90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c037040 tx=0x7f2a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.247+0000 7f2a84ff9640 1 -- 172.21.15.82:0/696811208 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c039050 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.247+0000 7f2a84ff9640 1 -- 172.21.15.82:0/696811208 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c045070 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.247+0000 7f2a84ff9640 1 -- 172.21.15.82:0/696811208 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c009860 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.247+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a880b4880 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.247+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a880b4e40 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.248+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8804f050 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.248+0000 7f2a84ff9640 1 -- 172.21.15.82:0/696811208 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2a7c04a460 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.248+0000 7f2a84ff9640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a7406bbd0 0x7f2a7406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.249+0000 7f2a877fe640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a7406bbd0 0x7f2a7406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.249+0000 7f2a84ff9640 1 -- 172.21.15.82:0/696811208 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a7c0ac930 con 0x7f2a700039b0 2024-03-14T12:16:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.249+0000 7f2a877fe640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a7406bbd0 0x7f2a7406e090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000c00 tx=0x7f2a78009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:54.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.253+0000 7f2a84ff9640 1 -- 172.21.15.82:0/696811208 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2a7c07b720 con 0x7f2a700039b0 2024-03-14T12:16:54.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.423+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2a880c8eb0 con 0x7f2a700039b0 2024-03-14T12:16:54.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.429+0000 7f2a84ff9640 1 -- 172.21.15.82:0/696811208 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a7c07b020 con 0x7f2a700039b0 2024-03-14T12:16:54.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a7406bbd0 msgr2=0x7f2a7406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a7406bbd0 0x7f2a7406e090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000c00 tx=0x7f2a78009040 comp rx=0 tx=0).stop 2024-03-14T12:16:54.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 msgr2=0x7f2a881eeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a881eeb90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c037040 tx=0x7f2a7c004730 comp rx=0 tx=0).stop 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 shutdown_connections 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a880b5360 0x7f2a880b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f2a7406bbd0 0x7f2a7406e090 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88144710 0x7f2a881ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 --2- 172.21.15.82:0/696811208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a700039b0 0x7f2a881eeb90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 >> 172.21.15.82:0/696811208 conn(0x7f2a880b3680 msgr2=0x7f2a880ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 shutdown_connections 2024-03-14T12:16:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.431+0000 7f2a8e537640 1 -- 172.21.15.82:0/696811208 wait complete. 2024-03-14T12:16:54.442 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:16:54.442 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:54.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:16:54.530 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:16:54.530 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:54.530 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:54.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:16:54.545 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:54.545 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:54.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:16:54.563 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:16:54.563 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:54.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:16:54.600 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:16:54.600 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:16:54.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:16:54.619 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-14T12:16:54.619 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:16:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.740+0000 7f14c02a4640 1 Processor -- start 2024-03-14T12:16:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.740+0000 7f14c02a4640 1 -- start start 2024-03-14T12:16:54.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14c02a4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f14b80d0760 0x7f14b80d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14c02a4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80cf370 0x7f14b80d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14c02a4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80d1080 con 0x7f14b80d1da0 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14c02a4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80cf8c0 con 0x7f14b80d0760 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14c02a4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80cfba0 con 0x7f14b80cf370 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14be040640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f14b80d0760 0x7f14b80d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14be040640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f14b80d0760 0x7f14b80d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34262/0 (socket says 172.21.15.82:34262) 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.741+0000 7f14be040640 1 -- 172.21.15.82:0/864687961 learned_addr learned my addr 172.21.15.82:0/864687961 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.742+0000 7f14bd83f640 1 --2- 172.21.15.82:0/864687961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80cf370 0x7f14b80d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.742+0000 7f14be040640 1 -- 172.21.15.82:0/864687961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80cf370 msgr2=0x7f14b80d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.742+0000 7f14be040640 1 --2- 172.21.15.82:0/864687961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80cf370 0x7f14b80d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.742+0000 7f14be040640 1 -- 172.21.15.82:0/864687961 >> v1:172.21.15.164:6789/0 conn(0x7f14b80d1da0 legacy=0x7f14b80d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.742+0000 7f14be040640 1 -- 172.21.15.82:0/864687961 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80b6c40 con 0x7f14b80d0760 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.742+0000 7f14be040640 1 --2- 172.21.15.82:0/864687961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d0760 0x7f14b80d0b40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009e50 tx=0x7f14b402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9ca9915a123e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:54.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.742+0000 7f14bd03e640 1 -- 172.21.15.82:0/864687961 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b402cb00 con 0x7f14b80d0760 2024-03-14T12:16:54.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14bd03e640 1 -- 172.21.15.82:0/864687961 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b402cca0 con 0x7f14b80d0760 2024-03-14T12:16:54.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14bd03e640 1 -- 172.21.15.82:0/864687961 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4034610 con 0x7f14b80d0760 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 -- 172.21.15.82:0/864687961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d0760 msgr2=0x7f14b80d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 --2- 172.21.15.82:0/864687961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d0760 0x7f14b80d0b40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009e50 tx=0x7f14b402bf60 comp rx=0 tx=0).stop 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 -- 172.21.15.82:0/864687961 shutdown_connections 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 --2- 172.21.15.82:0/864687961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80cf370 0x7f14b80d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 --2- 172.21.15.82:0/864687961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d0760 0x7f14b80d0b40 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 -- 172.21.15.82:0/864687961 >> 172.21.15.82:0/864687961 conn(0x7f14b809d030 msgr2=0x7f14b80d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 -- 172.21.15.82:0/864687961 shutdown_connections 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.743+0000 7f14c02a4640 1 -- 172.21.15.82:0/864687961 wait complete. 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.744+0000 7f14c02a4640 1 Processor -- start 2024-03-14T12:16:54.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.744+0000 7f14c02a4640 1 -- start start 2024-03-14T12:16:54.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14c02a4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b80cf370 0x7f14b80cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14c02a4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80d0760 0x7f14b80c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14c02a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d1da0 0x7f14b8154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14c02a4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80d4ba0 con 0x7f14b80d1da0 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14c02a4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80b7200 con 0x7f14b80cf370 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14c02a4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b81617a0 con 0x7f14b80d0760 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d1da0 0x7f14b8154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d1da0 0x7f14b8154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34278/0 (socket says 172.21.15.82:34278) 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 -- 172.21.15.82:0/1174033905 learned_addr learned my addr 172.21.15.82:0/1174033905 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14bd83f640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80d0760 0x7f14b80c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be040640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b80cf370 0x7f14b80cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 -- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80d0760 msgr2=0x7f14b80c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80d0760 0x7f14b80c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 -- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b80cf370 msgr2=0x7f14b80cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b80cf370 0x7f14b80cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.745+0000 7f14be841640 1 -- 172.21.15.82:0/1174033905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80c7920 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.746+0000 7f14be841640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d1da0 0x7f14b8154480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c410 tx=0x7f14b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.746+0000 7f14aeffd640 1 -- 172.21.15.82:0/1174033905 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011790 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.746+0000 7f14aeffd640 1 -- 172.21.15.82:0/1174033905 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0017070 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.746+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b80c97a0 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.746+0000 7f14aeffd640 1 -- 172.21.15.82:0/1174033905 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0011bb0 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.746+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b80c94b0 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.747+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b80c7af0 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.747+0000 7f14aeffd640 1 -- 172.21.15.82:0/1174033905 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f14b001c460 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.747+0000 7f14aeffd640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f149c06bc00 0x7f149c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.748+0000 7f14aeffd640 1 -- 172.21.15.82:0/1174033905 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f14b008a340 con 0x7f14b80d1da0 2024-03-14T12:16:54.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.748+0000 7f14be040640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f149c06bc00 0x7f149c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:54.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.748+0000 7f14be040640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f149c06bc00 0x7f149c06e0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f14b4035000 tx=0x7f14b4030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:54.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.752+0000 7f14aeffd640 1 -- 172.21.15.82:0/1174033905 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f14b80c7af0 con 0x7f14b80d1da0 2024-03-14T12:16:54.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.959+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14b804f050 con 0x7f14b80d1da0 2024-03-14T12:16:54.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.959+0000 7f14aeffd640 1 -- 172.21.15.82:0/1174033905 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f14b0058b70 con 0x7f14b80d1da0 2024-03-14T12:16:54.959 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:54.959 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5111,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/4078909817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4078909817},{"type":"v1","addr":"172.21.15.164:6834","nonce":4078909817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2349002857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2349002857},{"type":"v1","addr":"172.21.15.164:6835","nonce":2349002857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/285826310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":285826310},{"type":"v1","addr":"172.21.15.82:6835","nonce":285826310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":6402,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1501620489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1501620489},{"type":"v1","addr":"172.21.15.82:6837","nonce":1501620489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2024-03-14T12:16:54.959 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 29 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f149c06bc00 msgr2=0x7f149c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f149c06bc00 0x7f149c06e0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f14b4035000 tx=0x7f14b4030000 comp rx=0 tx=0).stop 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d1da0 msgr2=0x7f14b8154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d1da0 0x7f14b8154480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c410 tx=0x7f14b000c8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 shutdown_connections 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14b80d1da0 0x7f14b8154480 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f149c06bc00 0x7f149c06e0c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14b80d0760 0x7f14b80c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 --2- 172.21.15.82:0/1174033905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14b80cf370 0x7f14b80cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.961+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 >> 172.21.15.82:0/1174033905 conn(0x7f14b809d030 msgr2=0x7f14b815ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:54.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.962+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 shutdown_connections 2024-03-14T12:16:54.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:54.962+0000 7f14c02a4640 1 -- 172.21.15.82:0/1174033905 wait complete. 2024-03-14T12:16:54.972 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T12:16:54.972 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-14T12:16:54.972 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T12:16:54.972 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:54.973 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T12:16:54.973 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-14T12:16:54.973 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T12:16:54.973 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:54.973 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T12:16:54.973 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-14T12:16:54.974 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T12:16:54.974 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:54.974 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T12:16:54.974 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-14T12:16:54.974 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T12:16:54.974 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:16:54.984 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T12:16:54.984 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T12:16:54.985 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T12:16:54.985 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T12:16:54.986 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T12:16:54.986 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T12:16:54.987 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T12:16:54.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T12:16:55.026 INFO:tasks.ceph.mds.a:Started 2024-03-14T12:16:55.027 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:55.027 INFO:tasks.ceph.mds.c:Started 2024-03-14T12:16:55.027 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:55.029 INFO:tasks.ceph.mds.b:Started 2024-03-14T12:16:55.029 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:55.030 INFO:tasks.ceph.mds.d:Started 2024-03-14T12:16:55.030 DEBUG:teuthology.parallel:result is None 2024-03-14T12:16:55.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-14T12:16:55.102 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-14T12:16:55.103 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:16:55.111 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-03-14T12:16:55.112 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-03-14T12:16:55.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.137+0000 7fc8e9680640 1 Processor -- start 2024-03-14T12:16:55.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.137+0000 7fc8e9680640 1 -- start start 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e9680640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 0x7fc8e40d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e9680640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40cf8c0 con 0x7fc8e40d0760 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e9680640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40cfba0 con 0x7fc8e40b7720 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e9680640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40b6c40 con 0x7fc8e40cf370 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e2ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc8e40d0760 0x7fc8e40d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53800/0 (socket says 172.21.15.82:53800) 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/10798497 learned_addr learned my addr 172.21.15.82:0/10798497 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e27fc640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 0x7fc8e40d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797519181 0 0) 0x7fc8e40cf8c0 con 0x7fc8e40d0760 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8cc003520 con 0x7fc8e40d0760 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 685476724 0 0) 0x7fc8cc003520 con 0x7fc8e40d0760 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 msgr2=0x7fc8e40d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:55.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 0x7fc8e40d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 >> v1:172.21.15.164:6789/0 conn(0x7fc8e40b7720 legacy=0x7fc8e40b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40b6f20 con 0x7fc8e40d0760 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fc8d4001670 con 0x7fc8e40d0760 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 >> v1:172.21.15.82:6789/0 conn(0x7fc8e40d0760 legacy=0x7fc8e40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.138+0000 7fc8e1ffb640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e1ffb640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8cc004390 0x7fc8cc004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e1ffb640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 0x7fc8cc0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40b6c40 con 0x7fc8cc0039e0 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc00b9f0 con 0x7fc8cc004390 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc00bbf0 con 0x7fc8e40cf370 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e27fc640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8cc004390 0x7fc8cc004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e37fe640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 0x7fc8cc0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 msgr2=0x7fc8cc0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 0x7fc8cc0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:55.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8cc004390 msgr2=0x7fc8cc004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8cc004390 0x7fc8cc004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/10798497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc003520 con 0x7fc8cc0039e0 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.139+0000 7fc8e37fe640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e40cf370 0x7fc8cc0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8cc003dc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c240 tx=0x7fc8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8ecadc331899e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4007d30 con 0x7fc8cc0039e0 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4030070 con 0x7fc8cc0039e0 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e1ffb640 1 -- 172.21.15.82:0/10798497 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d402cd60 con 0x7fc8cc0039e0 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e9680640 1 -- 172.21.15.82:0/10798497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 msgr2=0x7fc8cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e9680640 1 --2- 172.21.15.82:0/10798497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8cc003dc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c240 tx=0x7fc8cc005830 comp rx=0 tx=0).stop 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e37fe640 1 -- 172.21.15.82:0/10798497 reap_dead start 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e9680640 1 -- 172.21.15.82:0/10798497 shutdown_connections 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e9680640 1 -- 172.21.15.82:0/10798497 >> 172.21.15.82:0/10798497 conn(0x7fc8e409d030 msgr2=0x7fc8e40d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e9680640 1 -- 172.21.15.82:0/10798497 shutdown_connections 2024-03-14T12:16:55.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.140+0000 7fc8e9680640 1 -- 172.21.15.82:0/10798497 wait complete. 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 Processor -- start 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 -- start start 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8e415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e4154b80 0x7fc8e415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8e40cb020 0x7fc8e40cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4161920 con 0x7fc8cc0039e0 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41617a0 con 0x7fc8e40cb020 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e9680640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40d4660 con 0x7fc8e4154b80 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8e415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8e415a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34336/0 (socket says 172.21.15.82:34336) 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.142+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/2778191989 learned_addr learned my addr 172.21.15.82:0/2778191989 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e27fc640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e4154b80 0x7fc8e415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/2778191989 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e4154b80 msgr2=0x7fc8e415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e4154b80 0x7fc8e415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/2778191989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8e40cb020 msgr2=0x7fc8e40cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8e40cb020 0x7fc8e40cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/2778191989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40c94e0 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8e415a600 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4001970 tx=0x7fc8e40ce7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8cb7fe640 1 -- 172.21.15.82:0/2778191989 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4007d30 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8cb7fe640 1 -- 172.21.15.82:0/2778191989 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4004c30 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e415b2d0 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8cb7fe640 1 -- 172.21.15.82:0/2778191989 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4007070 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.143+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e415b890 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.144+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e404f050 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.144+0000 7fc8cb7fe640 1 -- 172.21.15.82:0/2778191989 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc8d4048020 con 0x7fc8cc0039e0 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.145+0000 7fc8cb7fe640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8d006bc00 0x7fc8d006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.145+0000 7fc8cb7fe640 1 -- 172.21.15.82:0/2778191989 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc8d40abc30 con 0x7fc8cc0039e0 2024-03-14T12:16:55.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.145+0000 7fc8e27fc640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8d006bc00 0x7fc8d006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:55.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.145+0000 7fc8e27fc640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8d006bc00 0x7fc8d006e0c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8e415a3f0 tx=0x7fc8d8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:55.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.149+0000 7fc8cb7fe640 1 -- 172.21.15.82:0/2778191989 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc8d407a990 con 0x7fc8cc0039e0 2024-03-14T12:16:55.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:55.331+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc8e40d0540 con 0x7fc8cc0039e0 2024-03-14T12:16:56.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.122+0000 7fc8cb7fe640 1 -- 172.21.15.82:0/2778191989 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc8d407a290 con 0x7fc8cc0039e0 2024-03-14T12:16:56.121 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-03-14T12:16:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.124+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8d006bc00 msgr2=0x7fc8d006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.124+0000 7fc8e9680640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8d006bc00 0x7fc8d006e0c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8e415a3f0 tx=0x7fc8d8008040 comp rx=0 tx=0).stop 2024-03-14T12:16:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.124+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 msgr2=0x7fc8e415a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.124+0000 7fc8e9680640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8e415a600 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4001970 tx=0x7fc8e40ce7b0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 shutdown_connections 2024-03-14T12:16:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8e40cb020 0x7fc8e40cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8d006bc00 0x7fc8d006e0c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc8e4154b80 0x7fc8e415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 --2- 172.21.15.82:0/2778191989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8cc0039e0 0x7fc8e415a600 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 >> 172.21.15.82:0/2778191989 conn(0x7fc8e409d030 msgr2=0x7fc8e40ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 shutdown_connections 2024-03-14T12:16:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.125+0000 7fc8e9680640 1 -- 172.21.15.82:0/2778191989 wait complete. 2024-03-14T12:16:56.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-14T12:16:56.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.242+0000 7f7e0d4e9640 1 Processor -- start 2024-03-14T12:16:56.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.242+0000 7f7e0d4e9640 1 -- start start 2024-03-14T12:16:56.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.243+0000 7f7e0d4e9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7e080d0760 0x7f7e080d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.243+0000 7f7e0d4e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 0x7f7e080b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.243+0000 7f7e0d4e9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e080d1080 con 0x7f7e080cf370 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.243+0000 7f7e0d4e9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e080cf8c0 con 0x7f7e080d0760 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.243+0000 7f7e0d4e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080cfba0 con 0x7f7e080b7720 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.243+0000 7f7e067fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7e080cf370 0x7f7e080d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53806/0 (socket says 172.21.15.82:53806) 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.243+0000 7f7e067fc640 1 -- 172.21.15.82:0/2898964323 learned_addr learned my addr 172.21.15.82:0/2898964323 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241206397 0 0) 0x7f7e080d1080 con 0x7f7e080cf370 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df0003520 con 0x7f7e080cf370 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2898964323 >> v2:172.21.15.164:3300/0 conn(0x7f7e080d0760 0x7f7e080d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e077fe640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 0x7f7e080b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590060314 0 0) 0x7f7df0003520 con 0x7f7e080cf370 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 msgr2=0x7f7e080b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 0x7f7e080b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 >> v2:172.21.15.164:3300/0 conn(0x7f7e080d0760 msgr2=0x7f7e080d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 --2- 172.21.15.82:0/2898964323 >> v2:172.21.15.164:3300/0 conn(0x7f7e080d0760 0x7f7e080d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080b6c40 con 0x7f7e080cf370 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f7dfc001670 con 0x7f7e080cf370 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 >> v1:172.21.15.82:6789/0 conn(0x7f7e080cf370 legacy=0x7f7e080d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 0x7f7df00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080cf8c0 con 0x7f7df00039b0 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080cfba0 con 0x7f7df0004800 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df000ba70 con 0x7f7e080b7720 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e067fc640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e077fe640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 0x7f7df00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e06ffd640 1 -- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 msgr2=0x7f7df00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080b7720 0x7f7df00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e06ffd640 1 -- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0004800 msgr2=0x7f7df0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e06ffd640 1 -- 172.21.15.82:0/2898964323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0003520 con 0x7f7df00039b0 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.244+0000 7f7e067fc640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df0004800 0x7f7df0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.245+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009ac0 tx=0x7f7df0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3132ab512a2f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.245+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8039070 con 0x7f7df00039b0 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.245+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df802c710 con 0x7f7df00039b0 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.245+0000 7f7e05ffb640 1 -- 172.21.15.82:0/2898964323 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8009690 con 0x7f7df00039b0 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.246+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2898964323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 msgr2=0x7f7df0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.246+0000 7f7e0d4e9640 1 --2- 172.21.15.82:0/2898964323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7df0003dd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009ac0 tx=0x7f7df0005830 comp rx=0 tx=0).stop 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.246+0000 7f7e077fe640 1 -- 172.21.15.82:0/2898964323 reap_dead start 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.246+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2898964323 shutdown_connections 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.246+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2898964323 >> 172.21.15.82:0/2898964323 conn(0x7f7e0809d030 msgr2=0x7f7e080d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.246+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2898964323 shutdown_connections 2024-03-14T12:16:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.246+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2898964323 wait complete. 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.247+0000 7f7e0d4e9640 1 Processor -- start 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.247+0000 7f7e0d4e9640 1 -- start start 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7e0815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e0814e900 0x7f7e080cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080cc740 0x7f7e0814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e081617a0 con 0x7f7df00039b0 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080b7200 con 0x7f7e0814e900 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08161920 con 0x7f7e080cc740 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7e0815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7e0815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34354/0 (socket says 172.21.15.82:34354) 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 -- 172.21.15.82:0/2245205775 learned_addr learned my addr 172.21.15.82:0/2245205775 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 -- 172.21.15.82:0/2245205775 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080cc740 msgr2=0x7f7e0814aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080cc740 0x7f7e0814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 -- 172.21.15.82:0/2245205775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e0814e900 msgr2=0x7f7e080cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e0814e900 0x7f7e080cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 -- 172.21.15.82:0/2245205775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080c7920 con 0x7f7df00039b0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e06ffd640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7e0815ce60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7df8000c00 tx=0x7f7df8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7def7fe640 1 -- 172.21.15.82:0/2245205775 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8039050 con 0x7f7df00039b0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7def7fe640 1 -- 172.21.15.82:0/2245205775 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df8045070 con 0x7f7df00039b0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7def7fe640 1 -- 172.21.15.82:0/2245205775 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80044e0 con 0x7f7df00039b0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e080c7c00 con 0x7f7df00039b0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.248+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e080c81c0 con 0x7f7df00039b0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.249+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0804f050 con 0x7f7df00039b0 2024-03-14T12:16:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.250+0000 7f7def7fe640 1 -- 172.21.15.82:0/2245205775 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7df8003a50 con 0x7f7df00039b0 2024-03-14T12:16:56.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.250+0000 7f7def7fe640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df406bc00 0x7f7df406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.250+0000 7f7e067fc640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df406bc00 0x7f7df406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.250+0000 7f7def7fe640 1 -- 172.21.15.82:0/2245205775 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7df80ac510 con 0x7f7df00039b0 2024-03-14T12:16:56.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.251+0000 7f7e067fc640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df406bc00 0x7f7df406e0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc001eb0 tx=0x7f7dfc001e20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.255+0000 7f7def7fe640 1 -- 172.21.15.82:0/2245205775 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7df804f020 con 0x7f7df00039b0 2024-03-14T12:16:56.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.494+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7e0815b960 con 0x7f7df00039b0 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.494+0000 7f7def7fe640 1 -- 172.21.15.82:0/2245205775 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7df80ac070 con 0x7f7df00039b0 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlAQmRIBAA45IMoIGH+bLaSIro/QUEmQ==", 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJljqfOLBAAulbwLxzEv+lHJOPUy3p4BQ==", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlt7bdJxAAH34bGUjVrlElC4nA11eNIQ==", 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlmeIuNBAAzNKyG5CdCVCURsWZywpTnw==", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDJ6fJlKwo/HBAA/Ma3FBjIdO1LF3RySRD0DQ==", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDM6fJlot+1GhAAEQqFqku2rdbqZ3rEBfVO2g==", 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDP6fJlPRIYGRAAUxqEbSYn6AhIE+3lXEqEOg==", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDS6fJlEYhJGBAApIl3qciLHwIJzDq5J2NkqA==", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDY6fJlL2cQHxAAeHY/pdO0TvpUciB22xvOUg==", 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDb6fJlrNxAHRAALwT6cIpCh9fE4NMoqV6PWA==", 2024-03-14T12:16:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDe6fJlPLi2GxAAeZ/Nk/340P6Y0ZJYojfNMw==", 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDh6fJlLhW0GhAAtvxrvm93pbGKTJbzzpIRiQ==", 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg==", 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlUlkBBhAA9QGPbDVg8uWX5WTR2W936g==", 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJlDQeZAhAAJW1K+HBJuwgzu4nVr8Mq8g==", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDG6fJldFHSCxAAci5DVZh9406uu1BZIAZXiw==", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-14T12:16:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlhoD/AxAA/LRLwtPkAs14z0fLh+eGjA==", 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJl5qEWHRAAI8LqGdvUwjHG/5pNgcAB1A==", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDF6fJlxtwQGRAArcSgLFwgKjmSqjq0sst8Gw==", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-14T12:16:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:16:56.507 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df406bc00 msgr2=0x7f7df406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df406bc00 0x7f7df406e0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc001eb0 tx=0x7f7dfc001e20 comp rx=0 tx=0).stop 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 msgr2=0x7f7e0815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7e0815ce60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7df8000c00 tx=0x7f7df8004730 comp rx=0 tx=0).stop 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 shutdown_connections 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7df406bc00 0x7f7df406e0c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e080cc740 0x7f7e0814aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e0814e900 0x7f7e080cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 --2- 172.21.15.82:0/2245205775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00039b0 0x7f7e0815ce60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 >> 172.21.15.82:0/2245205775 conn(0x7f7e0809d030 msgr2=0x7f7e080b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 shutdown_connections 2024-03-14T12:16:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.496+0000 7f7e0d4e9640 1 -- 172.21.15.82:0/2245205775 wait complete. 2024-03-14T12:16:56.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:16:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.610+0000 7f7a8c7f1640 1 Processor -- start 2024-03-14T12:16:56.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.610+0000 7f7a8c7f1640 1 -- start start 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8c7f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 0x7f7a840cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8c7f1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a840b1340 con 0x7f7a8406b420 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8c7f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a840cd030 con 0x7f7a840afff0 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8c7f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840b4ee0 con 0x7f7a840b0f60 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8ad8e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7a8406b420 0x7f7a840b5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53820/0 (socket says 172.21.15.82:53820) 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8ad8e640 1 -- 172.21.15.82:0/2530377800 learned_addr learned my addr 172.21.15.82:0/2530377800 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740352518 0 0) 0x7f7a840b1340 con 0x7f7a8406b420 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a70003520 con 0x7f7a8406b420 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.611+0000 7f7a8a58d640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 0x7f7a840cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2881494714 0 0) 0x7f7a70003520 con 0x7f7a8406b420 2024-03-14T12:16:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 msgr2=0x7f7a840cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 0x7f7a840cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 >> v1:172.21.15.164:6789/0 conn(0x7f7a840afff0 legacy=0x7f7a8406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840cd310 con 0x7f7a8406b420 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f7a7c0019f0 con 0x7f7a8406b420 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 >> v1:172.21.15.82:6789/0 conn(0x7f7a8406b420 legacy=0x7f7a840b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 0x7f7a700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a70003f60 0x7f7a70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840b4ee0 con 0x7f7a70004d40 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7000baa0 con 0x7f7a70003f60 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7000bca0 con 0x7f7a840b0f60 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8ad8e640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8a58d640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 0x7f7a700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a89d8c640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a70003f60 0x7f7a70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8ad8e640 1 -- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 msgr2=0x7f7a700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8ad8e640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 0x7f7a700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8ad8e640 1 -- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a70003f60 msgr2=0x7f7a70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8ad8e640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a70003f60 0x7f7a70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8ad8e640 1 -- 172.21.15.82:0/2530377800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70003520 con 0x7f7a70004d40 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a8a58d640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a840b0f60 0x7f7a700039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.612+0000 7f7a89d8c640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a70003f60 0x7f7a70004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.613+0000 7f7a8ad8e640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a700051c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00a830 tx=0x7f7a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853cb6c915180550 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.613+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00a480 con 0x7f7a70004d40 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.613+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c034630 con 0x7f7a70004d40 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.613+0000 7f7a8958b640 1 -- 172.21.15.82:0/2530377800 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c034aa0 con 0x7f7a70004d40 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.613+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/2530377800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 msgr2=0x7f7a700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.613+0000 7f7a8c7f1640 1 --2- 172.21.15.82:0/2530377800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a700051c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00a830 tx=0x7f7a70005830 comp rx=0 tx=0).stop 2024-03-14T12:16:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.613+0000 7f7a8ad8e640 1 -- 172.21.15.82:0/2530377800 reap_dead start 2024-03-14T12:16:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.614+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/2530377800 shutdown_connections 2024-03-14T12:16:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.614+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/2530377800 >> 172.21.15.82:0/2530377800 conn(0x7f7a84072310 msgr2=0x7f7a84072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.614+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/2530377800 shutdown_connections 2024-03-14T12:16:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.614+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/2530377800 wait complete. 2024-03-14T12:16:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 Processor -- start 2024-03-14T12:16:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 -- start start 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a84154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a8406d780 0x7f7a8406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a840cdb10 0x7f7a840cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840b0450 con 0x7f7a70004d40 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840cd8d0 con 0x7f7a840cdb10 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8c7f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840cc9a0 con 0x7f7a8406d780 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8a58d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a84154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8a58d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a84154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34366/0 (socket says 172.21.15.82:34366) 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.615+0000 7f7a8a58d640 1 -- 172.21.15.82:0/623795194 learned_addr learned my addr 172.21.15.82:0/623795194 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a89d8c640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a8406d780 0x7f7a8406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8ad8e640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a840cdb10 0x7f7a840cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8a58d640 1 -- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a8406d780 msgr2=0x7f7a8406dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8a58d640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a8406d780 0x7f7a8406dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8a58d640 1 -- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a840cdb10 msgr2=0x7f7a840cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8a58d640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a840cdb10 0x7f7a840cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8a58d640 1 -- 172.21.15.82:0/623795194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840ce7d0 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a89d8c640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a8406d780 0x7f7a8406dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8ad8e640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a840cdb10 0x7f7a840cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8a58d640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a84154e60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7a80011730 tx=0x7f7a80011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a7affd640 1 -- 172.21.15.82:0/623795194 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000f9c0 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a7affd640 1 -- 172.21.15.82:0/623795194 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8000fb60 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a840ceab0 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a7affd640 1 -- 172.21.15.82:0/623795194 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80015460 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.616+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a840cf070 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.617+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8404f050 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.617+0000 7f7a7affd640 1 -- 172.21.15.82:0/623795194 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7a80015600 con 0x7f7a70004d40 2024-03-14T12:16:56.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.618+0000 7f7a7affd640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a6806bbd0 0x7f7a6806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.618+0000 7f7a89d8c640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a6806bbd0 0x7f7a6806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.618+0000 7f7a7affd640 1 -- 172.21.15.82:0/623795194 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7a8008b170 con 0x7f7a70004d40 2024-03-14T12:16:56.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.618+0000 7f7a89d8c640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a6806bbd0 0x7f7a6806e090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7a840c6ed0 tx=0x7f7a74009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.622+0000 7f7a7affd640 1 -- 172.21.15.82:0/623795194 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7a80019030 con 0x7f7a70004d40 2024-03-14T12:16:56.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.795+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a84161220 con 0x7f7a70004d40 2024-03-14T12:16:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.796+0000 7f7a7affd640 1 -- 172.21.15.82:0/623795194 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f7a80059fb0 con 0x7f7a70004d40 2024-03-14T12:16:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:56.795 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":65,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:56.125538+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a6806bbd0 msgr2=0x7f7a6806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a6806bbd0 0x7f7a6806e090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7a840c6ed0 tx=0x7f7a74009040 comp rx=0 tx=0).stop 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 msgr2=0x7f7a84154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a84154e60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7a80011730 tx=0x7f7a80011bf0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 shutdown_connections 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a840cdb10 0x7f7a840cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7a6806bbd0 0x7f7a6806e090 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7a8406d780 0x7f7a8406dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 --2- 172.21.15.82:0/623795194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a70004d40 0x7f7a84154e60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 >> 172.21.15.82:0/623795194 conn(0x7f7a84072310 msgr2=0x7f7a840cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 shutdown_connections 2024-03-14T12:16:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.798+0000 7f7a8c7f1640 1 -- 172.21.15.82:0/623795194 wait complete. 2024-03-14T12:16:56.808 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:16:56.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:16:56.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.913+0000 7feee2d05640 1 Processor -- start 2024-03-14T12:16:56.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.914+0000 7feee2d05640 1 -- start start 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.914+0000 7feee2d05640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feedc15bde0 0x7feedc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.914+0000 7feee2d05640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 0x7feedc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.914+0000 7feee2d05640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc0ce380 con 0x7feedc0cd250 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.914+0000 7feee2d05640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0cef90 con 0x7feedc15bde0 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.914+0000 7feee2d05640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b5a30 con 0x7feedc0d08c0 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedbfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feedc0cd250 0x7feedc0d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53834/0 (socket says 172.21.15.82:53834) 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedbfff640 1 -- 172.21.15.82:0/4059357299 learned_addr learned my addr 172.21.15.82:0/4059357299 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feee12a2640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 0x7feedc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feee0aa1640 1 --2- 172.21.15.82:0/4059357299 >> v2:172.21.15.164:3300/0 conn(0x7feedc15bde0 0x7feedc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305613789 0 0) 0x7feedc0ce380 con 0x7feedc0cd250 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec4003520 con 0x7feedc0cd250 2024-03-14T12:16:56.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 296906381 0 0) 0x7feec4003520 con 0x7feedc0cd250 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 msgr2=0x7feedc0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 0x7feedc0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 >> v2:172.21.15.164:3300/0 conn(0x7feedc15bde0 msgr2=0x7feedc15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 --2- 172.21.15.82:0/4059357299 >> v2:172.21.15.164:3300/0 conn(0x7feedc15bde0 0x7feedc15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0ce660 con 0x7feedc0cd250 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7feed0001670 con 0x7feedc0cd250 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 >> v1:172.21.15.82:6789/0 conn(0x7feedc0cd250 legacy=0x7feedc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 0x7feec4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec4004d40 0x7feec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cef90 con 0x7feec4004d40 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b5a30 con 0x7feec40039b0 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec400b9c0 con 0x7feedc0d08c0 2024-03-14T12:16:56.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feee0aa1640 1 --2- 172.21.15.82:0/4059357299 >> v2:172.21.15.164:3300/0 conn(0x7feedc15bde0 0x7feedc15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feee12a2640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec4004d40 0x7feec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feedbfff640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 0x7feec4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.915+0000 7feee0aa1640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feee12a2640 1 -- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 msgr2=0x7feec4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feee12a2640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc0d08c0 0x7feec4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feee12a2640 1 -- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40039b0 msgr2=0x7feec4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feee12a2640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feee12a2640 1 -- 172.21.15.82:0/4059357299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4003520 con 0x7feec4004d40 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feee0aa1640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feee12a2640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec4004d40 0x7feec40051c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feed4009f90 tx=0x7feec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2db2aa73a31841b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4039070 con 0x7feec4004d40 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed402cbd0 con 0x7feec4004d40 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.916+0000 7feedb7fe640 1 -- 172.21.15.82:0/4059357299 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4009c10 con 0x7feec4004d40 2024-03-14T12:16:56.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.917+0000 7feee2d05640 1 -- 172.21.15.82:0/4059357299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec4004d40 msgr2=0x7feec40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.917+0000 7feee2d05640 1 --2- 172.21.15.82:0/4059357299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec4004d40 0x7feec40051c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feed4009f90 tx=0x7feec4005830 comp rx=0 tx=0).stop 2024-03-14T12:16:56.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.917+0000 7feee12a2640 1 -- 172.21.15.82:0/4059357299 reap_dead start 2024-03-14T12:16:56.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.917+0000 7feee2d05640 1 -- 172.21.15.82:0/4059357299 shutdown_connections 2024-03-14T12:16:56.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.917+0000 7feee2d05640 1 -- 172.21.15.82:0/4059357299 >> 172.21.15.82:0/4059357299 conn(0x7feedc0d1bd0 msgr2=0x7feedc0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:56.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.917+0000 7feee2d05640 1 -- 172.21.15.82:0/4059357299 shutdown_connections 2024-03-14T12:16:56.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.917+0000 7feee2d05640 1 -- 172.21.15.82:0/4059357299 wait complete. 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.918+0000 7feee2d05640 1 Processor -- start 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.918+0000 7feee2d05640 1 -- start start 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee2d05640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc150150 0x7feedc09c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee2d05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feedc15d540 0x7feedc15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee2d05640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feedc09d270 0x7feedc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee2d05640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cf420 con 0x7feedc15d540 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee2d05640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cec20 con 0x7feedc09d270 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee2d05640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cd690 con 0x7feedc150150 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feedc15d540 0x7feedc15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feedc15d540 0x7feedc15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34396/0 (socket says 172.21.15.82:34396) 2024-03-14T12:16:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 -- 172.21.15.82:0/3277956791 learned_addr learned my addr 172.21.15.82:0/3277956791 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee0aa1640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc150150 0x7feedc09c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee12a2640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feedc09d270 0x7feedc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 -- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc150150 msgr2=0x7feedc09c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc150150 0x7feedc09c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 -- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feedc09d270 msgr2=0x7feedc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feedc09d270 0x7feedc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feedbfff640 1 -- 172.21.15.82:0/3277956791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc15c1c0 con 0x7feedc15d540 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee0aa1640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc150150 0x7feedc09c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.919+0000 7feee12a2640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feedc09d270 0x7feedc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.920+0000 7feedbfff640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feedc15d540 0x7feedc15d9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feed00072c0 tx=0x7feed0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.920+0000 7feed97fa640 1 -- 172.21.15.82:0/3277956791 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed000fa80 con 0x7feedc15d540 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.920+0000 7feed97fa640 1 -- 172.21.15.82:0/3277956791 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed0018070 con 0x7feedc15d540 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.920+0000 7feed97fa640 1 -- 172.21.15.82:0/3277956791 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed000fea0 con 0x7feedc15d540 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.920+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc15c4a0 con 0x7feedc15d540 2024-03-14T12:16:56.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.920+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc15ca60 con 0x7feedc15d540 2024-03-14T12:16:56.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.921+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc04f050 con 0x7feedc15d540 2024-03-14T12:16:56.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.921+0000 7feed97fa640 1 -- 172.21.15.82:0/3277956791 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7feed001d460 con 0x7feedc15d540 2024-03-14T12:16:56.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.922+0000 7feed97fa640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feec806bc00 0x7feec806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:56.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.922+0000 7feee0aa1640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feec806bc00 0x7feec806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:56.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.922+0000 7feed97fa640 1 -- 172.21.15.82:0/3277956791 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feed008a960 con 0x7feedc15d540 2024-03-14T12:16:56.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.922+0000 7feee0aa1640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feec806bc00 0x7feec806e0c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feecc009c60 tx=0x7feecc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:56.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:56.926+0000 7feed97fa640 1 -- 172.21.15.82:0/3277956791 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7feed00597a0 con 0x7feedc15d540 2024-03-14T12:16:57.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.096+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feedc0b7b50 con 0x7feedc15d540 2024-03-14T12:16:57.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.097+0000 7feed97fa640 1 -- 172.21.15.82:0/3277956791 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7feed00590a0 con 0x7feedc15d540 2024-03-14T12:16:57.098 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:16:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feec806bc00 msgr2=0x7feec806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feec806bc00 0x7feec806e0c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feecc009c60 tx=0x7feecc008040 comp rx=0 tx=0).stop 2024-03-14T12:16:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feedc15d540 msgr2=0x7feedc15d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feedc15d540 0x7feedc15d9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feed00072c0 tx=0x7feed0007780 comp rx=0 tx=0).stop 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 shutdown_connections 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feedc09d270 0x7feedc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feedc15d540 0x7feedc15d9c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feec806bc00 0x7feec806e0c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 --2- 172.21.15.82:0/3277956791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feedc150150 0x7feedc09c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 >> 172.21.15.82:0/3277956791 conn(0x7feedc0d1bd0 msgr2=0x7feedc0d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 shutdown_connections 2024-03-14T12:16:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.099+0000 7feee2d05640 1 -- 172.21.15.82:0/3277956791 wait complete. 2024-03-14T12:16:57.109 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T12:16:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.211+0000 7fb32cc0b640 1 Processor -- start 2024-03-14T12:16:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.211+0000 7fb32cc0b640 1 -- start start 2024-03-14T12:16:57.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.212+0000 7fb32cc0b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3280b4190 0x7fb328160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.212+0000 7fb32cc0b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32806fc70 con 0x7fb32806ed90 2024-03-14T12:16:57.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.212+0000 7fb32cc0b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3280ce610 con 0x7fb3280cc770 2024-03-14T12:16:57.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.212+0000 7fb32cc0b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32815d340 con 0x7fb3280b4190 2024-03-14T12:16:57.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.212+0000 7fb32659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb32806ed90 0x7fb32806f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53846/0 (socket says 172.21.15.82:53846) 2024-03-14T12:16:57.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.212+0000 7fb32659c640 1 -- 172.21.15.82:0/3870959053 learned_addr learned my addr 172.21.15.82:0/3870959053 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695623524 0 0) 0x7fb318001490 con 0x7fb32806ed90 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb310003520 con 0x7fb32806ed90 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb325d9b640 1 --2- 172.21.15.82:0/3870959053 >> v2:172.21.15.164:3300/0 conn(0x7fb3280b4190 0x7fb328160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2473006889 0 0) 0x7fb310003520 con 0x7fb32806ed90 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 >> v2:172.21.15.164:3300/0 conn(0x7fb3280b4190 msgr2=0x7fb328160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 --2- 172.21.15.82:0/3870959053 >> v2:172.21.15.164:3300/0 conn(0x7fb3280b4190 0x7fb328160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 >> v1:172.21.15.164:6789/0 conn(0x7fb3280cc770 legacy=0x7fb3280ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3280ce8f0 con 0x7fb32806ed90 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fb318001bc0 con 0x7fb32806ed90 2024-03-14T12:16:57.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 >> v1:172.21.15.82:6789/0 conn(0x7fb32806ed90 legacy=0x7fb32806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3100039b0 0x7fb310003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb310004800 0x7fb310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb31000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32815d340 con 0x7fb3100051c0 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31000c0f0 con 0x7fb3100039b0 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31000c2f0 con 0x7fb310004800 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb326d9d640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb31000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb32659c640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3100039b0 0x7fb310003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb325d9b640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb310004800 0x7fb310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb326d9d640 1 -- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb310004800 msgr2=0x7fb310004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb326d9d640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb310004800 0x7fb310004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb326d9d640 1 -- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3100039b0 msgr2=0x7fb310003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:57.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb326d9d640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3100039b0 0x7fb310003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.213+0000 7fb326d9d640 1 -- 172.21.15.82:0/3870959053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310003520 con 0x7fb3100051c0 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.214+0000 7fb326d9d640 1 -- 172.21.15.82:0/3870959053 reap_dead start 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.214+0000 7fb326d9d640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb31000b9c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb320003d80 tx=0x7fb310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca26d28c4cf9e050 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.214+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320037070 con 0x7fb3100051c0 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.214+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32002be50 con 0x7fb3100051c0 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.214+0000 7fb32559a640 1 -- 172.21.15.82:0/3870959053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320003980 con 0x7fb3100051c0 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.215+0000 7fb32cc0b640 1 -- 172.21.15.82:0/3870959053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 msgr2=0x7fb31000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.215+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb31000b9c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb320003d80 tx=0x7fb310005830 comp rx=0 tx=0).stop 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.215+0000 7fb32cc0b640 1 -- 172.21.15.82:0/3870959053 shutdown_connections 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.215+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/3870959053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb31000b9c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.215+0000 7fb32cc0b640 1 -- 172.21.15.82:0/3870959053 >> 172.21.15.82:0/3870959053 conn(0x7fb3280b3670 msgr2=0x7fb3280b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.215+0000 7fb32cc0b640 1 -- 172.21.15.82:0/3870959053 shutdown_connections 2024-03-14T12:16:57.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.215+0000 7fb32cc0b640 1 -- 172.21.15.82:0/3870959053 wait complete. 2024-03-14T12:16:57.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.216+0000 7fb32cc0b640 1 Processor -- start 2024-03-14T12:16:57.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.216+0000 7fb32cc0b640 1 -- start start 2024-03-14T12:16:57.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32cc0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb32814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32cc0b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3280c7f10 0x7fb3280c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32cc0b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3280c8390 0x7fb3280c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32cc0b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280cf190 con 0x7fb3100051c0 2024-03-14T12:16:57.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32cc0b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280ceeb0 con 0x7fb3280c8390 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32cc0b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb328160420 con 0x7fb3280c7f10 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb32814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb32814fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34408/0 (socket says 172.21.15.82:34408) 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 -- 172.21.15.82:0/2912332612 learned_addr learned my addr 172.21.15.82:0/2912332612 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb325d9b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3280c7f10 0x7fb3280c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb326d9d640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3280c8390 0x7fb3280c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 -- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3280c7f10 msgr2=0x7fb3280c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3280c7f10 0x7fb3280c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 -- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3280c8390 msgr2=0x7fb3280c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3280c8390 0x7fb3280c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 -- 172.21.15.82:0/2912332612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328150440 con 0x7fb3100051c0 2024-03-14T12:16:57.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb326d9d640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3280c8390 0x7fb3280c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb325d9b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3280c7f10 0x7fb3280c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.217+0000 7fb32659c640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb32814fb90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb31800aaf0 tx=0x7fb3180016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.218+0000 7fb316ffd640 1 -- 172.21.15.82:0/2912332612 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318017020 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.218+0000 7fb316ffd640 1 -- 172.21.15.82:0/2912332612 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318007a30 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.218+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3281506e0 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.218+0000 7fb316ffd640 1 -- 172.21.15.82:0/2912332612 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318013630 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.218+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb328150ca0 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.219+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32804f050 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.219+0000 7fb316ffd640 1 -- 172.21.15.82:0/2912332612 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb31801a020 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.219+0000 7fb316ffd640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb30806bbd0 0x7fb30806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.219+0000 7fb316ffd640 1 -- 172.21.15.82:0/2912332612 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb31808b190 con 0x7fb3100051c0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.219+0000 7fb325d9b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb30806bbd0 0x7fb30806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.220+0000 7fb325d9b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb30806bbd0 0x7fb30806e090 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb3280cbcc0 tx=0x7fb31c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:57.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.224+0000 7fb316ffd640 1 -- 172.21.15.82:0/2912332612 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb318021020 con 0x7fb3100051c0 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.399+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb32806fa50 con 0x7fb3100051c0 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.400+0000 7fb316ffd640 1 -- 172.21.15.82:0/2912332612 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb32806fc70 con 0x7fb3100051c0 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-03-14T12:16:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:16:57.400 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-03-14T12:16:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb30806bbd0 msgr2=0x7fb30806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb30806bbd0 0x7fb30806e090 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb3280cbcc0 tx=0x7fb31c008040 comp rx=0 tx=0).stop 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 msgr2=0x7fb32814fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb32814fb90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb31800aaf0 tx=0x7fb3180016d0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 shutdown_connections 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3280c8390 0x7fb3280c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb30806bbd0 0x7fb30806e090 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3280c7f10 0x7fb3280c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 --2- 172.21.15.82:0/2912332612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3100051c0 0x7fb32814fb90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 >> 172.21.15.82:0/2912332612 conn(0x7fb3280b3670 msgr2=0x7fb3280cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 shutdown_connections 2024-03-14T12:16:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.402+0000 7fb32cc0b640 1 -- 172.21.15.82:0/2912332612 wait complete. 2024-03-14T12:16:57.412 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T12:16:57.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T12:16:57.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.517+0000 7fa9390bf640 1 Processor -- start 2024-03-14T12:16:57.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.517+0000 7fa9390bf640 1 -- start start 2024-03-14T12:16:57.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa9390bf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa934162430 0x7fa9340b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa9390bf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa9340b4130 0x7fa93406bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa9390bf640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9341629f0 con 0x7fa93415e1b0 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa9390bf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa934162c90 con 0x7fa934162430 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa9390bf640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9340b4560 con 0x7fa9340b4130 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa93259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa934162430 0x7fa9340b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa932d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa93415e1b0 0x7fa934161d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53852/0 (socket says 172.21.15.82:53852) 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa932d9d640 1 -- 172.21.15.82:0/3993804630 learned_addr learned my addr 172.21.15.82:0/3993804630 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa93359e640 1 --2- 172.21.15.82:0/3993804630 >> v2:172.21.15.164:3300/0 conn(0x7fa9340b4130 0x7fa93406bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa93259c640 1 -- 172.21.15.82:0/3993804630 >> v2:172.21.15.164:3300/0 conn(0x7fa9340b4130 msgr2=0x7fa93406bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa93259c640 1 --2- 172.21.15.82:0/3993804630 >> v2:172.21.15.164:3300/0 conn(0x7fa9340b4130 0x7fa93406bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa93259c640 1 -- 172.21.15.82:0/3993804630 >> v1:172.21.15.82:6789/0 conn(0x7fa93415e1b0 legacy=0x7fa934161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa931d9b640 1 -- 172.21.15.82:0/3993804630 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058047555 0 0) 0x7fa9341629f0 con 0x7fa93415e1b0 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa93259c640 1 -- 172.21.15.82:0/3993804630 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9340d0aa0 con 0x7fa934162430 2024-03-14T12:16:57.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.518+0000 7fa93359e640 1 --2- 172.21.15.82:0/3993804630 >> v2:172.21.15.164:3300/0 conn(0x7fa9340b4130 0x7fa93406bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:16:57.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.519+0000 7fa93259c640 1 --2- 172.21.15.82:0/3993804630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa934162430 0x7fa9340b2720 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa934072840 tx=0x7fa92802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a823f997832165f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.519+0000 7fa931d9b640 1 -- 172.21.15.82:0/3993804630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928035020 con 0x7fa934162430 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.519+0000 7fa931d9b640 1 -- 172.21.15.82:0/3993804630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa928031830 con 0x7fa934162430 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.519+0000 7fa931d9b640 1 -- 172.21.15.82:0/3993804630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928031c00 con 0x7fa934162430 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 -- 172.21.15.82:0/3993804630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa934162430 msgr2=0x7fa9340b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3993804630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa934162430 0x7fa9340b2720 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa934072840 tx=0x7fa92802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 -- 172.21.15.82:0/3993804630 shutdown_connections 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3993804630 >> v2:172.21.15.164:3300/0 conn(0x7fa9340b4130 0x7fa93406bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3993804630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa934162430 0x7fa9340b2720 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 -- 172.21.15.82:0/3993804630 >> 172.21.15.82:0/3993804630 conn(0x7fa9340b3610 msgr2=0x7fa9340b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 -- 172.21.15.82:0/3993804630 shutdown_connections 2024-03-14T12:16:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.520+0000 7fa9390bf640 1 -- 172.21.15.82:0/3993804630 wait complete. 2024-03-14T12:16:57.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.521+0000 7fa9390bf640 1 Processor -- start 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.521+0000 7fa9390bf640 1 -- start start 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa9390bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9340b4130 0x7fa9341582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa9390bf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa93415e1b0 0x7fa9340caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa9390bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa934162430 0x7fa93415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa9390bf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93415f5a0 con 0x7fa9340b4130 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa9390bf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9340d1020 con 0x7fa934162430 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa9390bf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93406c490 con 0x7fa93415e1b0 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9340b4130 0x7fa9341582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9340b4130 0x7fa9341582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34432/0 (socket says 172.21.15.82:34432) 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 -- 172.21.15.82:0/3497372629 learned_addr learned my addr 172.21.15.82:0/3497372629 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa93359e640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa934162430 0x7fa93415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa93259c640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa93415e1b0 0x7fa9340caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 -- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa93415e1b0 msgr2=0x7fa9340caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa93415e1b0 0x7fa9340caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 -- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa934162430 msgr2=0x7fa93415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa934162430 0x7fa93415cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 -- 172.21.15.82:0/3497372629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9341f78a0 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa932d9d640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9340b4130 0x7fa9341582c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c500 tx=0x7fa92400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa91b7fe640 1 -- 172.21.15.82:0/3497372629 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924018070 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.522+0000 7fa91b7fe640 1 -- 172.21.15.82:0/3497372629 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa924007580 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.523+0000 7fa91b7fe640 1 -- 172.21.15.82:0/3497372629 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924004a20 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.523+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9341f7b80 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.523+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9341f8140 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.524+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9340cf9d0 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.524+0000 7fa91b7fe640 1 -- 172.21.15.82:0/3497372629 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa924004bc0 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.524+0000 7fa91b7fe640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa92006bbd0 0x7fa92006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.524+0000 7fa91b7fe640 1 -- 172.21.15.82:0/3497372629 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9240899a0 con 0x7fa9340b4130 2024-03-14T12:16:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.524+0000 7fa93259c640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa92006bbd0 0x7fa92006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:57.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.525+0000 7fa93259c640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa92006bbd0 0x7fa92006e090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa928036000 tx=0x7fa928008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:57.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.529+0000 7fa91b7fe640 1 -- 172.21.15.82:0/3497372629 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa9340cf9d0 con 0x7fa9340b4130 2024-03-14T12:16:57.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:57.698+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa93404f050 con 0x7fa9340b4130 2024-03-14T12:16:58.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.128+0000 7fa91b7fe640 1 -- 172.21.15.82:0/3497372629 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa924058870 con 0x7fa9340b4130 2024-03-14T12:16:58.128 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-03-14T12:16:58.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa92006bbd0 msgr2=0x7fa92006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:58.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa92006bbd0 0x7fa92006e090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa928036000 tx=0x7fa928008030 comp rx=0 tx=0).stop 2024-03-14T12:16:58.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9340b4130 msgr2=0x7fa9341582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:58.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9340b4130 0x7fa9341582c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c500 tx=0x7fa92400c9c0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 shutdown_connections 2024-03-14T12:16:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa934162430 0x7fa93415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa92006bbd0 0x7fa92006e090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa93415e1b0 0x7fa9340caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 --2- 172.21.15.82:0/3497372629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9340b4130 0x7fa9341582c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 >> 172.21.15.82:0/3497372629 conn(0x7fa9340b3610 msgr2=0x7fa9340b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 shutdown_connections 2024-03-14T12:16:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.131+0000 7fa9390bf640 1 -- 172.21.15.82:0/3497372629 wait complete. 2024-03-14T12:16:58.145 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T12:16:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.256+0000 7f1e6ce30640 1 Processor -- start 2024-03-14T12:16:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.256+0000 7f1e6ce30640 1 -- start start 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e6ce30640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 0x7f1e680b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e6ce30640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e680706c0 con 0x7f1e6815dfd0 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e6ce30640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e680b3980 con 0x7f1e6815dbf0 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e6ce30640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68070940 con 0x7f1e680726b0 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e5ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1e6815dfd0 0x7f1e680b2c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53862/0 (socket says 172.21.15.82:53862) 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e5ffff640 1 -- 172.21.15.82:0/235721381 learned_addr learned my addr 172.21.15.82:0/235721381 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e66d9d640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 0x7f1e680b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703905406 0 0) 0x7f1e50001110 con 0x7f1e6815dfd0 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e54003520 con 0x7f1e6815dfd0 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224615445 0 0) 0x7f1e58001110 con 0x7f1e6815dbf0 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e50001110 con 0x7f1e6815dbf0 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2591196156 0 0) 0x7f1e54003520 con 0x7f1e6815dfd0 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 msgr2=0x7f1e680b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 0x7f1e680b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 >> v1:172.21.15.164:6789/0 conn(0x7f1e6815dbf0 legacy=0x7f1e68071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68070c20 con 0x7f1e6815dfd0 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f1e50001840 con 0x7f1e6815dfd0 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 >> v1:172.21.15.82:6789/0 conn(0x7f1e6815dfd0 legacy=0x7f1e680b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e54003b70 0x7f1e54003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 0x7f1e54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68070940 con 0x7f1e54004d40 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5400baa0 con 0x7f1e54003b70 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.257+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5400bca0 con 0x7f1e680726b0 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e66d9d640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e6659c640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e54003b70 0x7f1e54003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e5ffff640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 0x7f1e54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e66d9d640 1 -- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 msgr2=0x7f1e54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e66d9d640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e680726b0 0x7f1e54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e66d9d640 1 -- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e54003b70 msgr2=0x7f1e54003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e66d9d640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e54003b70 0x7f1e54003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e66d9d640 1 -- 172.21.15.82:0/235721381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54003520 con 0x7f1e54004d40 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.258+0000 7f1e66d9d640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e540051c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009fc0 tx=0x7f1e54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea33b72329ad9b7c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60039070 con 0x7f1e54004d40 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6002cc20 con 0x7f1e54004d40 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e65d9b640 1 -- 172.21.15.82:0/235721381 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60009c10 con 0x7f1e54004d40 2024-03-14T12:16:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e6ce30640 1 -- 172.21.15.82:0/235721381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 msgr2=0x7f1e540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:58.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e6ce30640 1 --2- 172.21.15.82:0/235721381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e540051c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009fc0 tx=0x7f1e54005830 comp rx=0 tx=0).stop 2024-03-14T12:16:58.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e66d9d640 1 -- 172.21.15.82:0/235721381 reap_dead start 2024-03-14T12:16:58.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e6ce30640 1 -- 172.21.15.82:0/235721381 shutdown_connections 2024-03-14T12:16:58.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e6ce30640 1 -- 172.21.15.82:0/235721381 >> 172.21.15.82:0/235721381 conn(0x7f1e680cf910 msgr2=0x7f1e680cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:58.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e6ce30640 1 -- 172.21.15.82:0/235721381 shutdown_connections 2024-03-14T12:16:58.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.259+0000 7f1e6ce30640 1 -- 172.21.15.82:0/235721381 wait complete. 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.260+0000 7f1e6ce30640 1 Processor -- start 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6ce30640 1 -- start start 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6ce30640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e68150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6ce30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ca90 0x7f1e68142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6ce30640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e681f6b10 0x7f1e68159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6ce30640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680b3360 con 0x7f1e54004d40 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6ce30640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680711e0 con 0x7f1e6815ca90 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6ce30640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680b34e0 con 0x7f1e681f6b10 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e68150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e68150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34450/0 (socket says 172.21.15.82:34450) 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e6659c640 1 -- 172.21.15.82:0/342408237 learned_addr learned my addr 172.21.15.82:0/342408237 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e66d9d640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e681f6b10 0x7f1e68159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.261+0000 7f1e5ffff640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ca90 0x7f1e68142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6659c640 1 -- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e681f6b10 msgr2=0x7f1e68159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6659c640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e681f6b10 0x7f1e68159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6659c640 1 -- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ca90 msgr2=0x7f1e68142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6659c640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ca90 0x7f1e68142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6659c640 1 -- 172.21.15.82:0/342408237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68145770 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6659c640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e68150ba0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800f810 tx=0x7f1e5800fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e5effd640 1 -- 172.21.15.82:0/342408237 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580052e0 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e5effd640 1 -- 172.21.15.82:0/342408237 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e58018070 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e5effd640 1 -- 172.21.15.82:0/342408237 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800d970 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e68142850 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.262+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6815bfb0 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.263+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6804f050 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.264+0000 7f1e5effd640 1 -- 172.21.15.82:0/342408237 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1e5800db10 con 0x7f1e54004d40 2024-03-14T12:16:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.264+0000 7f1e5effd640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e4806bbd0 0x7f1e4806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:58.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.264+0000 7f1e5effd640 1 -- 172.21.15.82:0/342408237 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1e5808a830 con 0x7f1e54004d40 2024-03-14T12:16:58.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.264+0000 7f1e5ffff640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e4806bbd0 0x7f1e4806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:58.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.264+0000 7f1e5ffff640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e4806bbd0 0x7f1e4806e090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1e681592f0 tx=0x7f1e50000f80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:58.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.269+0000 7f1e5effd640 1 -- 172.21.15.82:0/342408237 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1e68142850 con 0x7f1e54004d40 2024-03-14T12:16:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:58.440+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1e680af700 con 0x7f1e54004d40 2024-03-14T12:16:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.139+0000 7f1e5effd640 1 -- 172.21.15.82:0/342408237 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1e680b3980 con 0x7f1e54004d40 2024-03-14T12:16:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-03-14T12:16:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.141+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e4806bbd0 msgr2=0x7f1e4806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.141+0000 7f1e6ce30640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e4806bbd0 0x7f1e4806e090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1e681592f0 tx=0x7f1e50000f80 comp rx=0 tx=0).stop 2024-03-14T12:16:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.141+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 msgr2=0x7f1e68150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.141+0000 7f1e6ce30640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e68150ba0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800f810 tx=0x7f1e5800fcd0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 shutdown_connections 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e4806bbd0 0x7f1e4806e090 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e681f6b10 0x7f1e68159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e6815ca90 0x7f1e68142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 --2- 172.21.15.82:0/342408237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54004d40 0x7f1e68150ba0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 >> 172.21.15.82:0/342408237 conn(0x7f1e680cf910 msgr2=0x7f1e680ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 shutdown_connections 2024-03-14T12:16:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.142+0000 7f1e6ce30640 1 -- 172.21.15.82:0/342408237 wait complete. 2024-03-14T12:16:59.164 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T12:16:59.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.272+0000 7f472c068640 1 Processor -- start 2024-03-14T12:16:59.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.272+0000 7f472c068640 1 -- start start 2024-03-14T12:16:59.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472c068640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f47241549d0 0x7f472406daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:59.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472c068640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f472415e570 0x7f47240707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:59.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472c068640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47240b2f70 con 0x7f47240cf180 2024-03-14T12:16:59.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472c068640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4724154db0 con 0x7f47241549d0 2024-03-14T12:16:59.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472c068640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472406d610 con 0x7f472415e570 2024-03-14T12:16:59.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472a605640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f472415e570 0x7f47240707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:59.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f4729e04640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f47241549d0 0x7f472406daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:59.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472a605640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f472415e570 0x7f47240707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:59180/0 (socket says 172.21.15.82:59180) 2024-03-14T12:16:59.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f4729e04640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f47241549d0 0x7f472406daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34464/0 (socket says 172.21.15.82:34464) 2024-03-14T12:16:59.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f472a605640 1 -- 172.21.15.82:0/2091524352 learned_addr learned my addr 172.21.15.82:0/2091524352 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:59.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f4729e04640 1 -- 172.21.15.82:0/2091524352 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f472415e570 msgr2=0x7f47240707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f4729e04640 1 --2- 172.21.15.82:0/2091524352 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f472415e570 0x7f47240707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f4729e04640 1 -- 172.21.15.82:0/2091524352 >> v1:172.21.15.164:6789/0 conn(0x7f47240cf180 legacy=0x7f472415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.273+0000 7f4729e04640 1 -- 172.21.15.82:0/2091524352 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47240b29b0 con 0x7f47241549d0 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.274+0000 7f4729e04640 1 --2- 172.21.15.82:0/2091524352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f472406daa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f47200097a0 tx=0x7f472002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245c9561e334bb47 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.274+0000 7f4728e02640 1 -- 172.21.15.82:0/2091524352 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720034070 con 0x7f47241549d0 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.274+0000 7f4728e02640 1 -- 172.21.15.82:0/2091524352 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720030610 con 0x7f47241549d0 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.274+0000 7f4728e02640 1 -- 172.21.15.82:0/2091524352 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47200309e0 con 0x7f47241549d0 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 -- 172.21.15.82:0/2091524352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 msgr2=0x7f472406daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 --2- 172.21.15.82:0/2091524352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f472406daa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f47200097a0 tx=0x7f472002b960 comp rx=0 tx=0).stop 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 -- 172.21.15.82:0/2091524352 shutdown_connections 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 --2- 172.21.15.82:0/2091524352 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f472415e570 0x7f47240707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 --2- 172.21.15.82:0/2091524352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f472406daa0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 -- 172.21.15.82:0/2091524352 >> 172.21.15.82:0/2091524352 conn(0x7f472406aef0 msgr2=0x7f472406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:16:59.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 -- 172.21.15.82:0/2091524352 shutdown_connections 2024-03-14T12:16:59.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.275+0000 7f472c068640 1 -- 172.21.15.82:0/2091524352 wait complete. 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.276+0000 7f472c068640 1 Processor -- start 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.276+0000 7f472c068640 1 -- start start 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472c068640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47240cf180 0x7f4724140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472c068640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f4724143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472c068640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415e570 0x7f47240cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472c068640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472406d2a0 con 0x7f47241549d0 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472c068640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472406cfc0 con 0x7f472415e570 2024-03-14T12:16:59.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472c068640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472415ec30 con 0x7f47240cf180 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f4724143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f4724143a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34480/0 (socket says 172.21.15.82:34480) 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 -- 172.21.15.82:0/629735788 learned_addr learned my addr 172.21.15.82:0/629735788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729e04640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47240cf180 0x7f4724140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472a605640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415e570 0x7f47240cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 -- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47240cf180 msgr2=0x7f4724140850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47240cf180 0x7f4724140850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 -- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415e570 msgr2=0x7f47240cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415e570 0x7f47240cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 -- 172.21.15.82:0/629735788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47241f2dd0 con 0x7f47241549d0 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729e04640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47240cf180 0x7f4724140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f472a605640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415e570 0x7f47240cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f4729603640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f4724143a10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f471400c3c0 tx=0x7f471400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.277+0000 7f471a7fc640 1 -- 172.21.15.82:0/629735788 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140052d0 con 0x7f47241549d0 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.278+0000 7f471a7fc640 1 -- 172.21.15.82:0/629735788 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714005490 con 0x7f47241549d0 2024-03-14T12:16:59.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.278+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47241f21c0 con 0x7f47241549d0 2024-03-14T12:16:59.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.278+0000 7f471a7fc640 1 -- 172.21.15.82:0/629735788 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714007260 con 0x7f47241549d0 2024-03-14T12:16:59.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.278+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47241f2780 con 0x7f47241549d0 2024-03-14T12:16:59.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.278+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472404f050 con 0x7f47241549d0 2024-03-14T12:16:59.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.279+0000 7f471a7fc640 1 -- 172.21.15.82:0/629735788 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4714007400 con 0x7f47241549d0 2024-03-14T12:16:59.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.279+0000 7f471a7fc640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f470806bbd0 0x7f470806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:16:59.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.279+0000 7f471a7fc640 1 -- 172.21.15.82:0/629735788 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4714089ef0 con 0x7f47241549d0 2024-03-14T12:16:59.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.280+0000 7f4729e04640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f470806bbd0 0x7f470806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:16:59.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.280+0000 7f4729e04640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f470806bbd0 0x7f470806e090 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4720036000 tx=0x7f472003b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:16:59.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.284+0000 7f471a7fc640 1 -- 172.21.15.82:0/629735788 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4714058a40 con 0x7f47241549d0 2024-03-14T12:16:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:16:59.491+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4724140fd0 con 0x7f47241549d0 2024-03-14T12:17:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.149+0000 7f471a7fc640 1 -- 172.21.15.82:0/629735788 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v32) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4714058340 con 0x7f47241549d0 2024-03-14T12:17:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 10 and data pool 11 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f470806bbd0 msgr2=0x7f470806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f470806bbd0 0x7f470806e090 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4720036000 tx=0x7f472003b040 comp rx=0 tx=0).stop 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 msgr2=0x7f4724143a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f4724143a10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f471400c3c0 tx=0x7f471400c880 comp rx=0 tx=0).stop 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 shutdown_connections 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f472415e570 0x7f47240cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47241549d0 0x7f4724143a10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f470806bbd0 0x7f470806e090 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4720036000 tx=0x7f472003b040 comp rx=0 tx=0).stop 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 --2- 172.21.15.82:0/629735788 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47240cf180 0x7f4724140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.156+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 >> 172.21.15.82:0/629735788 conn(0x7f472406aef0 msgr2=0x7f47240af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:00.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.157+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 shutdown_connections 2024-03-14T12:17:00.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.158+0000 7f472c068640 1 -- 172.21.15.82:0/629735788 wait complete. 2024-03-14T12:17:00.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:17:00.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.279+0000 7f74d0f40640 1 Processor -- start 2024-03-14T12:17:00.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74d0f40640 1 -- start start 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74d0f40640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f74cc071740 0x7f74cc071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74d0f40640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f74cc15e670 0x7f74cc0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74d0f40640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc143f60 0x7f74cc15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74d0f40640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc072060 con 0x7f74cc071740 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74d0f40640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc15ebe0 con 0x7f74cc15e670 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74d0f40640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc15eec0 con 0x7f74cc143f60 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.280+0000 7f74ca59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f74cc071740 0x7f74cc071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74ca59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f74cc071740 0x7f74cc071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34484/0 (socket says 172.21.15.82:34484) 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74ca59c640 1 -- 172.21.15.82:0/4107617740 learned_addr learned my addr 172.21.15.82:0/4107617740 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74c3fff640 1 --2- 172.21.15.82:0/4107617740 >> v2:172.21.15.164:3300/0 conn(0x7f74cc15e670 0x7f74cc0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74cad9d640 1 --2- 172.21.15.82:0/4107617740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc143f60 0x7f74cc15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74ca59c640 1 -- 172.21.15.82:0/4107617740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc143f60 msgr2=0x7f74cc15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74ca59c640 1 --2- 172.21.15.82:0/4107617740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc143f60 0x7f74cc15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74ca59c640 1 -- 172.21.15.82:0/4107617740 >> v2:172.21.15.164:3300/0 conn(0x7f74cc15e670 msgr2=0x7f74cc0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74ca59c640 1 --2- 172.21.15.82:0/4107617740 >> v2:172.21.15.164:3300/0 conn(0x7f74cc15e670 0x7f74cc0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74ca59c640 1 -- 172.21.15.82:0/4107617740 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc15f1a0 con 0x7f74cc071740 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74c3fff640 1 --2- 172.21.15.82:0/4107617740 >> v2:172.21.15.164:3300/0 conn(0x7f74cc15e670 0x7f74cc0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.281+0000 7f74cad9d640 1 --2- 172.21.15.82:0/4107617740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc143f60 0x7f74cc15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74ca59c640 1 --2- 172.21.15.82:0/4107617740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc071740 0x7f74cc071b20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f74cc072ac0 tx=0x7f74bc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9e3c386a911e72 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74c9d9b640 1 -- 172.21.15.82:0/4107617740 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc035020 con 0x7f74cc071740 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74c9d9b640 1 -- 172.21.15.82:0/4107617740 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc02d040 con 0x7f74cc071740 2024-03-14T12:17:00.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74c9d9b640 1 -- 172.21.15.82:0/4107617740 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc031a80 con 0x7f74cc071740 2024-03-14T12:17:00.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 -- 172.21.15.82:0/4107617740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc071740 msgr2=0x7f74cc071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 --2- 172.21.15.82:0/4107617740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc071740 0x7f74cc071b20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f74cc072ac0 tx=0x7f74bc02b900 comp rx=0 tx=0).stop 2024-03-14T12:17:00.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 -- 172.21.15.82:0/4107617740 shutdown_connections 2024-03-14T12:17:00.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 --2- 172.21.15.82:0/4107617740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc143f60 0x7f74cc15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 --2- 172.21.15.82:0/4107617740 >> v2:172.21.15.164:3300/0 conn(0x7f74cc15e670 0x7f74cc0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 --2- 172.21.15.82:0/4107617740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc071740 0x7f74cc071b20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 -- 172.21.15.82:0/4107617740 >> 172.21.15.82:0/4107617740 conn(0x7f74cc0b1c70 msgr2=0x7f74cc0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 -- 172.21.15.82:0/4107617740 shutdown_connections 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.282+0000 7f74d0f40640 1 -- 172.21.15.82:0/4107617740 wait complete. 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 Processor -- start 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 -- start start 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc071740 0x7f74cc158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc143f60 0x7f74cc15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc15e670 0x7f74cc15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc15b080 con 0x7f74cc15e670 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc143ce0 con 0x7f74cc143f60 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74d0f40640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc144670 con 0x7f74cc071740 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc15e670 0x7f74cc15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc15e670 0x7f74cc15dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34492/0 (socket says 172.21.15.82:34492) 2024-03-14T12:17:00.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.284+0000 7f74cad9d640 1 -- 172.21.15.82:0/2052237003 learned_addr learned my addr 172.21.15.82:0/2052237003 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74ca59c640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc071740 0x7f74cc158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74c3fff640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc143f60 0x7f74cc15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74cad9d640 1 -- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc071740 msgr2=0x7f74cc158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74cad9d640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc071740 0x7f74cc158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74cad9d640 1 -- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc143f60 msgr2=0x7f74cc15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74cad9d640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc143f60 0x7f74cc15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74cad9d640 1 -- 172.21.15.82:0/2052237003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc13b380 con 0x7f74cc15e670 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74ca59c640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc071740 0x7f74cc158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74c3fff640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc143f60 0x7f74cc15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74cad9d640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc15e670 0x7f74cc15dc60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f74c4007e10 tx=0x7f74c400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74c2ffd640 1 -- 172.21.15.82:0/2052237003 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c401a070 con 0x7f74cc15e670 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74c2ffd640 1 -- 172.21.15.82:0/2052237003 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c4003b90 con 0x7f74cc15e670 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74c2ffd640 1 -- 172.21.15.82:0/2052237003 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004030 con 0x7f74cc15e670 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc13b660 con 0x7f74cc15e670 2024-03-14T12:17:00.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.285+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc13bc20 con 0x7f74cc15e670 2024-03-14T12:17:00.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.286+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc0cfdc0 con 0x7f74cc15e670 2024-03-14T12:17:00.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.286+0000 7f74c2ffd640 1 -- 172.21.15.82:0/2052237003 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f74c401d070 con 0x7f74cc15e670 2024-03-14T12:17:00.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.287+0000 7f74c2ffd640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f74ac06bc00 0x7f74ac06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.287+0000 7f74c2ffd640 1 -- 172.21.15.82:0/2052237003 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74c408ad00 con 0x7f74cc15e670 2024-03-14T12:17:00.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.287+0000 7f74ca59c640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f74ac06bc00 0x7f74ac06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.287+0000 7f74ca59c640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f74ac06bc00 0x7f74ac06e0c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.291+0000 7f74c2ffd640 1 -- 172.21.15.82:0/2052237003 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f74cc0cfdc0 con 0x7f74cc15e670 2024-03-14T12:17:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.464+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74cc071b20 con 0x7f74cc15e670 2024-03-14T12:17:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.465+0000 7f74c2ffd640 1 -- 172.21.15.82:0/2052237003 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f74cc071b20 con 0x7f74cc15e670 2024-03-14T12:17:00.465 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:00.465 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":68,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:59.492741+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:17:00.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f74ac06bc00 msgr2=0x7f74ac06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f74ac06bc00 0x7f74ac06e0c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc03c040 comp rx=0 tx=0).stop 2024-03-14T12:17:00.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc15e670 msgr2=0x7f74cc15dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc15e670 0x7f74cc15dc60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f74c4007e10 tx=0x7f74c400d4a0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 shutdown_connections 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74cc15e670 0x7f74cc15dc60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc143f60 0x7f74cc15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f74ac06bc00 0x7f74ac06e0c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 --2- 172.21.15.82:0/2052237003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc071740 0x7f74cc158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 >> 172.21.15.82:0/2052237003 conn(0x7f74cc0b1c70 msgr2=0x7f74cc0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 shutdown_connections 2024-03-14T12:17:00.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.467+0000 7f74d0f40640 1 -- 172.21.15.82:0/2052237003 wait complete. 2024-03-14T12:17:00.478 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:17:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.579+0000 7f387408c640 1 Processor -- start 2024-03-14T12:17:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.579+0000 7f387408c640 1 -- start start 2024-03-14T12:17:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f387408c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0d0760 0x7f386c0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f387408c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f386c0b7720 0x7f386c0b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f387408c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0d1080 con 0x7f386c0cf370 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f387408c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0cf8c0 con 0x7f386c0b7720 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f387408c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0cfba0 con 0x7f386c0d0760 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f3872629640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f386c0b7720 0x7f386c0b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f3872629640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f386c0b7720 0x7f386c0b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34494/0 (socket says 172.21.15.82:34494) 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.580+0000 7f3872629640 1 -- 172.21.15.82:0/3012210960 learned_addr learned my addr 172.21.15.82:0/3012210960 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.581+0000 7f3871e28640 1 --2- 172.21.15.82:0/3012210960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0d0760 0x7f386c0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.581+0000 7f3872629640 1 -- 172.21.15.82:0/3012210960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0d0760 msgr2=0x7f386c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.581+0000 7f3872629640 1 --2- 172.21.15.82:0/3012210960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0d0760 0x7f386c0d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.581+0000 7f3872629640 1 -- 172.21.15.82:0/3012210960 >> v1:172.21.15.164:6789/0 conn(0x7f386c0cf370 legacy=0x7f386c0d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.581+0000 7f3872629640 1 -- 172.21.15.82:0/3012210960 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0b6c40 con 0x7f386c0b7720 2024-03-14T12:17:00.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.581+0000 7f3871e28640 1 --2- 172.21.15.82:0/3012210960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0d0760 0x7f386c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.581+0000 7f3872629640 1 --2- 172.21.15.82:0/3012210960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0b7720 0x7f386c0b7b90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3864009e50 tx=0x7f386402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648cc46919367fa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f3870e26640 1 -- 172.21.15.82:0/3012210960 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386402cb90 con 0x7f386c0b7720 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f3870e26640 1 -- 172.21.15.82:0/3012210960 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386402cd30 con 0x7f386c0b7720 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f3870e26640 1 -- 172.21.15.82:0/3012210960 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864034610 con 0x7f386c0b7720 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 -- 172.21.15.82:0/3012210960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0b7720 msgr2=0x7f386c0b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 --2- 172.21.15.82:0/3012210960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0b7720 0x7f386c0b7b90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3864009e50 tx=0x7f386402bff0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 -- 172.21.15.82:0/3012210960 shutdown_connections 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 --2- 172.21.15.82:0/3012210960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0b7720 0x7f386c0b7b90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 --2- 172.21.15.82:0/3012210960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0d0760 0x7f386c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 -- 172.21.15.82:0/3012210960 >> 172.21.15.82:0/3012210960 conn(0x7f386c09d030 msgr2=0x7f386c0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 -- 172.21.15.82:0/3012210960 shutdown_connections 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.582+0000 7f387408c640 1 -- 172.21.15.82:0/3012210960 wait complete. 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.583+0000 7f387408c640 1 Processor -- start 2024-03-14T12:17:00.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.583+0000 7f387408c640 1 -- start start 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f387408c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0b7720 0x7f386c155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f387408c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386c0cf370 0x7f386c153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f387408c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0d0760 0x7f386c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f387408c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c1617a0 con 0x7f386c0d0760 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f387408c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b7200 con 0x7f386c0cf370 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f387408c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c161920 con 0x7f386c0b7720 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f3872629640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0d0760 0x7f386c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f3872629640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0d0760 0x7f386c157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34504/0 (socket says 172.21.15.82:34504) 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f3872629640 1 -- 172.21.15.82:0/1467578257 learned_addr learned my addr 172.21.15.82:0/1467578257 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f3871e28640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0b7720 0x7f386c155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.584+0000 7f3871627640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386c0cf370 0x7f386c153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3872629640 1 -- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0b7720 msgr2=0x7f386c155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3872629640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0b7720 0x7f386c155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3872629640 1 -- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386c0cf370 msgr2=0x7f386c153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3872629640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386c0cf370 0x7f386c153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3872629640 1 -- 172.21.15.82:0/1467578257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c155770 con 0x7f386c0d0760 2024-03-14T12:17:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3871e28640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0b7720 0x7f386c155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3871627640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386c0cf370 0x7f386c153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f3872629640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0d0760 0x7f386c157cb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3864036040 tx=0x7f3864004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f38627fc640 1 -- 172.21.15.82:0/1467578257 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864043020 con 0x7f386c0d0760 2024-03-14T12:17:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f38627fc640 1 -- 172.21.15.82:0/1467578257 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386403e450 con 0x7f386c0d0760 2024-03-14T12:17:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c0c7650 con 0x7f386c0d0760 2024-03-14T12:17:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f38627fc640 1 -- 172.21.15.82:0/1467578257 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864009510 con 0x7f386c0d0760 2024-03-14T12:17:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.585+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c0c7c10 con 0x7f386c0d0760 2024-03-14T12:17:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.586+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c04f050 con 0x7f386c0d0760 2024-03-14T12:17:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.586+0000 7f38627fc640 1 -- 172.21.15.82:0/1467578257 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3864046020 con 0x7f386c0d0760 2024-03-14T12:17:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.587+0000 7f38627fc640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f385006bbd0 0x7f385006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.587+0000 7f3871e28640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f385006bbd0 0x7f385006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.587+0000 7f38627fc640 1 -- 172.21.15.82:0/1467578257 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38640ad160 con 0x7f386c0d0760 2024-03-14T12:17:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.587+0000 7f3871e28640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f385006bbd0 0x7f385006e090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3868006fd0 tx=0x7f3868008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.591+0000 7f38627fc640 1 -- 172.21.15.82:0/1467578257 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f386404d030 con 0x7f386c0d0760 2024-03-14T12:17:00.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.766+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f386c0d2b90 con 0x7f386c0d0760 2024-03-14T12:17:00.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.767+0000 7f38627fc640 1 -- 172.21.15.82:0/1467578257 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f386407bc50 con 0x7f386c0d0760 2024-03-14T12:17:00.766 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:00.766 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":68,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:16:59.492741+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:17:00.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f385006bbd0 msgr2=0x7f385006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f385006bbd0 0x7f385006e090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3868006fd0 tx=0x7f3868008040 comp rx=0 tx=0).stop 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0d0760 msgr2=0x7f386c157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0d0760 0x7f386c157cb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3864036040 tx=0x7f3864004a40 comp rx=0 tx=0).stop 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 shutdown_connections 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0d0760 0x7f386c157cb0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386c0cf370 0x7f386c153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f385006bbd0 0x7f385006e090 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 --2- 172.21.15.82:0/1467578257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f386c0b7720 0x7f386c155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 >> 172.21.15.82:0/1467578257 conn(0x7f386c09d030 msgr2=0x7f386c0d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 shutdown_connections 2024-03-14T12:17:00.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.769+0000 7f387408c640 1 -- 172.21.15.82:0/1467578257 wait complete. 2024-03-14T12:17:00.779 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T12:17:00.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.885+0000 7f0277685640 1 Processor -- start 2024-03-14T12:17:00.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.885+0000 7f0277685640 1 -- start start 2024-03-14T12:17:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0277685640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027015f8a0 0x7f027006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0277685640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f027006b250 0x7f02700b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0277685640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02700b3bd0 con 0x7f0270145830 2024-03-14T12:17:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0277685640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027015fea0 con 0x7f027006b250 2024-03-14T12:17:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0277685640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0270160180 con 0x7f027015f8a0 2024-03-14T12:17:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0274c20640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027015f8a0 0x7f027006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0275421640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0270145830 0x7f027015cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:35250/0 (socket says 172.21.15.82:35250) 2024-03-14T12:17:00.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0275421640 1 -- 172.21.15.82:0/2019806356 learned_addr learned my addr 172.21.15.82:0/2019806356 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:00.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0275c22640 1 --2- 172.21.15.82:0/2019806356 >> v2:172.21.15.164:3300/0 conn(0x7f027006b250 0x7f02700b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0274c20640 1 -- 172.21.15.82:0/2019806356 >> v2:172.21.15.164:3300/0 conn(0x7f027006b250 msgr2=0x7f02700b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:00.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0274c20640 1 --2- 172.21.15.82:0/2019806356 >> v2:172.21.15.164:3300/0 conn(0x7f027006b250 0x7f02700b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0274c20640 1 -- 172.21.15.82:0/2019806356 >> v1:172.21.15.164:6789/0 conn(0x7f0270145830 legacy=0x7f027015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0274c20640 1 -- 172.21.15.82:0/2019806356 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270160460 con 0x7f027015f8a0 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.886+0000 7f0275c22640 1 --2- 172.21.15.82:0/2019806356 >> v2:172.21.15.164:3300/0 conn(0x7f027006b250 0x7f02700b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.887+0000 7f0274c20640 1 --2- 172.21.15.82:0/2019806356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027015f8a0 0x7f027006ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02600097a0 tx=0x7f026002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81d4c8e36a7d6bd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.887+0000 7f0267fff640 1 -- 172.21.15.82:0/2019806356 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02600315a0 con 0x7f027015f8a0 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.887+0000 7f0267fff640 1 -- 172.21.15.82:0/2019806356 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0260037070 con 0x7f027015f8a0 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.887+0000 7f0267fff640 1 -- 172.21.15.82:0/2019806356 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260031960 con 0x7f027015f8a0 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 -- 172.21.15.82:0/2019806356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027015f8a0 msgr2=0x7f027006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 --2- 172.21.15.82:0/2019806356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027015f8a0 0x7f027006ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02600097a0 tx=0x7f026002b840 comp rx=0 tx=0).stop 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 -- 172.21.15.82:0/2019806356 shutdown_connections 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 --2- 172.21.15.82:0/2019806356 >> v2:172.21.15.164:3300/0 conn(0x7f027006b250 0x7f02700b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 --2- 172.21.15.82:0/2019806356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f027015f8a0 0x7f027006ad10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 -- 172.21.15.82:0/2019806356 >> 172.21.15.82:0/2019806356 conn(0x7f02700b2830 msgr2=0x7f02700b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 -- 172.21.15.82:0/2019806356 shutdown_connections 2024-03-14T12:17:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.888+0000 7f0277685640 1 -- 172.21.15.82:0/2019806356 wait complete. 2024-03-14T12:17:00.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.889+0000 7f0277685640 1 Processor -- start 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.889+0000 7f0277685640 1 -- start start 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0277685640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f027006b250 0x7f02701f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0277685640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0270145830 0x7f02701f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0277685640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f027015f8a0 0x7f027015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0277685640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02700cf1f0 con 0x7f027006b250 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0277685640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02700cf070 con 0x7f027015f8a0 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0277685640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02700b45c0 con 0x7f0270145830 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f027006b250 0x7f02701f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f027006b250 0x7f02701f1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34506/0 (socket says 172.21.15.82:34506) 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 -- 172.21.15.82:0/3905159233 learned_addr learned my addr 172.21.15.82:0/3905159233 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0274c20640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0270145830 0x7f02701f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275c22640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f027015f8a0 0x7f027015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 -- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0270145830 msgr2=0x7f02701f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0270145830 0x7f02701f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 -- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f027015f8a0 msgr2=0x7f027015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f027015f8a0 0x7f027015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.890+0000 7f0275421640 1 -- 172.21.15.82:0/3905159233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270158270 con 0x7f027006b250 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.891+0000 7f0275421640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f027006b250 0x7f02701f1a50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c400 tx=0x7f026c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.891+0000 7f0265ffb640 1 -- 172.21.15.82:0/3905159233 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c017070 con 0x7f027006b250 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.891+0000 7f0265ffb640 1 -- 172.21.15.82:0/3905159233 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c005ce0 con 0x7f027006b250 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.891+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02701585b0 con 0x7f027006b250 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.891+0000 7f0265ffb640 1 -- 172.21.15.82:0/3905159233 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0072f0 con 0x7f027006b250 2024-03-14T12:17:00.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.891+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0270158df0 con 0x7f027006b250 2024-03-14T12:17:00.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.892+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027004f050 con 0x7f027006b250 2024-03-14T12:17:00.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.892+0000 7f0265ffb640 1 -- 172.21.15.82:0/3905159233 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f026c01a070 con 0x7f027006b250 2024-03-14T12:17:00.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.893+0000 7f0265ffb640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025406bbd0 0x7f025406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:00.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.893+0000 7f0265ffb640 1 -- 172.21.15.82:0/3905159233 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f026c089e20 con 0x7f027006b250 2024-03-14T12:17:00.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.893+0000 7f0274c20640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025406bbd0 0x7f025406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:00.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.893+0000 7f0274c20640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025406bbd0 0x7f025406e090 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0260036000 tx=0x7f026003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:00.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:00.897+0000 7f0265ffb640 1 -- 172.21.15.82:0/3905159233 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f02701585b0 con 0x7f027006b250 2024-03-14T12:17:01.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.105+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f02700b3690 con 0x7f027006b250 2024-03-14T12:17:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.154+0000 7f0265ffb640 1 -- 172.21.15.82:0/3905159233 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f026c058240 con 0x7f027006b250 2024-03-14T12:17:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025406bbd0 msgr2=0x7f025406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025406bbd0 0x7f025406e090 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0260036000 tx=0x7f026003d040 comp rx=0 tx=0).stop 2024-03-14T12:17:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f027006b250 msgr2=0x7f02701f1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f027006b250 0x7f02701f1a50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f026c00c400 tx=0x7f026c00c800 comp rx=0 tx=0).stop 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 shutdown_connections 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f027015f8a0 0x7f027015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f025406bbd0 0x7f025406e090 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0270145830 0x7f02701f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 --2- 172.21.15.82:0/3905159233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f027006b250 0x7f02701f1a50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 >> 172.21.15.82:0/3905159233 conn(0x7f02700b2830 msgr2=0x7f02700b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 shutdown_connections 2024-03-14T12:17:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.156+0000 7f0277685640 1 -- 172.21.15.82:0/3905159233 wait complete. 2024-03-14T12:17:01.166 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T12:17:01.166 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T12:17:01.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.269+0000 7f347fd3b640 1 Processor -- start 2024-03-14T12:17:01.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.270+0000 7f347fd3b640 1 -- start start 2024-03-14T12:17:01.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.270+0000 7f347fd3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34780d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:01.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.270+0000 7f347fd3b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34780d0c40 0x7f34780cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:01.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.270+0000 7f347fd3b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780d25a0 con 0x7f347815dff0 2024-03-14T12:17:01.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.270+0000 7f347fd3b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34780d10a0 con 0x7f34780d0c40 2024-03-14T12:17:01.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.270+0000 7f347fd3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780d2350 con 0x7f34780d12a0 2024-03-14T12:17:01.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347d2d6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34780d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:01.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347e2d8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34780d0c40 0x7f34780cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:01.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347d2d6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34780d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:59254/0 (socket says 172.21.15.82:59254) 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347e2d8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f34780d0c40 0x7f34780cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:53684/0 (socket says 172.21.15.82:53684) 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347d2d6640 1 -- 172.21.15.82:0/2164204035 learned_addr learned my addr 172.21.15.82:0/2164204035 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347dad7640 1 --1- 172.21.15.82:0/2164204035 >> v1:172.21.15.164:6789/0 conn(0x7f347815dff0 0x7f34780d4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:35254/0 (socket says 172.21.15.82:35254) 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347d2d6640 1 -- 172.21.15.82:0/2164204035 >> v2:172.21.15.164:3300/0 conn(0x7f34780d0c40 msgr2=0x7f34780cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347d2d6640 1 --2- 172.21.15.82:0/2164204035 >> v2:172.21.15.164:3300/0 conn(0x7f34780d0c40 0x7f34780cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347d2d6640 1 -- 172.21.15.82:0/2164204035 >> v1:172.21.15.164:6789/0 conn(0x7f347815dff0 legacy=0x7f34780d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347cad5640 1 -- 172.21.15.82:0/2164204035 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676549355 0 0) 0x7f34780d25a0 con 0x7f347815dff0 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347d2d6640 1 -- 172.21.15.82:0/2164204035 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780d2880 con 0x7f34780d12a0 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.271+0000 7f347e2d8640 1 --2- 172.21.15.82:0/2164204035 >> v2:172.21.15.164:3300/0 conn(0x7f34780d0c40 0x7f34780cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.272+0000 7f347d2d6640 1 --2- 172.21.15.82:0/2164204035 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34780d0700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3474009870 tx=0x7f347402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac50eedb39d9c1a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.272+0000 7f347cad5640 1 -- 172.21.15.82:0/2164204035 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474034070 con 0x7f34780d12a0 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.272+0000 7f347cad5640 1 -- 172.21.15.82:0/2164204035 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3474030840 con 0x7f34780d12a0 2024-03-14T12:17:01.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.272+0000 7f347cad5640 1 -- 172.21.15.82:0/2164204035 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474030c10 con 0x7f34780d12a0 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 -- 172.21.15.82:0/2164204035 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 msgr2=0x7f34780d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 --2- 172.21.15.82:0/2164204035 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34780d0700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3474009870 tx=0x7f347402b980 comp rx=0 tx=0).stop 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 -- 172.21.15.82:0/2164204035 shutdown_connections 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 --2- 172.21.15.82:0/2164204035 >> v2:172.21.15.164:3300/0 conn(0x7f34780d0c40 0x7f34780cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 --2- 172.21.15.82:0/2164204035 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34780d0700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 -- 172.21.15.82:0/2164204035 >> 172.21.15.82:0/2164204035 conn(0x7f347809d0d0 msgr2=0x7f34780d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 -- 172.21.15.82:0/2164204035 shutdown_connections 2024-03-14T12:17:01.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.273+0000 7f347fd3b640 1 -- 172.21.15.82:0/2164204035 wait complete. 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.274+0000 7f347fd3b640 1 Processor -- start 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.274+0000 7f347fd3b640 1 -- start start 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347fd3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34780d0c40 0x7f3478160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347fd3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34781f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347fd3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f347815dff0 0x7f34780ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347fd3b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347815e550 con 0x7f347815dff0 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347fd3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347815e3d0 con 0x7f34780d0c40 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347fd3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780cb860 con 0x7f34780d12a0 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f347815dff0 0x7f34780ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f347815dff0 0x7f34780ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34522/0 (socket says 172.21.15.82:34522) 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 -- 172.21.15.82:0/1826021288 learned_addr learned my addr 172.21.15.82:0/1826021288 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347dad7640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34780d0c40 0x7f3478160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347d2d6640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34781f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:01.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 -- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 msgr2=0x7f34781f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34781f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 -- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34780d0c40 msgr2=0x7f3478160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34780d0c40 0x7f3478160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 -- 172.21.15.82:0/1826021288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780c7f80 con 0x7f347815dff0 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.275+0000 7f347e2d8640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f347815dff0 0x7f34780ca160 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f347000efc0 tx=0x7f3470009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.276+0000 7f346e7fc640 1 -- 172.21.15.82:0/1826021288 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470018030 con 0x7f347815dff0 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.276+0000 7f346e7fc640 1 -- 172.21.15.82:0/1826021288 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347000cde0 con 0x7f347815dff0 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.276+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34780ca6a0 con 0x7f347815dff0 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.276+0000 7f346e7fc640 1 -- 172.21.15.82:0/1826021288 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470003bb0 con 0x7f347815dff0 2024-03-14T12:17:01.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.276+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34781610a0 con 0x7f347815dff0 2024-03-14T12:17:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.276+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347814aa00 con 0x7f347815dff0 2024-03-14T12:17:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.277+0000 7f346e7fc640 1 -- 172.21.15.82:0/1826021288 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3470005d10 con 0x7f347815dff0 2024-03-14T12:17:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.277+0000 7f346e7fc640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f345c06bbd0 0x7f345c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.277+0000 7f346e7fc640 1 -- 172.21.15.82:0/1826021288 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f347008b360 con 0x7f347815dff0 2024-03-14T12:17:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.277+0000 7f347dad7640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f345c06bbd0 0x7f345c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.278+0000 7f347dad7640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f345c06bbd0 0x7f345c06e090 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3468007fd0 tx=0x7f3468007de0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:01.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.282+0000 7f346e7fc640 1 -- 172.21.15.82:0/1826021288 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f347814aa00 con 0x7f347815dff0 2024-03-14T12:17:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:01.489+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f347804f050 con 0x7f347815dff0 2024-03-14T12:17:02.125 INFO:tasks.ceph.mds.b.smithi164.stderr:2024-03-14T12:17:02.126+0000 7ff1edec5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:17:02.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.161+0000 7f346e7fc640 1 -- 172.21.15.82:0/1826021288 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3470022070 con 0x7f347815dff0 2024-03-14T12:17:02.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.163+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f345c06bbd0 msgr2=0x7f345c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.163+0000 7f347fd3b640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f345c06bbd0 0x7f345c06e090 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3468007fd0 tx=0x7f3468007de0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f347815dff0 msgr2=0x7f34780ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f347815dff0 0x7f34780ca160 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f347000efc0 tx=0x7f3470009f90 comp rx=0 tx=0).stop 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 shutdown_connections 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f347815dff0 0x7f34780ca160 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f345c06bbd0 0x7f345c06e090 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34780d12a0 0x7f34781f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 --2- 172.21.15.82:0/1826021288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34780d0c40 0x7f3478160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 >> 172.21.15.82:0/1826021288 conn(0x7f347809d0d0 msgr2=0x7f347815d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 shutdown_connections 2024-03-14T12:17:02.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.164+0000 7f347fd3b640 1 -- 172.21.15.82:0/1826021288 wait complete. 2024-03-14T12:17:02.176 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T12:17:02.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:02.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.283+0000 7fa36ab0a640 1 Processor -- start 2024-03-14T12:17:02.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.283+0000 7fa36ab0a640 1 -- start start 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa36ab0a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3640b0200 0x7fa3640ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa36ab0a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3640cec70 con 0x7fa36406b670 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa36ab0a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3640d0410 con 0x7fa36406eb20 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa36ab0a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa364071810 con 0x7fa3640b0200 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa3688a6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa36406b670 0x7fa364072260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44456/0 (socket says 172.21.15.82:44456) 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa3688a6640 1 -- 172.21.15.82:0/851732295 learned_addr learned my addr 172.21.15.82:0/851732295 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa363fff640 1 --2- 172.21.15.82:0/851732295 >> v2:172.21.15.82:3300/0 conn(0x7fa3640b0200 0x7fa3640ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa363fff640 1 -- 172.21.15.82:0/851732295 >> v1:172.21.15.164:6789/0 conn(0x7fa36406eb20 legacy=0x7fa3640cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa363fff640 1 -- 172.21.15.82:0/851732295 >> v1:172.21.15.82:6789/0 conn(0x7fa36406b670 legacy=0x7fa364072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.284+0000 7fa363fff640 1 -- 172.21.15.82:0/851732295 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640d06f0 con 0x7fa3640b0200 2024-03-14T12:17:02.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.285+0000 7fa363fff640 1 --2- 172.21.15.82:0/851732295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3640b0200 0x7fa3640ce730 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa3580097a0 tx=0x7fa35802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12ea6b676cf0bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.285+0000 7fa3637fe640 1 -- 172.21.15.82:0/851732295 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358035020 con 0x7fa3640b0200 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.285+0000 7fa3637fe640 1 -- 172.21.15.82:0/851732295 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358031830 con 0x7fa3640b0200 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.285+0000 7fa3637fe640 1 -- 172.21.15.82:0/851732295 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358031c00 con 0x7fa3640b0200 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.286+0000 7fa36ab0a640 1 -- 172.21.15.82:0/851732295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3640b0200 msgr2=0x7fa3640ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.286+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/851732295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3640b0200 0x7fa3640ce730 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa3580097a0 tx=0x7fa35802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.286+0000 7fa36ab0a640 1 -- 172.21.15.82:0/851732295 shutdown_connections 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.286+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/851732295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3640b0200 0x7fa3640ce730 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.286+0000 7fa36ab0a640 1 -- 172.21.15.82:0/851732295 >> 172.21.15.82:0/851732295 conn(0x7fa3640d0b60 msgr2=0x7fa3640d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.286+0000 7fa36ab0a640 1 -- 172.21.15.82:0/851732295 shutdown_connections 2024-03-14T12:17:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.286+0000 7fa36ab0a640 1 -- 172.21.15.82:0/851732295 wait complete. 2024-03-14T12:17:02.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.287+0000 7fa36ab0a640 1 Processor -- start 2024-03-14T12:17:02.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.287+0000 7fa36ab0a640 1 -- start start 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa36ab0a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa36406b670 0x7fa3640c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa36ab0a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa36406eb20 0x7fa3640c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa36ab0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3640b0200 0x7fa3640c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa36ab0a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3640d0100 con 0x7fa36406eb20 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa36ab0a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa364071dd0 con 0x7fa3640b0200 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa36ab0a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3640cb8c0 con 0x7fa36406b670 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa36406eb20 0x7fa3640c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa36406eb20 0x7fa3640c34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46220/0 (socket says 172.21.15.82:46220) 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 -- 172.21.15.82:0/830952023 learned_addr learned my addr 172.21.15.82:0/830952023 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa3688a6640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa36406b670 0x7fa3640c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa3690a7640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3640b0200 0x7fa3640c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 -- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa36406b670 msgr2=0x7fa3640c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa36406b670 0x7fa3640c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 -- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3640b0200 msgr2=0x7fa3640c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3640b0200 0x7fa3640c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 -- 172.21.15.82:0/830952023 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640b4820 con 0x7fa36406eb20 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa3688a6640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa36406b670 0x7fa3640c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa3690a7640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3640b0200 0x7fa3640c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.288+0000 7fa363fff640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa36406eb20 0x7fa3640c34e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa358000c00 tx=0x7fa3580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.289+0000 7fa3617fa640 1 -- 172.21.15.82:0/830952023 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358040070 con 0x7fa36406eb20 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.289+0000 7fa3617fa640 1 -- 172.21.15.82:0/830952023 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358039a00 con 0x7fa36406eb20 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.289+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3640b4b00 con 0x7fa36406eb20 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.289+0000 7fa3617fa640 1 -- 172.21.15.82:0/830952023 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358035040 con 0x7fa36406eb20 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.289+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3640b50c0 con 0x7fa36406eb20 2024-03-14T12:17:02.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.290+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36404f050 con 0x7fa36406eb20 2024-03-14T12:17:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.290+0000 7fa3617fa640 1 -- 172.21.15.82:0/830952023 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa35804b060 con 0x7fa36406eb20 2024-03-14T12:17:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.290+0000 7fa3617fa640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa35006bbd0 0x7fa35006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.290+0000 7fa3617fa640 1 -- 172.21.15.82:0/830952023 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3580ac480 con 0x7fa36406eb20 2024-03-14T12:17:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.290+0000 7fa3688a6640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa35006bbd0 0x7fa35006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.291+0000 7fa3688a6640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa35006bbd0 0x7fa35006e090 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa3640b1bb0 tx=0x7fa354009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.295+0000 7fa3617fa640 1 -- 172.21.15.82:0/830952023 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa35807af70 con 0x7fa36406eb20 2024-03-14T12:17:02.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.500+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3641614d0 con 0x7fa36406eb20 2024-03-14T12:17:02.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.501+0000 7fa3617fa640 1 -- 172.21.15.82:0/830952023 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fa35807a870 con 0x7fa36406eb20 2024-03-14T12:17:02.501 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:02.501 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:01.490623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/4033072140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4033072140},{"type":"v1","addr":"172.21.15.164:6834","nonce":4033072140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:02.501 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 34 2024-03-14T12:17:02.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa35006bbd0 msgr2=0x7fa35006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa35006bbd0 0x7fa35006e090 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa3640b1bb0 tx=0x7fa354009040 comp rx=0 tx=0).stop 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa36406eb20 msgr2=0x7fa3640c34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa36406eb20 0x7fa3640c34e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa358000c00 tx=0x7fa3580046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 shutdown_connections 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3640b0200 0x7fa3640c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa36406eb20 0x7fa3640c34e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa35006bbd0 0x7fa35006e090 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 --2- 172.21.15.82:0/830952023 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa36406b670 0x7fa3640c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 >> 172.21.15.82:0/830952023 conn(0x7fa3640d0b60 msgr2=0x7fa3640d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 shutdown_connections 2024-03-14T12:17:02.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.503+0000 7fa36ab0a640 1 -- 172.21.15.82:0/830952023 wait complete. 2024-03-14T12:17:02.513 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:17:02.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.614+0000 7fdd651cd640 1 Processor -- start 2024-03-14T12:17:02.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.614+0000 7fdd651cd640 1 -- start start 2024-03-14T12:17:02.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd651cd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdd60071740 0x7fdd60071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd651cd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdd6015e670 0x7fdd600b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd651cd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd60072060 con 0x7fdd60143f60 2024-03-14T12:17:02.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd651cd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6015ebe0 con 0x7fdd60071740 2024-03-14T12:17:02.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd651cd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6015eec0 con 0x7fdd6015e670 2024-03-14T12:17:02.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdd60071740 0x7fdd60071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdd60071740 0x7fdd60071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46224/0 (socket says 172.21.15.82:46224) 2024-03-14T12:17:02.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/4063588209 learned_addr learned my addr 172.21.15.82:0/4063588209 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5e59c640 1 --2- 172.21.15.82:0/4063588209 >> v2:172.21.15.164:3300/0 conn(0x7fdd6015e670 0x7fdd600b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/4063588209 >> v2:172.21.15.164:3300/0 conn(0x7fdd6015e670 msgr2=0x7fdd600b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5ed9d640 1 --2- 172.21.15.82:0/4063588209 >> v2:172.21.15.164:3300/0 conn(0x7fdd6015e670 0x7fdd600b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/4063588209 >> v1:172.21.15.164:6789/0 conn(0x7fdd60143f60 legacy=0x7fdd6015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/4063588209 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6015f1a0 con 0x7fdd60071740 2024-03-14T12:17:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.615+0000 7fdd5e59c640 1 --2- 172.21.15.82:0/4063588209 >> v2:172.21.15.164:3300/0 conn(0x7fdd6015e670 0x7fdd600b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.616+0000 7fdd5ed9d640 1 --2- 172.21.15.82:0/4063588209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60071b20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fdd60072ac0 tx=0x7fdd5802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f048bd04adc7d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.616+0000 7fdd5dd9b640 1 -- 172.21.15.82:0/4063588209 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58034070 con 0x7fdd60071740 2024-03-14T12:17:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.616+0000 7fdd5dd9b640 1 -- 172.21.15.82:0/4063588209 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58030860 con 0x7fdd60071740 2024-03-14T12:17:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.616+0000 7fdd5dd9b640 1 -- 172.21.15.82:0/4063588209 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58030c30 con 0x7fdd60071740 2024-03-14T12:17:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 -- 172.21.15.82:0/4063588209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 msgr2=0x7fdd60071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 --2- 172.21.15.82:0/4063588209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60071b20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fdd60072ac0 tx=0x7fdd5802b9a0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 -- 172.21.15.82:0/4063588209 shutdown_connections 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 --2- 172.21.15.82:0/4063588209 >> v2:172.21.15.164:3300/0 conn(0x7fdd6015e670 0x7fdd600b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 --2- 172.21.15.82:0/4063588209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60071b20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 -- 172.21.15.82:0/4063588209 >> 172.21.15.82:0/4063588209 conn(0x7fdd600b1c70 msgr2=0x7fdd600cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 -- 172.21.15.82:0/4063588209 shutdown_connections 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.617+0000 7fdd651cd640 1 -- 172.21.15.82:0/4063588209 wait complete. 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.618+0000 7fdd651cd640 1 Processor -- start 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.618+0000 7fdd651cd640 1 -- start start 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd651cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd651cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd60143f60 0x7fdd6015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd651cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd6015e670 0x7fdd6013cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd651cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6015b080 con 0x7fdd60071740 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd651cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60143ce0 con 0x7fdd60143f60 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd651cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60144670 con 0x7fdd6015e670 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60139b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46236/0 (socket says 172.21.15.82:46236) 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/1374702564 learned_addr learned my addr 172.21.15.82:0/1374702564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5e59c640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd60143f60 0x7fdd6015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5f59e640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd6015e670 0x7fdd6013cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd6015e670 msgr2=0x7fdd6013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd6015e670 0x7fdd6013cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd60143f60 msgr2=0x7fdd6015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd60143f60 0x7fdd6015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 -- 172.21.15.82:0/1374702564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6013a0a0 con 0x7fdd60071740 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5f59e640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd6015e670 0x7fdd6013cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5e59c640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd60143f60 0x7fdd6015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd5ed9d640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60139b60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c00 tx=0x7fdd580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd477fe640 1 -- 172.21.15.82:0/1374702564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58034040 con 0x7fdd60071740 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd477fe640 1 -- 172.21.15.82:0/1374702564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58039d90 con 0x7fdd60071740 2024-03-14T12:17:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6015dee0 con 0x7fdd60071740 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.619+0000 7fdd477fe640 1 -- 172.21.15.82:0/1374702564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58043630 con 0x7fdd60071740 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.620+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6015c1c0 con 0x7fdd60071740 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.621+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6004f050 con 0x7fdd60071740 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.621+0000 7fdd477fe640 1 -- 172.21.15.82:0/1374702564 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdd58037070 con 0x7fdd60071740 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.621+0000 7fdd477fe640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdd4c06bbd0 0x7fdd4c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.621+0000 7fdd477fe640 1 -- 172.21.15.82:0/1374702564 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd580acc40 con 0x7fdd60071740 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.621+0000 7fdd5e59c640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdd4c06bbd0 0x7fdd4c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.622+0000 7fdd5e59c640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdd4c06bbd0 0x7fdd4c06e090 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd54006fd0 tx=0x7fdd54008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.626+0000 7fdd477fe640 1 -- 172.21.15.82:0/1374702564 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdd5803e070 con 0x7fdd60071740 2024-03-14T12:17:02.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.799+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd6013e440 con 0x7fdd60071740 2024-03-14T12:17:02.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.800+0000 7fdd477fe640 1 -- 172.21.15.82:0/1374702564 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16656 (secure 0 0 0) 0x7fdd5807b730 con 0x7fdd60071740 2024-03-14T12:17:02.800 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:02.800 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:17:01.141090+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:17:02.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.802+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdd4c06bbd0 msgr2=0x7fdd4c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.802+0000 7fdd651cd640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdd4c06bbd0 0x7fdd4c06e090 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd54006fd0 tx=0x7fdd54008040 comp rx=0 tx=0).stop 2024-03-14T12:17:02.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.802+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 msgr2=0x7fdd60139b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.802+0000 7fdd651cd640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60139b60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c00 tx=0x7fdd580046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 shutdown_connections 2024-03-14T12:17:02.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdd4c06bbd0 0x7fdd4c06e090 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd6015e670 0x7fdd6013cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd60143f60 0x7fdd6015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 --2- 172.21.15.82:0/1374702564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd60071740 0x7fdd60139b60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 >> 172.21.15.82:0/1374702564 conn(0x7fdd600b1c70 msgr2=0x7fdd6013efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:02.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 shutdown_connections 2024-03-14T12:17:02.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.803+0000 7fdd651cd640 1 -- 172.21.15.82:0/1374702564 wait complete. 2024-03-14T12:17:02.813 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T12:17:02.813 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:17:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.915+0000 7f6036953640 1 Processor -- start 2024-03-14T12:17:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.915+0000 7f6036953640 1 -- start start 2024-03-14T12:17:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6036953640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f603015dde0 0x7f603015e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6036953640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60300b3da0 con 0x7f6030151230 2024-03-14T12:17:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6036953640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60300b4080 con 0x7f60300b4b70 2024-03-14T12:17:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6036953640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6030070d40 con 0x7f603015dde0 2024-03-14T12:17:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6034ef0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f603015dde0 0x7f603015e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f602ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6030151230 0x7f60300cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44458/0 (socket says 172.21.15.82:44458) 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6034ef0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f603015dde0 0x7f603015e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46244/0 (socket says 172.21.15.82:46244) 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f602ffff640 1 -- 172.21.15.82:0/768992879 learned_addr learned my addr 172.21.15.82:0/768992879 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6034ef0640 1 -- 172.21.15.82:0/768992879 >> v1:172.21.15.164:6789/0 conn(0x7f60300b4b70 legacy=0x7f6030070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6034ef0640 1 -- 172.21.15.82:0/768992879 >> v1:172.21.15.82:6789/0 conn(0x7f6030151230 legacy=0x7f60300cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f6034ef0640 1 -- 172.21.15.82:0/768992879 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300b4360 con 0x7f603015dde0 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.916+0000 7f602effd640 1 -- 172.21.15.82:0/768992879 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010056537 0 0) 0x7f60300b3da0 con 0x7f6030151230 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.917+0000 7f6034ef0640 1 --2- 172.21.15.82:0/768992879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603015dde0 0x7f603015e210 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f602800bfa0 tx=0x7f602802e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be52688f2f567865 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.917+0000 7f602effd640 1 -- 172.21.15.82:0/768992879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602802ea50 con 0x7f603015dde0 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.917+0000 7f602effd640 1 -- 172.21.15.82:0/768992879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602802ebf0 con 0x7f603015dde0 2024-03-14T12:17:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.917+0000 7f602effd640 1 -- 172.21.15.82:0/768992879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028009540 con 0x7f603015dde0 2024-03-14T12:17:02.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.918+0000 7f6036953640 1 -- 172.21.15.82:0/768992879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603015dde0 msgr2=0x7f603015e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.918+0000 7f6036953640 1 --2- 172.21.15.82:0/768992879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603015dde0 0x7f603015e210 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f602800bfa0 tx=0x7f602802e0b0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.918+0000 7f6036953640 1 -- 172.21.15.82:0/768992879 shutdown_connections 2024-03-14T12:17:02.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.918+0000 7f6036953640 1 --2- 172.21.15.82:0/768992879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603015dde0 0x7f603015e210 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.918+0000 7f6036953640 1 -- 172.21.15.82:0/768992879 >> 172.21.15.82:0/768992879 conn(0x7f60300b1c70 msgr2=0x7f60300cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:02.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.918+0000 7f6036953640 1 -- 172.21.15.82:0/768992879 shutdown_connections 2024-03-14T12:17:02.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.918+0000 7f6036953640 1 -- 172.21.15.82:0/768992879 wait complete. 2024-03-14T12:17:02.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.919+0000 7f6036953640 1 Processor -- start 2024-03-14T12:17:02.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.919+0000 7f6036953640 1 -- start start 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60300b4b70 0x7f6030147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6030151230 0x7f60300ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f603015dde0 0x7f603014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603006c690 con 0x7f60300b4b70 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603006c3b0 con 0x7f603015dde0 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603015e750 con 0x7f6030151230 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60300b4b70 0x7f6030147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60300b4b70 0x7f6030147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46254/0 (socket says 172.21.15.82:46254) 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 -- 172.21.15.82:0/542556932 learned_addr learned my addr 172.21.15.82:0/542556932 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602f7fe640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6030151230 0x7f60300ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6034ef0640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f603015dde0 0x7f603014d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 -- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6030151230 msgr2=0x7f60300ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6030151230 0x7f60300ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 -- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f603015dde0 msgr2=0x7f603014d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f603015dde0 0x7f603014d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 -- 172.21.15.82:0/542556932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300cd790 con 0x7f60300b4b70 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6034ef0640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f603015dde0 0x7f603014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602f7fe640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6030151230 0x7f60300ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602ffff640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60300b4b70 0x7f6030147890 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f602000c500 tx=0x7f602000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602cff9640 1 -- 172.21.15.82:0/542556932 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020018070 con 0x7f60300b4b70 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602cff9640 1 -- 172.21.15.82:0/542556932 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020007580 con 0x7f60300b4b70 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60300cda70 con 0x7f60300b4b70 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f602cff9640 1 -- 172.21.15.82:0/542556932 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020004a20 con 0x7f60300b4b70 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.920+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60300ce030 con 0x7f60300b4b70 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.921+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603004f050 con 0x7f60300b4b70 2024-03-14T12:17:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.922+0000 7f602cff9640 1 -- 172.21.15.82:0/542556932 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f602001b070 con 0x7f60300b4b70 2024-03-14T12:17:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.922+0000 7f602cff9640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f601c06bc00 0x7f601c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.922+0000 7f602cff9640 1 -- 172.21.15.82:0/542556932 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6020089d50 con 0x7f60300b4b70 2024-03-14T12:17:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.922+0000 7f602f7fe640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f601c06bc00 0x7f601c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.923+0000 7f602f7fe640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f601c06bc00 0x7f601c06e0c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6030154ae0 tx=0x7f6024009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:02.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:02.927+0000 7f602cff9640 1 -- 172.21.15.82:0/542556932 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f602008d030 con 0x7f60300b4b70 2024-03-14T12:17:03.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.093+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60300b2f40 con 0x7f601c06bc00 2024-03-14T12:17:03.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.099+0000 7f602cff9640 1 -- 172.21.15.82:0/542556932 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240144 (secure 0 0 0) 0x7f60300b2f40 con 0x7f601c06bc00 2024-03-14T12:17:03.101 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:03.102 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:17:03.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f601c06bc00 msgr2=0x7f601c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f601c06bc00 0x7f601c06e0c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6030154ae0 tx=0x7f6024009040 comp rx=0 tx=0).stop 2024-03-14T12:17:03.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60300b4b70 msgr2=0x7f6030147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60300b4b70 0x7f6030147890 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f602000c500 tx=0x7f602000c9c0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 shutdown_connections 2024-03-14T12:17:03.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f603015dde0 0x7f603014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f601c06bc00 0x7f601c06e0c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6030151230 0x7f60300ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 --2- 172.21.15.82:0/542556932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60300b4b70 0x7f6030147890 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 >> 172.21.15.82:0/542556932 conn(0x7f60300b1c70 msgr2=0x7f603006cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:03.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 shutdown_connections 2024-03-14T12:17:03.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.102+0000 7f6036953640 1 -- 172.21.15.82:0/542556932 wait complete. 2024-03-14T12:17:03.112 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-03-14T12:17:01.536626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261028,"kb_used_data":7372,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713692,"statfs":{"total":773094113280,"available":772826820608,"internally_reserved":0,"allocated":7548928,"data_stored":3804931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":259575347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000601"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.139342+0000","last_change":"2024-03-14T12:16:58.139342+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.139342+0000","last_undegraded":"2024-03-14T12:16:58.139342+0000","last_fullsized":"2024-03-14T12:16:58.139342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.134124+0000","last_change":"2024-03-14T12:16:58.134124+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.134124+0000","last_undegraded":"2024-03-14T12:16:58.134124+0000","last_fullsized":"2024-03-14T12:16:58.134124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136759+0000","last_change":"2024-03-14T12:16:58.136759+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136759+0000","last_undegraded":"2024-03-14T12:16:58.136759+0000","last_fullsized":"2024-03-14T12:16:58.136759+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141131+0000","last_change":"2024-03-14T12:16:58.141131+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141131+0000","last_undegraded":"2024-03-14T12:16:58.141131+0000","last_fullsized":"2024-03-14T12:16:58.141131+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.138972+0000","last_change":"2024-03-14T12:16:58.138972+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.138972+0000","last_undegraded":"2024-03-14T12:16:58.138972+0000","last_fullsized":"2024-03-14T12:16:58.138972+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.142257+0000","last_change":"2024-03-14T12:16:58.142257+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.142257+0000","last_undegraded":"2024-03-14T12:16:58.142257+0000","last_fullsized":"2024-03-14T12:16:58.142257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137339+0000","last_change":"2024-03-14T12:16:58.137339+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137339+0000","last_undegraded":"2024-03-14T12:16:58.137339+0000","last_fullsized":"2024-03-14T12:16:58.137339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.135605+0000","last_change":"2024-03-14T12:16:58.135605+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.135605+0000","last_undegraded":"2024-03-14T12:16:58.135605+0000","last_fullsized":"2024-03-14T12:16:58.135605+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136433+0000","last_change":"2024-03-14T12:16:58.136433+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136433+0000","last_undegraded":"2024-03-14T12:16:58.136433+0000","last_fullsized":"2024-03-14T12:16:58.136433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141079+0000","last_change":"2024-03-14T12:16:58.141079+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141079+0000","last_undegraded":"2024-03-14T12:16:58.141079+0000","last_fullsized":"2024-03-14T12:16:58.141079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.138951+0000","last_change":"2024-03-14T12:16:58.138951+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.138951+0000","last_undegraded":"2024-03-14T12:16:58.138951+0000","last_fullsized":"2024-03-14T12:16:58.138951+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137854+0000","last_change":"2024-03-14T12:16:58.137854+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137854+0000","last_undegraded":"2024-03-14T12:16:58.137854+0000","last_fullsized":"2024-03-14T12:16:58.137854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137352+0000","last_change":"2024-03-14T12:16:58.137352+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137352+0000","last_undegraded":"2024-03-14T12:16:58.137352+0000","last_fullsized":"2024-03-14T12:16:58.137352+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141664+0000","last_change":"2024-03-14T12:16:58.141664+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141664+0000","last_undegraded":"2024-03-14T12:16:58.141664+0000","last_fullsized":"2024-03-14T12:16:58.141664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136603+0000","last_change":"2024-03-14T12:16:58.136603+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136603+0000","last_undegraded":"2024-03-14T12:16:58.136603+0000","last_fullsized":"2024-03-14T12:16:58.136603+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136093+0000","last_change":"2024-03-14T12:16:58.136093+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136093+0000","last_undegraded":"2024-03-14T12:16:58.136093+0000","last_fullsized":"2024-03-14T12:16:58.136093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.142113+0000","last_change":"2024-03-14T12:16:58.142113+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.142113+0000","last_undegraded":"2024-03-14T12:16:58.142113+0000","last_fullsized":"2024-03-14T12:16:58.142113+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136820+0000","last_change":"2024-03-14T12:16:58.136820+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136820+0000","last_undegraded":"2024-03-14T12:16:58.136820+0000","last_fullsized":"2024-03-14T12:16:58.136820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141263+0000","last_change":"2024-03-14T12:16:58.141263+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141263+0000","last_undegraded":"2024-03-14T12:16:58.141263+0000","last_fullsized":"2024-03-14T12:16:58.141263+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.139319+0000","last_change":"2024-03-14T12:16:58.139319+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.139319+0000","last_undegraded":"2024-03-14T12:16:58.139319+0000","last_fullsized":"2024-03-14T12:16:58.139319+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141139+0000","last_change":"2024-03-14T12:16:58.141139+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141139+0000","last_undegraded":"2024-03-14T12:16:58.141139+0000","last_fullsized":"2024-03-14T12:16:58.141139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136100+0000","last_change":"2024-03-14T12:16:58.136100+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136100+0000","last_undegraded":"2024-03-14T12:16:58.136100+0000","last_fullsized":"2024-03-14T12:16:58.136100+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.134049+0000","last_change":"2024-03-14T12:16:58.134049+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.134049+0000","last_undegraded":"2024-03-14T12:16:58.134049+0000","last_fullsized":"2024-03-14T12:16:58.134049+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141077+0000","last_change":"2024-03-14T12:16:58.141077+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141077+0000","last_undegraded":"2024-03-14T12:16:58.141077+0000","last_fullsized":"2024-03-14T12:16:58.141077+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136290+0000","last_change":"2024-03-14T12:16:58.136290+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136290+0000","last_undegraded":"2024-03-14T12:16:58.136290+0000","last_fullsized":"2024-03-14T12:16:58.136290+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.135596+0000","last_change":"2024-03-14T12:16:58.135596+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.135596+0000","last_undegraded":"2024-03-14T12:16:58.135596+0000","last_fullsized":"2024-03-14T12:16:58.135596+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.133661+0000","last_change":"2024-03-14T12:16:58.133661+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.133661+0000","last_undegraded":"2024-03-14T12:16:58.133661+0000","last_fullsized":"2024-03-14T12:16:58.133661+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.135546+0000","last_change":"2024-03-14T12:16:58.135546+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.135546+0000","last_undegraded":"2024-03-14T12:16:58.135546+0000","last_fullsized":"2024-03-14T12:16:58.135546+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137350+0000","last_change":"2024-03-14T12:16:58.137350+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137350+0000","last_undegraded":"2024-03-14T12:16:58.137350+0000","last_fullsized":"2024-03-14T12:16:58.137350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136093+0000","last_change":"2024-03-14T12:16:58.136093+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136093+0000","last_undegraded":"2024-03-14T12:16:58.136093+0000","last_fullsized":"2024-03-14T12:16:58.136093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141052+0000","last_change":"2024-03-14T12:16:58.141052+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141052+0000","last_undegraded":"2024-03-14T12:16:58.141052+0000","last_fullsized":"2024-03-14T12:16:58.141052+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141063+0000","last_change":"2024-03-14T12:16:58.141063+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141063+0000","last_undegraded":"2024-03-14T12:16:58.141063+0000","last_fullsized":"2024-03-14T12:16:58.141063+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":253,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-03-14T12:16:58.139802+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:16:58.139802+0000","last_peered":"2024-03-14T12:16:58.139802+0000","last_clean":"2024-03-14T12:16:58.139802+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:16:58.139802+0000","last_undegraded":"2024-03-14T12:16:58.139802+0000","last_fullsized":"2024-03-14T12:16:58.139802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1396736,"data_stored":922028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":331660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":922028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:17:03.116 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:17:03.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.253+0000 7f0fcbc20640 1 Processor -- start 2024-03-14T12:17:03.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.253+0000 7f0fcbc20640 1 -- start start 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fcbc20640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0fc40d0760 0x7f0fc40d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fcbc20640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40cf370 0x7f0fc40d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fcbc20640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0fc40d1da0 0x7f0fc40d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fcbc20640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc40d1080 con 0x7f0fc40d1da0 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fcbc20640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc40cf8c0 con 0x7f0fc40d0760 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fcbc20640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40cfba0 con 0x7f0fc40cf370 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fca1bd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0fc40d1da0 0x7f0fc40d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fca1bd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0fc40d1da0 0x7f0fc40d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46262/0 (socket says 172.21.15.82:46262) 2024-03-14T12:17:03.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fca1bd640 1 -- 172.21.15.82:0/3193154601 learned_addr learned my addr 172.21.15.82:0/3193154601 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.254+0000 7f0fc91bb640 1 --2- 172.21.15.82:0/3193154601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40cf370 0x7f0fc40d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fc99bc640 1 --2- 172.21.15.82:0/3193154601 >> v2:172.21.15.164:3300/0 conn(0x7f0fc40d0760 0x7f0fc40d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fca1bd640 1 -- 172.21.15.82:0/3193154601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40cf370 msgr2=0x7f0fc40d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fca1bd640 1 --2- 172.21.15.82:0/3193154601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40cf370 0x7f0fc40d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fca1bd640 1 -- 172.21.15.82:0/3193154601 >> v2:172.21.15.164:3300/0 conn(0x7f0fc40d0760 msgr2=0x7f0fc40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fca1bd640 1 --2- 172.21.15.82:0/3193154601 >> v2:172.21.15.164:3300/0 conn(0x7f0fc40d0760 0x7f0fc40d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fca1bd640 1 -- 172.21.15.82:0/3193154601 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40b6c40 con 0x7f0fc40d1da0 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fc91bb640 1 --2- 172.21.15.82:0/3193154601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40cf370 0x7f0fc40d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.255+0000 7f0fca1bd640 1 --2- 172.21.15.82:0/3193154601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40d1da0 0x7f0fc40d2210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0fc408ca90 tx=0x7f0fbc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3100ece5a5d47 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fc89ba640 1 -- 172.21.15.82:0/3193154601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc034070 con 0x7f0fc40d1da0 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fc89ba640 1 -- 172.21.15.82:0/3193154601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc030860 con 0x7f0fc40d1da0 2024-03-14T12:17:03.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fc89ba640 1 -- 172.21.15.82:0/3193154601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc030c30 con 0x7f0fc40d1da0 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3193154601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40d1da0 msgr2=0x7f0fc40d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3193154601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40d1da0 0x7f0fc40d2210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0fc408ca90 tx=0x7f0fbc02b9a0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3193154601 shutdown_connections 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3193154601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40d1da0 0x7f0fc40d2210 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3193154601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40cf370 0x7f0fc40d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3193154601 >> v2:172.21.15.164:3300/0 conn(0x7f0fc40d0760 0x7f0fc40d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3193154601 >> 172.21.15.82:0/3193154601 conn(0x7f0fc409d030 msgr2=0x7f0fc40d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3193154601 shutdown_connections 2024-03-14T12:17:03.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.256+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3193154601 wait complete. 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.257+0000 7f0fcbc20640 1 Processor -- start 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.257+0000 7f0fcbc20640 1 -- start start 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fcbc20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40cf370 0x7f0fc40c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fcbc20640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40d0760 0x7f0fc40cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fcbc20640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fc40d1da0 0x7f0fc40cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fcbc20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40d4ba0 con 0x7f0fc40cf370 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fcbc20640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40b7200 con 0x7f0fc40d1da0 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fcbc20640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc41617a0 con 0x7f0fc40d0760 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40cf370 0x7f0fc40c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40cf370 0x7f0fc40c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46272/0 (socket says 172.21.15.82:46272) 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 -- 172.21.15.82:0/3807655878 learned_addr learned my addr 172.21.15.82:0/3807655878 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc91bb640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40d0760 0x7f0fc40cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 -- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40d0760 msgr2=0x7f0fc40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fca1bd640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fc40d1da0 0x7f0fc40cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40d0760 0x7f0fc40cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 -- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fc40d1da0 msgr2=0x7f0fc40cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fc40d1da0 0x7f0fc40cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc99bc640 1 -- 172.21.15.82:0/3807655878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40cbd50 con 0x7f0fc40cf370 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.258+0000 7f0fc91bb640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40d0760 0x7f0fc40cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.259+0000 7f0fc99bc640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40cf370 0x7f0fc40c9380 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000efc0 tx=0x7f0fc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.259+0000 7f0fba7fc640 1 -- 172.21.15.82:0/3807655878 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0018020 con 0x7f0fc40cf370 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.259+0000 7f0fba7fc640 1 -- 172.21.15.82:0/3807655878 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0014480 con 0x7f0fc40cf370 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.259+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc40cc030 con 0x7f0fc40cf370 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.259+0000 7f0fba7fc640 1 -- 172.21.15.82:0/3807655878 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0014850 con 0x7f0fc40cf370 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.259+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc40cc5f0 con 0x7f0fc40cf370 2024-03-14T12:17:03.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.260+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc404f050 con 0x7f0fc40cf370 2024-03-14T12:17:03.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.260+0000 7f0fba7fc640 1 -- 172.21.15.82:0/3807655878 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0fc0005d10 con 0x7f0fc40cf370 2024-03-14T12:17:03.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.261+0000 7f0fba7fc640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0fa806bbd0 0x7f0fa806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.261+0000 7f0fc91bb640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0fa806bbd0 0x7f0fa806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.261+0000 7f0fba7fc640 1 -- 172.21.15.82:0/3807655878 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fc008b570 con 0x7f0fc40cf370 2024-03-14T12:17:03.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.261+0000 7f0fc91bb640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0fa806bbd0 0x7f0fa806e090 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006fd0 tx=0x7f0fb4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.265+0000 7f0fba7fc640 1 -- 172.21.15.82:0/3807655878 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0fc0022070 con 0x7f0fc40cf370 2024-03-14T12:17:03.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.433+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0fc4073030 con 0x7f0fa806bbd0 2024-03-14T12:17:03.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.438+0000 7f0fba7fc640 1 -- 172.21.15.82:0/3807655878 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240144 (secure 0 0 0) 0x7f0fc4073030 con 0x7f0fa806bbd0 2024-03-14T12:17:03.440 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:03.441 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0fa806bbd0 msgr2=0x7f0fa806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0fa806bbd0 0x7f0fa806e090 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006fd0 tx=0x7f0fb4008040 comp rx=0 tx=0).stop 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40cf370 msgr2=0x7f0fc40c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40cf370 0x7f0fc40c9380 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000efc0 tx=0x7f0fc000c490 comp rx=0 tx=0).stop 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 shutdown_connections 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fc40d1da0 0x7f0fc40cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f0fa806bbd0 0x7f0fa806e090 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0fc40d0760 0x7f0fc40cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 --2- 172.21.15.82:0/3807655878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc40cf370 0x7f0fc40c9380 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 >> 172.21.15.82:0/3807655878 conn(0x7f0fc409d030 msgr2=0x7f0fc40d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 shutdown_connections 2024-03-14T12:17:03.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.441+0000 7f0fcbc20640 1 -- 172.21.15.82:0/3807655878 wait complete. 2024-03-14T12:17:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-03-14T12:17:01.536626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261028,"kb_used_data":7372,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713692,"statfs":{"total":773094113280,"available":772826820608,"internally_reserved":0,"allocated":7548928,"data_stored":3804931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":259575347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000601"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.139342+0000","last_change":"2024-03-14T12:16:58.139342+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.139342+0000","last_undegraded":"2024-03-14T12:16:58.139342+0000","last_fullsized":"2024-03-14T12:16:58.139342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.134124+0000","last_change":"2024-03-14T12:16:58.134124+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.134124+0000","last_undegraded":"2024-03-14T12:16:58.134124+0000","last_fullsized":"2024-03-14T12:16:58.134124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136759+0000","last_change":"2024-03-14T12:16:58.136759+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136759+0000","last_undegraded":"2024-03-14T12:16:58.136759+0000","last_fullsized":"2024-03-14T12:16:58.136759+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141131+0000","last_change":"2024-03-14T12:16:58.141131+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141131+0000","last_undegraded":"2024-03-14T12:16:58.141131+0000","last_fullsized":"2024-03-14T12:16:58.141131+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.138972+0000","last_change":"2024-03-14T12:16:58.138972+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.138972+0000","last_undegraded":"2024-03-14T12:16:58.138972+0000","last_fullsized":"2024-03-14T12:16:58.138972+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.142257+0000","last_change":"2024-03-14T12:16:58.142257+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.142257+0000","last_undegraded":"2024-03-14T12:16:58.142257+0000","last_fullsized":"2024-03-14T12:16:58.142257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137339+0000","last_change":"2024-03-14T12:16:58.137339+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137339+0000","last_undegraded":"2024-03-14T12:16:58.137339+0000","last_fullsized":"2024-03-14T12:16:58.137339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.135605+0000","last_change":"2024-03-14T12:16:58.135605+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.135605+0000","last_undegraded":"2024-03-14T12:16:58.135605+0000","last_fullsized":"2024-03-14T12:16:58.135605+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136433+0000","last_change":"2024-03-14T12:16:58.136433+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136433+0000","last_undegraded":"2024-03-14T12:16:58.136433+0000","last_fullsized":"2024-03-14T12:16:58.136433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141079+0000","last_change":"2024-03-14T12:16:58.141079+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141079+0000","last_undegraded":"2024-03-14T12:16:58.141079+0000","last_fullsized":"2024-03-14T12:16:58.141079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.138951+0000","last_change":"2024-03-14T12:16:58.138951+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.138951+0000","last_undegraded":"2024-03-14T12:16:58.138951+0000","last_fullsized":"2024-03-14T12:16:58.138951+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137854+0000","last_change":"2024-03-14T12:16:58.137854+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137854+0000","last_undegraded":"2024-03-14T12:16:58.137854+0000","last_fullsized":"2024-03-14T12:16:58.137854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137352+0000","last_change":"2024-03-14T12:16:58.137352+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137352+0000","last_undegraded":"2024-03-14T12:16:58.137352+0000","last_fullsized":"2024-03-14T12:16:58.137352+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141664+0000","last_change":"2024-03-14T12:16:58.141664+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141664+0000","last_undegraded":"2024-03-14T12:16:58.141664+0000","last_fullsized":"2024-03-14T12:16:58.141664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136603+0000","last_change":"2024-03-14T12:16:58.136603+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136603+0000","last_undegraded":"2024-03-14T12:16:58.136603+0000","last_fullsized":"2024-03-14T12:16:58.136603+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136093+0000","last_change":"2024-03-14T12:16:58.136093+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136093+0000","last_undegraded":"2024-03-14T12:16:58.136093+0000","last_fullsized":"2024-03-14T12:16:58.136093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.142113+0000","last_change":"2024-03-14T12:16:58.142113+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.142113+0000","last_undegraded":"2024-03-14T12:16:58.142113+0000","last_fullsized":"2024-03-14T12:16:58.142113+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136820+0000","last_change":"2024-03-14T12:16:58.136820+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136820+0000","last_undegraded":"2024-03-14T12:16:58.136820+0000","last_fullsized":"2024-03-14T12:16:58.136820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141263+0000","last_change":"2024-03-14T12:16:58.141263+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141263+0000","last_undegraded":"2024-03-14T12:16:58.141263+0000","last_fullsized":"2024-03-14T12:16:58.141263+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.139319+0000","last_change":"2024-03-14T12:16:58.139319+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.139319+0000","last_undegraded":"2024-03-14T12:16:58.139319+0000","last_fullsized":"2024-03-14T12:16:58.139319+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141139+0000","last_change":"2024-03-14T12:16:58.141139+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141139+0000","last_undegraded":"2024-03-14T12:16:58.141139+0000","last_fullsized":"2024-03-14T12:16:58.141139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136100+0000","last_change":"2024-03-14T12:16:58.136100+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136100+0000","last_undegraded":"2024-03-14T12:16:58.136100+0000","last_fullsized":"2024-03-14T12:16:58.136100+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.134049+0000","last_change":"2024-03-14T12:16:58.134049+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.134049+0000","last_undegraded":"2024-03-14T12:16:58.134049+0000","last_fullsized":"2024-03-14T12:16:58.134049+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141077+0000","last_change":"2024-03-14T12:16:58.141077+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141077+0000","last_undegraded":"2024-03-14T12:16:58.141077+0000","last_fullsized":"2024-03-14T12:16:58.141077+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136290+0000","last_change":"2024-03-14T12:16:58.136290+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136290+0000","last_undegraded":"2024-03-14T12:16:58.136290+0000","last_fullsized":"2024-03-14T12:16:58.136290+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.135596+0000","last_change":"2024-03-14T12:16:58.135596+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.135596+0000","last_undegraded":"2024-03-14T12:16:58.135596+0000","last_fullsized":"2024-03-14T12:16:58.135596+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.133661+0000","last_change":"2024-03-14T12:16:58.133661+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.133661+0000","last_undegraded":"2024-03-14T12:16:58.133661+0000","last_fullsized":"2024-03-14T12:16:58.133661+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.135546+0000","last_change":"2024-03-14T12:16:58.135546+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.135546+0000","last_undegraded":"2024-03-14T12:16:58.135546+0000","last_fullsized":"2024-03-14T12:16:58.135546+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.137350+0000","last_change":"2024-03-14T12:16:58.137350+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.137350+0000","last_undegraded":"2024-03-14T12:16:58.137350+0000","last_fullsized":"2024-03-14T12:16:58.137350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.136093+0000","last_change":"2024-03-14T12:16:58.136093+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.136093+0000","last_undegraded":"2024-03-14T12:16:58.136093+0000","last_fullsized":"2024-03-14T12:16:58.136093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141052+0000","last_change":"2024-03-14T12:16:58.141052+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141052+0000","last_undegraded":"2024-03-14T12:16:58.141052+0000","last_fullsized":"2024-03-14T12:16:58.141052+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T12:16:59.129472+0000","last_change":"2024-03-14T12:16:59.129472+0000","last_active":"2024-03-14T12:16:59.129472+0000","last_peered":"2024-03-14T12:16:59.129472+0000","last_clean":"2024-03-14T12:16:59.129472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:59.129472+0000","last_undegraded":"2024-03-14T12:16:59.129472+0000","last_fullsized":"2024-03-14T12:16:59.129472+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-03-14T12:16:58.141063+0000","last_change":"2024-03-14T12:16:58.141063+0000","last_active":"2024-03-14T12:16:58.124166+0000","last_peered":"2024-03-14T12:16:58.124166+0000","last_clean":"2024-03-14T12:16:58.124166+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T12:16:58.141063+0000","last_undegraded":"2024-03-14T12:16:58.141063+0000","last_fullsized":"2024-03-14T12:16:58.141063+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":253,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-03-14T12:16:58.139802+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:16:58.139802+0000","last_peered":"2024-03-14T12:16:58.139802+0000","last_clean":"2024-03-14T12:16:58.139802+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:16:58.139802+0000","last_undegraded":"2024-03-14T12:16:58.139802+0000","last_fullsized":"2024-03-14T12:16:58.139802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1396736,"data_stored":922028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":331660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":790528,"data_stored":325843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":922028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:17:03.455 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:17:03.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.592+0000 7f8b3f2ab640 1 Processor -- start 2024-03-14T12:17:03.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.593+0000 7f8b3f2ab640 1 -- start start 2024-03-14T12:17:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.593+0000 7f8b3f2ab640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8b380cf370 0x7f8b380d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.593+0000 7f8b3f2ab640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b380cf8c0 con 0x7f8b380d0760 2024-03-14T12:17:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.593+0000 7f8b3f2ab640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b380cfba0 con 0x7f8b380b7720 2024-03-14T12:17:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.593+0000 7f8b3f2ab640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b380b6c40 con 0x7f8b380cf370 2024-03-14T12:17:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b3d047640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8b380d0760 0x7f8b380d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44470/0 (socket says 172.21.15.82:44470) 2024-03-14T12:17:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b3d047640 1 -- 172.21.15.82:0/249416804 learned_addr learned my addr 172.21.15.82:0/249416804 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592010179 0 0) 0x7f8b380cf8c0 con 0x7f8b380d0760 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b3c846640 1 --2- 172.21.15.82:0/249416804 >> v2:172.21.15.164:3300/0 conn(0x7f8b380cf370 0x7f8b380d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b24003520 con 0x7f8b380d0760 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107322260 0 0) 0x7f8b24003520 con 0x7f8b380d0760 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 >> v2:172.21.15.164:3300/0 conn(0x7f8b380cf370 msgr2=0x7f8b380d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 --2- 172.21.15.82:0/249416804 >> v2:172.21.15.164:3300/0 conn(0x7f8b380cf370 0x7f8b380d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 >> v1:172.21.15.164:6789/0 conn(0x7f8b380b7720 legacy=0x7f8b380b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380b6f20 con 0x7f8b380d0760 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f8b28001670 con 0x7f8b380d0760 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 >> v1:172.21.15.82:6789/0 conn(0x7f8b380d0760 legacy=0x7f8b380d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b240039e0 0x7f8b24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b24004800 0x7f8b24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24005210 0x7f8b2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380b6c40 con 0x7f8b24004800 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2400c120 con 0x7f8b240039e0 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2400c320 con 0x7f8b24005210 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b3c846640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b24004800 0x7f8b24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b3d047640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b240039e0 0x7f8b24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.594+0000 7f8b3d848640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24005210 0x7f8b2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3c846640 1 -- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24005210 msgr2=0x7f8b2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3c846640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24005210 0x7f8b2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3c846640 1 -- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b240039e0 msgr2=0x7f8b24003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3c846640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b240039e0 0x7f8b24003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3c846640 1 -- 172.21.15.82:0/249416804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24003520 con 0x7f8b24004800 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3d848640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24005210 0x7f8b2400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3d848640 1 -- 172.21.15.82:0/249416804 reap_dead start 2024-03-14T12:17:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b3c846640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b24004800 0x7f8b24004c80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ba30 tx=0x7f8b24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185b529c45ac5160 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34036020 con 0x7f8b24004800 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34032640 con 0x7f8b24004800 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.595+0000 7f8b2ffff640 1 -- 172.21.15.82:0/249416804 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400b600 con 0x7f8b24004800 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.596+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/249416804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b24004800 msgr2=0x7f8b24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.596+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b24004800 0x7f8b24004c80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ba30 tx=0x7f8b24005830 comp rx=0 tx=0).stop 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.596+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/249416804 shutdown_connections 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.596+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/249416804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b24004800 0x7f8b24004c80 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.596+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/249416804 >> 172.21.15.82:0/249416804 conn(0x7f8b3809d030 msgr2=0x7f8b380d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.596+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/249416804 shutdown_connections 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.596+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/249416804 wait complete. 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.597+0000 7f8b3f2ab640 1 Processor -- start 2024-03-14T12:17:03.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.597+0000 7f8b3f2ab640 1 -- start start 2024-03-14T12:17:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3f2ab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24004800 0x7f8b380c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3f2ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b380cae20 0x7f8b380cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3f2ab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b380cb2a0 0x7f8b380cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3f2ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38161920 con 0x7f8b380cae20 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3f2ab640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b381617a0 con 0x7f8b380cb2a0 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3f2ab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380d0b40 con 0x7f8b24004800 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b380cae20 0x7f8b380cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b380cae20 0x7f8b380cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46292/0 (socket says 172.21.15.82:46292) 2024-03-14T12:17:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 -- 172.21.15.82:0/1789193933 learned_addr learned my addr 172.21.15.82:0/1789193933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3d047640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24004800 0x7f8b380c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3d848640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b380cb2a0 0x7f8b380cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 -- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24004800 msgr2=0x7f8b380c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24004800 0x7f8b380c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 -- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b380cb2a0 msgr2=0x7f8b380cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b380cb2a0 0x7f8b380cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 -- 172.21.15.82:0/1789193933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380cc830 con 0x7f8b380cae20 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3d047640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24004800 0x7f8b380c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3c846640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b380cae20 0x7f8b380cd1a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400bb50 tx=0x7f8b34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b2dffb640 1 -- 172.21.15.82:0/1789193933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34041070 con 0x7f8b380cae20 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b2dffb640 1 -- 172.21.15.82:0/1789193933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34030a50 con 0x7f8b380cae20 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b380cb790 con 0x7f8b380cae20 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.598+0000 7f8b2dffb640 1 -- 172.21.15.82:0/1789193933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34036040 con 0x7f8b380cae20 2024-03-14T12:17:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.599+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b380cbd50 con 0x7f8b380cae20 2024-03-14T12:17:03.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.600+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3804f050 con 0x7f8b380cae20 2024-03-14T12:17:03.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.600+0000 7f8b2dffb640 1 -- 172.21.15.82:0/1789193933 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8b3404c060 con 0x7f8b380cae20 2024-03-14T12:17:03.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.600+0000 7f8b2dffb640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8b1c06bbd0 0x7f8b1c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.601+0000 7f8b2dffb640 1 -- 172.21.15.82:0/1789193933 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b340ac900 con 0x7f8b380cae20 2024-03-14T12:17:03.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.601+0000 7f8b3d047640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8b1c06bbd0 0x7f8b1c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.601+0000 7f8b3d047640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8b1c06bbd0 0x7f8b1c06e090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b380d49f0 tx=0x7f8b28009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.605+0000 7f8b2dffb640 1 -- 172.21.15.82:0/1789193933 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8b3407b360 con 0x7f8b380cae20 2024-03-14T12:17:03.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.772+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b380d0540 con 0x7f8b1c06bbd0 2024-03-14T12:17:03.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.778+0000 7f8b2dffb640 1 -- 172.21.15.82:0/1789193933 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252069 (secure 0 0 0) 0x7f8b380d0540 con 0x7f8b1c06bbd0 2024-03-14T12:17:03.780 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:03.781 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.781+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8b1c06bbd0 msgr2=0x7f8b1c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.781+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8b1c06bbd0 0x7f8b1c06e090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b380d49f0 tx=0x7f8b28009040 comp rx=0 tx=0).stop 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b380cae20 msgr2=0x7f8b380cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b380cae20 0x7f8b380cd1a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400bb50 tx=0x7f8b34004730 comp rx=0 tx=0).stop 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 shutdown_connections 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b380cb2a0 0x7f8b380cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b380cae20 0x7f8b380cd1a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8b1c06bbd0 0x7f8b1c06e090 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 --2- 172.21.15.82:0/1789193933 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b24004800 0x7f8b380c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 >> 172.21.15.82:0/1789193933 conn(0x7f8b3809d030 msgr2=0x7f8b380d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:03.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 shutdown_connections 2024-03-14T12:17:03.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.782+0000 7f8b3f2ab640 1 -- 172.21.15.82:0/1789193933 wait complete. 2024-03-14T12:17:03.792 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-03-14T12:17:03.537053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261412,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713308,"statfs":{"total":773094113280,"available":772826427392,"internally_reserved":0,"allocated":7970816,"data_stored":3993636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":259575347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.407008"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197760+0000","last_change":"2024-03-14T12:17:00.165695+0000","last_active":"2024-03-14T12:17:02.197760+0000","last_peered":"2024-03-14T12:17:02.197760+0000","last_clean":"2024-03-14T12:17:02.197760+0000","last_became_active":"2024-03-14T12:17:00.165201+0000","last_became_peered":"2024-03-14T12:17:00.165201+0000","last_unstale":"2024-03-14T12:17:02.197760+0000","last_undegraded":"2024-03-14T12:17:02.197760+0000","last_fullsized":"2024-03-14T12:17:02.197760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:23:16.151346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198669+0000","last_change":"2024-03-14T12:17:00.158662+0000","last_active":"2024-03-14T12:17:02.198669+0000","last_peered":"2024-03-14T12:17:02.198669+0000","last_clean":"2024-03-14T12:17:02.198669+0000","last_became_active":"2024-03-14T12:17:00.158337+0000","last_became_peered":"2024-03-14T12:17:00.158337+0000","last_unstale":"2024-03-14T12:17:02.198669+0000","last_undegraded":"2024-03-14T12:17:02.198669+0000","last_fullsized":"2024-03-14T12:17:02.198669+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:54.810418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191019+0000","last_change":"2024-03-14T12:17:00.155441+0000","last_active":"2024-03-14T12:17:02.191019+0000","last_peered":"2024-03-14T12:17:02.191019+0000","last_clean":"2024-03-14T12:17:02.191019+0000","last_became_active":"2024-03-14T12:17:00.155153+0000","last_became_peered":"2024-03-14T12:17:00.155153+0000","last_unstale":"2024-03-14T12:17:02.191019+0000","last_undegraded":"2024-03-14T12:17:02.191019+0000","last_fullsized":"2024-03-14T12:17:02.191019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:46.598883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196932+0000","last_change":"2024-03-14T12:17:00.158103+0000","last_active":"2024-03-14T12:17:02.196932+0000","last_peered":"2024-03-14T12:17:02.196932+0000","last_clean":"2024-03-14T12:17:02.196932+0000","last_became_active":"2024-03-14T12:17:00.157827+0000","last_became_peered":"2024-03-14T12:17:00.157827+0000","last_unstale":"2024-03-14T12:17:02.196932+0000","last_undegraded":"2024-03-14T12:17:02.196932+0000","last_fullsized":"2024-03-14T12:17:02.196932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:48.140138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144471+0000","last_change":"2024-03-14T12:17:00.158865+0000","last_active":"2024-03-14T12:17:01.144471+0000","last_peered":"2024-03-14T12:17:01.144471+0000","last_clean":"2024-03-14T12:17:01.144471+0000","last_became_active":"2024-03-14T12:17:00.158562+0000","last_became_peered":"2024-03-14T12:17:00.158562+0000","last_unstale":"2024-03-14T12:17:01.144471+0000","last_undegraded":"2024-03-14T12:17:01.144471+0000","last_fullsized":"2024-03-14T12:17:01.144471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:51.747175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198057+0000","last_change":"2024-03-14T12:17:00.150382+0000","last_active":"2024-03-14T12:17:02.198057+0000","last_peered":"2024-03-14T12:17:02.198057+0000","last_clean":"2024-03-14T12:17:02.198057+0000","last_became_active":"2024-03-14T12:17:00.150078+0000","last_became_peered":"2024-03-14T12:17:00.150078+0000","last_unstale":"2024-03-14T12:17:02.198057+0000","last_undegraded":"2024-03-14T12:17:02.198057+0000","last_fullsized":"2024-03-14T12:17:02.198057+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:19.263347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145699+0000","last_change":"2024-03-14T12:17:00.163727+0000","last_active":"2024-03-14T12:17:01.145699+0000","last_peered":"2024-03-14T12:17:01.145699+0000","last_clean":"2024-03-14T12:17:01.145699+0000","last_became_active":"2024-03-14T12:17:00.163450+0000","last_became_peered":"2024-03-14T12:17:00.163450+0000","last_unstale":"2024-03-14T12:17:01.145699+0000","last_undegraded":"2024-03-14T12:17:01.145699+0000","last_fullsized":"2024-03-14T12:17:01.145699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:00:17.473435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198050+0000","last_change":"2024-03-14T12:17:00.157697+0000","last_active":"2024-03-14T12:17:02.198050+0000","last_peered":"2024-03-14T12:17:02.198050+0000","last_clean":"2024-03-14T12:17:02.198050+0000","last_became_active":"2024-03-14T12:17:00.157426+0000","last_became_peered":"2024-03-14T12:17:00.157426+0000","last_unstale":"2024-03-14T12:17:02.198050+0000","last_undegraded":"2024-03-14T12:17:02.198050+0000","last_fullsized":"2024-03-14T12:17:02.198050+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:03:04.017798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144864+0000","last_change":"2024-03-14T12:17:00.155164+0000","last_active":"2024-03-14T12:17:01.144864+0000","last_peered":"2024-03-14T12:17:01.144864+0000","last_clean":"2024-03-14T12:17:01.144864+0000","last_became_active":"2024-03-14T12:17:00.154702+0000","last_became_peered":"2024-03-14T12:17:00.154702+0000","last_unstale":"2024-03-14T12:17:01.144864+0000","last_undegraded":"2024-03-14T12:17:01.144864+0000","last_fullsized":"2024-03-14T12:17:01.144864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:50.379295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146290+0000","last_change":"2024-03-14T12:17:00.156268+0000","last_active":"2024-03-14T12:17:01.146290+0000","last_peered":"2024-03-14T12:17:01.146290+0000","last_clean":"2024-03-14T12:17:01.146290+0000","last_became_active":"2024-03-14T12:17:00.149471+0000","last_became_peered":"2024-03-14T12:17:00.149471+0000","last_unstale":"2024-03-14T12:17:01.146290+0000","last_undegraded":"2024-03-14T12:17:01.146290+0000","last_fullsized":"2024-03-14T12:17:01.146290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:39.201880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197442+0000","last_change":"2024-03-14T12:17:00.160595+0000","last_active":"2024-03-14T12:17:02.197442+0000","last_peered":"2024-03-14T12:17:02.197442+0000","last_clean":"2024-03-14T12:17:02.197442+0000","last_became_active":"2024-03-14T12:17:00.160248+0000","last_became_peered":"2024-03-14T12:17:00.160248+0000","last_unstale":"2024-03-14T12:17:02.197442+0000","last_undegraded":"2024-03-14T12:17:02.197442+0000","last_fullsized":"2024-03-14T12:17:02.197442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196931+0000","last_change":"2024-03-14T12:17:00.161414+0000","last_active":"2024-03-14T12:17:02.196931+0000","last_peered":"2024-03-14T12:17:02.196931+0000","last_clean":"2024-03-14T12:17:02.196931+0000","last_became_active":"2024-03-14T12:17:00.161137+0000","last_became_peered":"2024-03-14T12:17:00.161137+0000","last_unstale":"2024-03-14T12:17:02.196931+0000","last_undegraded":"2024-03-14T12:17:02.196931+0000","last_fullsized":"2024-03-14T12:17:02.196931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:37.868936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159671+0000","last_change":"2024-03-14T12:17:00.157788+0000","last_active":"2024-03-14T12:17:02.159671+0000","last_peered":"2024-03-14T12:17:02.159671+0000","last_clean":"2024-03-14T12:17:02.159671+0000","last_became_active":"2024-03-14T12:17:00.157422+0000","last_became_peered":"2024-03-14T12:17:00.157422+0000","last_unstale":"2024-03-14T12:17:02.159671+0000","last_undegraded":"2024-03-14T12:17:02.159671+0000","last_fullsized":"2024-03-14T12:17:02.159671+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144432+0000","last_change":"2024-03-14T12:17:00.158475+0000","last_active":"2024-03-14T12:17:01.144432+0000","last_peered":"2024-03-14T12:17:01.144432+0000","last_clean":"2024-03-14T12:17:01.144432+0000","last_became_active":"2024-03-14T12:17:00.157438+0000","last_became_peered":"2024-03-14T12:17:00.157438+0000","last_unstale":"2024-03-14T12:17:01.144432+0000","last_undegraded":"2024-03-14T12:17:01.144432+0000","last_fullsized":"2024-03-14T12:17:01.144432+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:35.836879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159452+0000","last_change":"2024-03-14T12:17:00.160877+0000","last_active":"2024-03-14T12:17:02.159452+0000","last_peered":"2024-03-14T12:17:02.159452+0000","last_clean":"2024-03-14T12:17:02.159452+0000","last_became_active":"2024-03-14T12:17:00.160612+0000","last_became_peered":"2024-03-14T12:17:00.160612+0000","last_unstale":"2024-03-14T12:17:02.159452+0000","last_undegraded":"2024-03-14T12:17:02.159452+0000","last_fullsized":"2024-03-14T12:17:02.159452+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147120+0000","last_change":"2024-03-14T12:17:00.156510+0000","last_active":"2024-03-14T12:17:01.147120+0000","last_peered":"2024-03-14T12:17:01.147120+0000","last_clean":"2024-03-14T12:17:01.147120+0000","last_became_active":"2024-03-14T12:17:00.155713+0000","last_became_peered":"2024-03-14T12:17:00.155713+0000","last_unstale":"2024-03-14T12:17:01.147120+0000","last_undegraded":"2024-03-14T12:17:01.147120+0000","last_fullsized":"2024-03-14T12:17:01.147120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:54.810418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144864+0000","last_change":"2024-03-14T12:17:00.154954+0000","last_active":"2024-03-14T12:17:01.144864+0000","last_peered":"2024-03-14T12:17:01.144864+0000","last_clean":"2024-03-14T12:17:01.144864+0000","last_became_active":"2024-03-14T12:17:00.152674+0000","last_became_peered":"2024-03-14T12:17:00.152674+0000","last_unstale":"2024-03-14T12:17:01.144864+0000","last_undegraded":"2024-03-14T12:17:01.144864+0000","last_fullsized":"2024-03-14T12:17:01.144864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:24.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196995+0000","last_change":"2024-03-14T12:17:00.163547+0000","last_active":"2024-03-14T12:17:02.196995+0000","last_peered":"2024-03-14T12:17:02.196995+0000","last_clean":"2024-03-14T12:17:02.196995+0000","last_became_active":"2024-03-14T12:17:00.163294+0000","last_became_peered":"2024-03-14T12:17:00.163294+0000","last_unstale":"2024-03-14T12:17:02.196995+0000","last_undegraded":"2024-03-14T12:17:02.196995+0000","last_fullsized":"2024-03-14T12:17:02.196995+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:48.571300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198420+0000","last_change":"2024-03-14T12:17:00.158303+0000","last_active":"2024-03-14T12:17:02.198420+0000","last_peered":"2024-03-14T12:17:02.198420+0000","last_clean":"2024-03-14T12:17:02.198420+0000","last_became_active":"2024-03-14T12:17:00.158026+0000","last_became_peered":"2024-03-14T12:17:00.158026+0000","last_unstale":"2024-03-14T12:17:02.198420+0000","last_undegraded":"2024-03-14T12:17:02.198420+0000","last_fullsized":"2024-03-14T12:17:02.198420+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:07.658231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159693+0000","last_change":"2024-03-14T12:17:00.163989+0000","last_active":"2024-03-14T12:17:02.159693+0000","last_peered":"2024-03-14T12:17:02.159693+0000","last_clean":"2024-03-14T12:17:02.159693+0000","last_became_active":"2024-03-14T12:17:00.163708+0000","last_became_peered":"2024-03-14T12:17:00.163708+0000","last_unstale":"2024-03-14T12:17:02.159693+0000","last_undegraded":"2024-03-14T12:17:02.159693+0000","last_fullsized":"2024-03-14T12:17:02.159693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:03:50.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146748+0000","last_change":"2024-03-14T12:17:00.154763+0000","last_active":"2024-03-14T12:17:01.146748+0000","last_peered":"2024-03-14T12:17:01.146748+0000","last_clean":"2024-03-14T12:17:01.146748+0000","last_became_active":"2024-03-14T12:17:00.154477+0000","last_became_peered":"2024-03-14T12:17:00.154477+0000","last_unstale":"2024-03-14T12:17:01.146748+0000","last_undegraded":"2024-03-14T12:17:01.146748+0000","last_fullsized":"2024-03-14T12:17:01.146748+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:48.450402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146147+0000","last_change":"2024-03-14T12:17:00.150815+0000","last_active":"2024-03-14T12:17:01.146147+0000","last_peered":"2024-03-14T12:17:01.146147+0000","last_clean":"2024-03-14T12:17:01.146147+0000","last_became_active":"2024-03-14T12:17:00.149442+0000","last_became_peered":"2024-03-14T12:17:00.149442+0000","last_unstale":"2024-03-14T12:17:01.146147+0000","last_undegraded":"2024-03-14T12:17:01.146147+0000","last_fullsized":"2024-03-14T12:17:01.146147+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:05.828371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197316+0000","last_change":"2024-03-14T12:17:00.158417+0000","last_active":"2024-03-14T12:17:02.197316+0000","last_peered":"2024-03-14T12:17:02.197316+0000","last_clean":"2024-03-14T12:17:02.197316+0000","last_became_active":"2024-03-14T12:17:00.158167+0000","last_became_peered":"2024-03-14T12:17:00.158167+0000","last_unstale":"2024-03-14T12:17:02.197316+0000","last_undegraded":"2024-03-14T12:17:02.197316+0000","last_fullsized":"2024-03-14T12:17:02.197316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197213+0000","last_change":"2024-03-14T12:17:00.171366+0000","last_active":"2024-03-14T12:17:02.197213+0000","last_peered":"2024-03-14T12:17:02.197213+0000","last_clean":"2024-03-14T12:17:02.197213+0000","last_became_active":"2024-03-14T12:17:00.170528+0000","last_became_peered":"2024-03-14T12:17:00.170528+0000","last_unstale":"2024-03-14T12:17:02.197213+0000","last_undegraded":"2024-03-14T12:17:02.197213+0000","last_fullsized":"2024-03-14T12:17:02.197213+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145897+0000","last_change":"2024-03-14T12:17:00.164124+0000","last_active":"2024-03-14T12:17:01.145897+0000","last_peered":"2024-03-14T12:17:01.145897+0000","last_clean":"2024-03-14T12:17:01.145897+0000","last_became_active":"2024-03-14T12:17:00.163838+0000","last_became_peered":"2024-03-14T12:17:00.163838+0000","last_unstale":"2024-03-14T12:17:01.145897+0000","last_undegraded":"2024-03-14T12:17:01.145897+0000","last_fullsized":"2024-03-14T12:17:01.145897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:19.263347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146009+0000","last_change":"2024-03-14T12:17:00.169710+0000","last_active":"2024-03-14T12:17:01.146009+0000","last_peered":"2024-03-14T12:17:01.146009+0000","last_clean":"2024-03-14T12:17:01.146009+0000","last_became_active":"2024-03-14T12:17:00.169437+0000","last_became_peered":"2024-03-14T12:17:00.169437+0000","last_unstale":"2024-03-14T12:17:01.146009+0000","last_undegraded":"2024-03-14T12:17:01.146009+0000","last_fullsized":"2024-03-14T12:17:01.146009+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191164+0000","last_change":"2024-03-14T12:17:00.161546+0000","last_active":"2024-03-14T12:17:02.191164+0000","last_peered":"2024-03-14T12:17:02.191164+0000","last_clean":"2024-03-14T12:17:02.191164+0000","last_became_active":"2024-03-14T12:17:00.161174+0000","last_became_peered":"2024-03-14T12:17:00.161174+0000","last_unstale":"2024-03-14T12:17:02.191164+0000","last_undegraded":"2024-03-14T12:17:02.191164+0000","last_fullsized":"2024-03-14T12:17:02.191164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:05.720182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147357+0000","last_change":"2024-03-14T12:17:00.150641+0000","last_active":"2024-03-14T12:17:01.147357+0000","last_peered":"2024-03-14T12:17:01.147357+0000","last_clean":"2024-03-14T12:17:01.147357+0000","last_became_active":"2024-03-14T12:17:00.149567+0000","last_became_peered":"2024-03-14T12:17:00.149567+0000","last_unstale":"2024-03-14T12:17:01.147357+0000","last_undegraded":"2024-03-14T12:17:01.147357+0000","last_fullsized":"2024-03-14T12:17:01.147357+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:25.157808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191238+0000","last_change":"2024-03-14T12:17:00.160024+0000","last_active":"2024-03-14T12:17:02.191238+0000","last_peered":"2024-03-14T12:17:02.191238+0000","last_clean":"2024-03-14T12:17:02.191238+0000","last_became_active":"2024-03-14T12:17:00.159513+0000","last_became_peered":"2024-03-14T12:17:00.159513+0000","last_unstale":"2024-03-14T12:17:02.191238+0000","last_undegraded":"2024-03-14T12:17:02.191238+0000","last_fullsized":"2024-03-14T12:17:02.191238+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:02.639220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145867+0000","last_change":"2024-03-14T12:17:00.171649+0000","last_active":"2024-03-14T12:17:01.145867+0000","last_peered":"2024-03-14T12:17:01.145867+0000","last_clean":"2024-03-14T12:17:01.145867+0000","last_became_active":"2024-03-14T12:17:00.171384+0000","last_became_peered":"2024-03-14T12:17:00.171384+0000","last_unstale":"2024-03-14T12:17:01.145867+0000","last_undegraded":"2024-03-14T12:17:01.145867+0000","last_fullsized":"2024-03-14T12:17:01.145867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:50:38.677924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146359+0000","last_change":"2024-03-14T12:17:00.164315+0000","last_active":"2024-03-14T12:17:01.146359+0000","last_peered":"2024-03-14T12:17:01.146359+0000","last_clean":"2024-03-14T12:17:01.146359+0000","last_became_active":"2024-03-14T12:17:00.164057+0000","last_became_peered":"2024-03-14T12:17:00.164057+0000","last_unstale":"2024-03-14T12:17:01.146359+0000","last_undegraded":"2024-03-14T12:17:01.146359+0000","last_fullsized":"2024-03-14T12:17:01.146359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198081+0000","last_change":"2024-03-14T12:17:00.156339+0000","last_active":"2024-03-14T12:17:02.198081+0000","last_peered":"2024-03-14T12:17:02.198081+0000","last_clean":"2024-03-14T12:17:02.198081+0000","last_became_active":"2024-03-14T12:17:00.156068+0000","last_became_peered":"2024-03-14T12:17:00.156068+0000","last_unstale":"2024-03-14T12:17:02.198081+0000","last_undegraded":"2024-03-14T12:17:02.198081+0000","last_fullsized":"2024-03-14T12:17:02.198081+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:05.828371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197795+0000","last_change":"2024-03-14T12:17:00.164288+0000","last_active":"2024-03-14T12:17:02.197795+0000","last_peered":"2024-03-14T12:17:02.197795+0000","last_clean":"2024-03-14T12:17:02.197795+0000","last_became_active":"2024-03-14T12:17:00.163969+0000","last_became_peered":"2024-03-14T12:17:00.163969+0000","last_unstale":"2024-03-14T12:17:02.197795+0000","last_undegraded":"2024-03-14T12:17:02.197795+0000","last_fullsized":"2024-03-14T12:17:02.197795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:10:36.197658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197849+0000","last_change":"2024-03-14T12:16:59.160707+0000","last_active":"2024-03-14T12:17:02.197849+0000","last_peered":"2024-03-14T12:17:02.197849+0000","last_clean":"2024-03-14T12:17:02.197849+0000","last_became_active":"2024-03-14T12:16:59.160442+0000","last_became_peered":"2024-03-14T12:16:59.160442+0000","last_unstale":"2024-03-14T12:17:02.197849+0000","last_undegraded":"2024-03-14T12:17:02.197849+0000","last_fullsized":"2024-03-14T12:17:02.197849+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:06.652925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144948+0000","last_change":"2024-03-14T12:17:00.152459+0000","last_active":"2024-03-14T12:17:01.144948+0000","last_peered":"2024-03-14T12:17:01.144948+0000","last_clean":"2024-03-14T12:17:01.144948+0000","last_became_active":"2024-03-14T12:17:00.152163+0000","last_became_peered":"2024-03-14T12:17:00.152163+0000","last_unstale":"2024-03-14T12:17:01.144948+0000","last_undegraded":"2024-03-14T12:17:01.144948+0000","last_fullsized":"2024-03-14T12:17:01.144948+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:52.251106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159748+0000","last_change":"2024-03-14T12:16:59.160021+0000","last_active":"2024-03-14T12:17:02.159748+0000","last_peered":"2024-03-14T12:17:02.159748+0000","last_clean":"2024-03-14T12:17:02.159748+0000","last_became_active":"2024-03-14T12:16:59.159736+0000","last_became_peered":"2024-03-14T12:16:59.159736+0000","last_unstale":"2024-03-14T12:17:02.159748+0000","last_undegraded":"2024-03-14T12:17:02.159748+0000","last_fullsized":"2024-03-14T12:17:02.159748+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:53.805112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198429+0000","last_change":"2024-03-14T12:17:00.162771+0000","last_active":"2024-03-14T12:17:02.198429+0000","last_peered":"2024-03-14T12:17:02.198429+0000","last_clean":"2024-03-14T12:17:02.198429+0000","last_became_active":"2024-03-14T12:17:00.162509+0000","last_became_peered":"2024-03-14T12:17:00.162509+0000","last_unstale":"2024-03-14T12:17:02.198429+0000","last_undegraded":"2024-03-14T12:17:02.198429+0000","last_fullsized":"2024-03-14T12:17:02.198429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:48.450402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146397+0000","last_change":"2024-03-14T12:16:59.150884+0000","last_active":"2024-03-14T12:17:01.146397+0000","last_peered":"2024-03-14T12:17:01.146397+0000","last_clean":"2024-03-14T12:17:01.146397+0000","last_became_active":"2024-03-14T12:16:59.150534+0000","last_became_peered":"2024-03-14T12:16:59.150534+0000","last_unstale":"2024-03-14T12:17:01.146397+0000","last_undegraded":"2024-03-14T12:17:01.146397+0000","last_fullsized":"2024-03-14T12:17:01.146397+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:39.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197636+0000","last_change":"2024-03-14T12:17:00.161330+0000","last_active":"2024-03-14T12:17:02.197636+0000","last_peered":"2024-03-14T12:17:02.197636+0000","last_clean":"2024-03-14T12:17:02.197636+0000","last_became_active":"2024-03-14T12:17:00.161060+0000","last_became_peered":"2024-03-14T12:17:00.161060+0000","last_unstale":"2024-03-14T12:17:02.197636+0000","last_undegraded":"2024-03-14T12:17:02.197636+0000","last_fullsized":"2024-03-14T12:17:02.197636+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:53:48.285207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198508+0000","last_change":"2024-03-14T12:16:59.160120+0000","last_active":"2024-03-14T12:17:02.198508+0000","last_peered":"2024-03-14T12:17:02.198508+0000","last_clean":"2024-03-14T12:17:02.198508+0000","last_became_active":"2024-03-14T12:16:59.159810+0000","last_became_peered":"2024-03-14T12:16:59.159810+0000","last_unstale":"2024-03-14T12:17:02.198508+0000","last_undegraded":"2024-03-14T12:17:02.198508+0000","last_fullsized":"2024-03-14T12:17:02.198508+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:23.661196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145933+0000","last_change":"2024-03-14T12:17:00.160986+0000","last_active":"2024-03-14T12:17:01.145933+0000","last_peered":"2024-03-14T12:17:01.145933+0000","last_clean":"2024-03-14T12:17:01.145933+0000","last_became_active":"2024-03-14T12:17:00.160699+0000","last_became_peered":"2024-03-14T12:17:00.160699+0000","last_unstale":"2024-03-14T12:17:01.145933+0000","last_undegraded":"2024-03-14T12:17:01.145933+0000","last_fullsized":"2024-03-14T12:17:01.145933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:04:30.945677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190985+0000","last_change":"2024-03-14T12:16:59.158332+0000","last_active":"2024-03-14T12:17:02.190985+0000","last_peered":"2024-03-14T12:17:02.190985+0000","last_clean":"2024-03-14T12:17:02.190985+0000","last_became_active":"2024-03-14T12:16:59.158049+0000","last_became_peered":"2024-03-14T12:16:59.158049+0000","last_unstale":"2024-03-14T12:17:02.190985+0000","last_undegraded":"2024-03-14T12:17:02.190985+0000","last_fullsized":"2024-03-14T12:17:02.190985+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:16:10.934601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146888+0000","last_change":"2024-03-14T12:17:00.159212+0000","last_active":"2024-03-14T12:17:01.146888+0000","last_peered":"2024-03-14T12:17:01.146888+0000","last_clean":"2024-03-14T12:17:01.146888+0000","last_became_active":"2024-03-14T12:17:00.158951+0000","last_became_peered":"2024-03-14T12:17:00.158951+0000","last_unstale":"2024-03-14T12:17:01.146888+0000","last_undegraded":"2024-03-14T12:17:01.146888+0000","last_fullsized":"2024-03-14T12:17:01.146888+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:03.354492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197981+0000","last_change":"2024-03-14T12:16:59.158539+0000","last_active":"2024-03-14T12:17:02.197981+0000","last_peered":"2024-03-14T12:17:02.197981+0000","last_clean":"2024-03-14T12:17:02.197981+0000","last_became_active":"2024-03-14T12:16:59.158223+0000","last_became_peered":"2024-03-14T12:16:59.158223+0000","last_unstale":"2024-03-14T12:17:02.197981+0000","last_undegraded":"2024-03-14T12:17:02.197981+0000","last_fullsized":"2024-03-14T12:17:02.197981+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:19:21.430991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144255+0000","last_change":"2024-03-14T12:17:00.146758+0000","last_active":"2024-03-14T12:17:01.144255+0000","last_peered":"2024-03-14T12:17:01.144255+0000","last_clean":"2024-03-14T12:17:01.144255+0000","last_became_active":"2024-03-14T12:17:00.146485+0000","last_became_peered":"2024-03-14T12:17:00.146485+0000","last_unstale":"2024-03-14T12:17:01.144255+0000","last_undegraded":"2024-03-14T12:17:01.144255+0000","last_fullsized":"2024-03-14T12:17:01.144255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:57:23.282105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196659+0000","last_change":"2024-03-14T12:16:59.153534+0000","last_active":"2024-03-14T12:17:02.196659+0000","last_peered":"2024-03-14T12:17:02.196659+0000","last_clean":"2024-03-14T12:17:02.196659+0000","last_became_active":"2024-03-14T12:16:59.153249+0000","last_became_peered":"2024-03-14T12:16:59.153249+0000","last_unstale":"2024-03-14T12:17:02.196659+0000","last_undegraded":"2024-03-14T12:17:02.196659+0000","last_fullsized":"2024-03-14T12:17:02.196659+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:57:22.276798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159422+0000","last_change":"2024-03-14T12:17:00.161414+0000","last_active":"2024-03-14T12:17:02.159422+0000","last_peered":"2024-03-14T12:17:02.159422+0000","last_clean":"2024-03-14T12:17:02.159422+0000","last_became_active":"2024-03-14T12:17:00.161178+0000","last_became_peered":"2024-03-14T12:17:00.161178+0000","last_unstale":"2024-03-14T12:17:02.159422+0000","last_undegraded":"2024-03-14T12:17:02.159422+0000","last_fullsized":"2024-03-14T12:17:02.159422+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:05:10.595088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144186+0000","last_change":"2024-03-14T12:16:59.156881+0000","last_active":"2024-03-14T12:17:01.144186+0000","last_peered":"2024-03-14T12:17:01.144186+0000","last_clean":"2024-03-14T12:17:01.144186+0000","last_became_active":"2024-03-14T12:16:59.156609+0000","last_became_peered":"2024-03-14T12:16:59.156609+0000","last_unstale":"2024-03-14T12:17:01.144186+0000","last_undegraded":"2024-03-14T12:17:01.144186+0000","last_fullsized":"2024-03-14T12:17:01.144186+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:14:57.357163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146160+0000","last_change":"2024-03-14T12:17:00.163114+0000","last_active":"2024-03-14T12:17:01.146160+0000","last_peered":"2024-03-14T12:17:01.146160+0000","last_clean":"2024-03-14T12:17:01.146160+0000","last_became_active":"2024-03-14T12:17:00.162835+0000","last_became_peered":"2024-03-14T12:17:00.162835+0000","last_unstale":"2024-03-14T12:17:01.146160+0000","last_undegraded":"2024-03-14T12:17:01.146160+0000","last_fullsized":"2024-03-14T12:17:01.146160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145963+0000","last_change":"2024-03-14T12:16:59.157422+0000","last_active":"2024-03-14T12:17:01.145963+0000","last_peered":"2024-03-14T12:17:01.145963+0000","last_clean":"2024-03-14T12:17:01.145963+0000","last_became_active":"2024-03-14T12:16:59.155980+0000","last_became_peered":"2024-03-14T12:16:59.155980+0000","last_unstale":"2024-03-14T12:17:01.145963+0000","last_undegraded":"2024-03-14T12:17:01.145963+0000","last_fullsized":"2024-03-14T12:17:01.145963+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:46.803639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197812+0000","last_change":"2024-03-14T12:17:00.167346+0000","last_active":"2024-03-14T12:17:02.197812+0000","last_peered":"2024-03-14T12:17:02.197812+0000","last_clean":"2024-03-14T12:17:02.197812+0000","last_became_active":"2024-03-14T12:17:00.167075+0000","last_became_peered":"2024-03-14T12:17:00.167075+0000","last_unstale":"2024-03-14T12:17:02.197812+0000","last_undegraded":"2024-03-14T12:17:02.197812+0000","last_fullsized":"2024-03-14T12:17:02.197812+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:36.504209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198643+0000","last_change":"2024-03-14T12:16:59.154772+0000","last_active":"2024-03-14T12:17:02.198643+0000","last_peered":"2024-03-14T12:17:02.198643+0000","last_clean":"2024-03-14T12:17:02.198643+0000","last_became_active":"2024-03-14T12:16:59.154363+0000","last_became_peered":"2024-03-14T12:16:59.154363+0000","last_unstale":"2024-03-14T12:17:02.198643+0000","last_undegraded":"2024-03-14T12:17:02.198643+0000","last_fullsized":"2024-03-14T12:17:02.198643+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:46.803639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198434+0000","last_change":"2024-03-14T12:17:00.156703+0000","last_active":"2024-03-14T12:17:02.198434+0000","last_peered":"2024-03-14T12:17:02.198434+0000","last_clean":"2024-03-14T12:17:02.198434+0000","last_became_active":"2024-03-14T12:17:00.156256+0000","last_became_peered":"2024-03-14T12:17:00.156256+0000","last_unstale":"2024-03-14T12:17:02.198434+0000","last_undegraded":"2024-03-14T12:17:02.198434+0000","last_fullsized":"2024-03-14T12:17:02.198434+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:03.354492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191221+0000","last_change":"2024-03-14T12:16:59.160192+0000","last_active":"2024-03-14T12:17:02.191221+0000","last_peered":"2024-03-14T12:17:02.191221+0000","last_clean":"2024-03-14T12:17:02.191221+0000","last_became_active":"2024-03-14T12:16:59.159901+0000","last_became_peered":"2024-03-14T12:16:59.159901+0000","last_unstale":"2024-03-14T12:17:02.191221+0000","last_undegraded":"2024-03-14T12:17:02.191221+0000","last_fullsized":"2024-03-14T12:17:02.191221+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:28:07.504769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144649+0000","last_change":"2024-03-14T12:17:00.150822+0000","last_active":"2024-03-14T12:17:01.144649+0000","last_peered":"2024-03-14T12:17:01.144649+0000","last_clean":"2024-03-14T12:17:01.144649+0000","last_became_active":"2024-03-14T12:17:00.150463+0000","last_became_peered":"2024-03-14T12:17:00.150463+0000","last_unstale":"2024-03-14T12:17:01.144649+0000","last_undegraded":"2024-03-14T12:17:01.144649+0000","last_fullsized":"2024-03-14T12:17:01.144649+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:39.380311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197081+0000","last_change":"2024-03-14T12:16:59.147830+0000","last_active":"2024-03-14T12:17:02.197081+0000","last_peered":"2024-03-14T12:17:02.197081+0000","last_clean":"2024-03-14T12:17:02.197081+0000","last_became_active":"2024-03-14T12:16:59.147503+0000","last_became_peered":"2024-03-14T12:16:59.147503+0000","last_unstale":"2024-03-14T12:17:02.197081+0000","last_undegraded":"2024-03-14T12:17:02.197081+0000","last_fullsized":"2024-03-14T12:17:02.197081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:48.936304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197326+0000","last_change":"2024-03-14T12:17:00.161960+0000","last_active":"2024-03-14T12:17:02.197326+0000","last_peered":"2024-03-14T12:17:02.197326+0000","last_clean":"2024-03-14T12:17:02.197326+0000","last_became_active":"2024-03-14T12:17:00.161650+0000","last_became_peered":"2024-03-14T12:17:00.161650+0000","last_unstale":"2024-03-14T12:17:02.197326+0000","last_undegraded":"2024-03-14T12:17:02.197326+0000","last_fullsized":"2024-03-14T12:17:02.197326+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196793+0000","last_change":"2024-03-14T12:16:59.156977+0000","last_active":"2024-03-14T12:17:02.196793+0000","last_peered":"2024-03-14T12:17:02.196793+0000","last_clean":"2024-03-14T12:17:02.196793+0000","last_became_active":"2024-03-14T12:16:59.156715+0000","last_became_peered":"2024-03-14T12:16:59.156715+0000","last_unstale":"2024-03-14T12:17:02.196793+0000","last_undegraded":"2024-03-14T12:17:02.196793+0000","last_fullsized":"2024-03-14T12:17:02.196793+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:34.831573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145991+0000","last_change":"2024-03-14T12:17:00.170808+0000","last_active":"2024-03-14T12:17:01.145991+0000","last_peered":"2024-03-14T12:17:01.145991+0000","last_clean":"2024-03-14T12:17:01.145991+0000","last_became_active":"2024-03-14T12:17:00.170529+0000","last_became_peered":"2024-03-14T12:17:00.170529+0000","last_unstale":"2024-03-14T12:17:01.145991+0000","last_undegraded":"2024-03-14T12:17:01.145991+0000","last_fullsized":"2024-03-14T12:17:01.145991+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:05:10.595088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198068+0000","last_change":"2024-03-14T12:16:59.162979+0000","last_active":"2024-03-14T12:17:02.198068+0000","last_peered":"2024-03-14T12:17:02.198068+0000","last_clean":"2024-03-14T12:17:02.198068+0000","last_became_active":"2024-03-14T12:16:59.162698+0000","last_became_peered":"2024-03-14T12:16:59.162698+0000","last_unstale":"2024-03-14T12:17:02.198068+0000","last_undegraded":"2024-03-14T12:17:02.198068+0000","last_fullsized":"2024-03-14T12:17:02.198068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:51.245799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159543+0000","last_change":"2024-03-14T12:17:00.157109+0000","last_active":"2024-03-14T12:17:02.159543+0000","last_peered":"2024-03-14T12:17:02.159543+0000","last_clean":"2024-03-14T12:17:02.159543+0000","last_became_active":"2024-03-14T12:17:00.156852+0000","last_became_peered":"2024-03-14T12:17:00.156852+0000","last_unstale":"2024-03-14T12:17:02.159543+0000","last_undegraded":"2024-03-14T12:17:02.159543+0000","last_fullsized":"2024-03-14T12:17:02.159543+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146075+0000","last_change":"2024-03-14T12:16:59.157658+0000","last_active":"2024-03-14T12:17:01.146075+0000","last_peered":"2024-03-14T12:17:01.146075+0000","last_clean":"2024-03-14T12:17:01.146075+0000","last_became_active":"2024-03-14T12:16:59.156521+0000","last_became_peered":"2024-03-14T12:16:59.156521+0000","last_unstale":"2024-03-14T12:17:01.146075+0000","last_undegraded":"2024-03-14T12:17:01.146075+0000","last_fullsized":"2024-03-14T12:17:01.146075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:18.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144334+0000","last_change":"2024-03-14T12:17:00.152560+0000","last_active":"2024-03-14T12:17:01.144334+0000","last_peered":"2024-03-14T12:17:01.144334+0000","last_clean":"2024-03-14T12:17:01.144334+0000","last_became_active":"2024-03-14T12:17:00.152302+0000","last_became_peered":"2024-03-14T12:17:00.152302+0000","last_unstale":"2024-03-14T12:17:01.144334+0000","last_undegraded":"2024-03-14T12:17:01.144334+0000","last_fullsized":"2024-03-14T12:17:01.144334+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:46.281680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147065+0000","last_change":"2024-03-14T12:16:59.157970+0000","last_active":"2024-03-14T12:17:01.147065+0000","last_peered":"2024-03-14T12:17:01.147065+0000","last_clean":"2024-03-14T12:17:01.147065+0000","last_became_active":"2024-03-14T12:16:59.157694+0000","last_became_peered":"2024-03-14T12:16:59.157694+0000","last_unstale":"2024-03-14T12:17:01.147065+0000","last_undegraded":"2024-03-14T12:17:01.147065+0000","last_fullsized":"2024-03-14T12:17:01.147065+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:47.565994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146444+0000","last_change":"2024-03-14T12:17:00.151385+0000","last_active":"2024-03-14T12:17:01.146444+0000","last_peered":"2024-03-14T12:17:01.146444+0000","last_clean":"2024-03-14T12:17:01.146444+0000","last_became_active":"2024-03-14T12:17:00.151140+0000","last_became_peered":"2024-03-14T12:17:00.151140+0000","last_unstale":"2024-03-14T12:17:01.146444+0000","last_undegraded":"2024-03-14T12:17:01.146444+0000","last_fullsized":"2024-03-14T12:17:01.146444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:07.658231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198497+0000","last_change":"2024-03-14T12:16:59.161389+0000","last_active":"2024-03-14T12:17:02.198497+0000","last_peered":"2024-03-14T12:17:02.198497+0000","last_clean":"2024-03-14T12:17:02.198497+0000","last_became_active":"2024-03-14T12:16:59.161130+0000","last_became_peered":"2024-03-14T12:16:59.161130+0000","last_unstale":"2024-03-14T12:17:02.198497+0000","last_undegraded":"2024-03-14T12:17:02.198497+0000","last_fullsized":"2024-03-14T12:17:02.198497+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:36.863630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197907+0000","last_change":"2024-03-14T12:17:00.168001+0000","last_active":"2024-03-14T12:17:02.197907+0000","last_peered":"2024-03-14T12:17:02.197907+0000","last_clean":"2024-03-14T12:17:02.197907+0000","last_became_active":"2024-03-14T12:17:00.167735+0000","last_became_peered":"2024-03-14T12:17:00.167735+0000","last_unstale":"2024-03-14T12:17:02.197907+0000","last_undegraded":"2024-03-14T12:17:02.197907+0000","last_fullsized":"2024-03-14T12:17:02.197907+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:16:11.939907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146483+0000","last_change":"2024-03-14T12:16:59.153745+0000","last_active":"2024-03-14T12:17:01.146483+0000","last_peered":"2024-03-14T12:17:01.146483+0000","last_clean":"2024-03-14T12:17:01.146483+0000","last_became_active":"2024-03-14T12:16:59.153456+0000","last_became_peered":"2024-03-14T12:16:59.153456+0000","last_unstale":"2024-03-14T12:17:01.146483+0000","last_undegraded":"2024-03-14T12:17:01.146483+0000","last_fullsized":"2024-03-14T12:17:01.146483+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:36:23.425887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197129+0000","last_change":"2024-03-14T12:17:00.161390+0000","last_active":"2024-03-14T12:17:02.197129+0000","last_peered":"2024-03-14T12:17:02.197129+0000","last_clean":"2024-03-14T12:17:02.197129+0000","last_became_active":"2024-03-14T12:17:00.161087+0000","last_became_peered":"2024-03-14T12:17:00.161087+0000","last_unstale":"2024-03-14T12:17:02.197129+0000","last_undegraded":"2024-03-14T12:17:02.197129+0000","last_fullsized":"2024-03-14T12:17:02.197129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:40.293976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198465+0000","last_change":"2024-03-14T12:16:59.151077+0000","last_active":"2024-03-14T12:17:02.198465+0000","last_peered":"2024-03-14T12:17:02.198465+0000","last_clean":"2024-03-14T12:17:02.198465+0000","last_became_active":"2024-03-14T12:16:59.150728+0000","last_became_peered":"2024-03-14T12:16:59.150728+0000","last_unstale":"2024-03-14T12:17:02.198465+0000","last_undegraded":"2024-03-14T12:17:02.198465+0000","last_fullsized":"2024-03-14T12:17:02.198465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:44.730420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144710+0000","last_change":"2024-03-14T12:17:00.148497+0000","last_active":"2024-03-14T12:17:01.144710+0000","last_peered":"2024-03-14T12:17:01.144710+0000","last_clean":"2024-03-14T12:17:01.144710+0000","last_became_active":"2024-03-14T12:17:00.148125+0000","last_became_peered":"2024-03-14T12:17:00.148125+0000","last_unstale":"2024-03-14T12:17:01.144710+0000","last_undegraded":"2024-03-14T12:17:01.144710+0000","last_fullsized":"2024-03-14T12:17:01.144710+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:19:22.436297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197406+0000","last_change":"2024-03-14T12:16:59.160922+0000","last_active":"2024-03-14T12:17:02.197406+0000","last_peered":"2024-03-14T12:17:02.197406+0000","last_clean":"2024-03-14T12:17:02.197406+0000","last_became_active":"2024-03-14T12:16:59.160607+0000","last_became_peered":"2024-03-14T12:16:59.160607+0000","last_unstale":"2024-03-14T12:17:02.197406+0000","last_undegraded":"2024-03-14T12:17:02.197406+0000","last_fullsized":"2024-03-14T12:17:02.197406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:38.196573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190932+0000","last_change":"2024-03-14T12:17:00.156143+0000","last_active":"2024-03-14T12:17:02.190932+0000","last_peered":"2024-03-14T12:17:02.190932+0000","last_clean":"2024-03-14T12:17:02.190932+0000","last_became_active":"2024-03-14T12:17:00.155844+0000","last_became_peered":"2024-03-14T12:17:00.155844+0000","last_unstale":"2024-03-14T12:17:02.190932+0000","last_undegraded":"2024-03-14T12:17:02.190932+0000","last_fullsized":"2024-03-14T12:17:02.190932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:49:56.360025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198120+0000","last_change":"2024-03-14T12:16:59.158103+0000","last_active":"2024-03-14T12:17:02.198120+0000","last_peered":"2024-03-14T12:17:02.198120+0000","last_clean":"2024-03-14T12:17:02.198120+0000","last_became_active":"2024-03-14T12:16:59.157770+0000","last_became_peered":"2024-03-14T12:16:59.157770+0000","last_unstale":"2024-03-14T12:17:02.198120+0000","last_undegraded":"2024-03-14T12:17:02.198120+0000","last_fullsized":"2024-03-14T12:17:02.198120+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:49.373989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190827+0000","last_change":"2024-03-14T12:17:00.159001+0000","last_active":"2024-03-14T12:17:02.190827+0000","last_peered":"2024-03-14T12:17:02.190827+0000","last_clean":"2024-03-14T12:17:02.190827+0000","last_became_active":"2024-03-14T12:17:00.158720+0000","last_became_peered":"2024-03-14T12:17:00.158720+0000","last_unstale":"2024-03-14T12:17:02.190827+0000","last_undegraded":"2024-03-14T12:17:02.190827+0000","last_fullsized":"2024-03-14T12:17:02.190827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:00:30.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146166+0000","last_change":"2024-03-14T12:16:59.149014+0000","last_active":"2024-03-14T12:17:01.146166+0000","last_peered":"2024-03-14T12:17:01.146166+0000","last_clean":"2024-03-14T12:17:01.146166+0000","last_became_active":"2024-03-14T12:16:59.148708+0000","last_became_peered":"2024-03-14T12:16:59.148708+0000","last_unstale":"2024-03-14T12:17:01.146166+0000","last_undegraded":"2024-03-14T12:17:01.146166+0000","last_fullsized":"2024-03-14T12:17:01.146166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:04.823064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190842+0000","last_change":"2024-03-14T12:17:00.158300+0000","last_active":"2024-03-14T12:17:02.190842+0000","last_peered":"2024-03-14T12:17:02.190842+0000","last_clean":"2024-03-14T12:17:02.190842+0000","last_became_active":"2024-03-14T12:17:00.158049+0000","last_became_peered":"2024-03-14T12:17:00.158049+0000","last_unstale":"2024-03-14T12:17:02.190842+0000","last_undegraded":"2024-03-14T12:17:02.190842+0000","last_fullsized":"2024-03-14T12:17:02.190842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:33:51.561745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159206+0000","last_change":"2024-03-14T12:16:59.163118+0000","last_active":"2024-03-14T12:17:02.159206+0000","last_peered":"2024-03-14T12:17:02.159206+0000","last_clean":"2024-03-14T12:17:02.159206+0000","last_became_active":"2024-03-14T12:16:59.162808+0000","last_became_peered":"2024-03-14T12:16:59.162808+0000","last_unstale":"2024-03-14T12:17:02.159206+0000","last_undegraded":"2024-03-14T12:17:02.159206+0000","last_fullsized":"2024-03-14T12:17:02.159206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:02.349185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159686+0000","last_change":"2024-03-14T12:17:00.161657+0000","last_active":"2024-03-14T12:17:02.159686+0000","last_peered":"2024-03-14T12:17:02.159686+0000","last_clean":"2024-03-14T12:17:02.159686+0000","last_became_active":"2024-03-14T12:17:00.160635+0000","last_became_peered":"2024-03-14T12:17:00.160635+0000","last_unstale":"2024-03-14T12:17:02.159686+0000","last_undegraded":"2024-03-14T12:17:02.159686+0000","last_fullsized":"2024-03-14T12:17:02.159686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198729+0000","last_change":"2024-03-14T12:16:59.160742+0000","last_active":"2024-03-14T12:17:02.198729+0000","last_peered":"2024-03-14T12:17:02.198729+0000","last_clean":"2024-03-14T12:17:02.198729+0000","last_became_active":"2024-03-14T12:16:59.160452+0000","last_became_peered":"2024-03-14T12:16:59.160452+0000","last_unstale":"2024-03-14T12:17:02.198729+0000","last_undegraded":"2024-03-14T12:17:02.198729+0000","last_fullsized":"2024-03-14T12:17:02.198729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:47.134831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146041+0000","last_change":"2024-03-14T12:17:00.161317+0000","last_active":"2024-03-14T12:17:01.146041+0000","last_peered":"2024-03-14T12:17:01.146041+0000","last_clean":"2024-03-14T12:17:01.146041+0000","last_became_active":"2024-03-14T12:17:00.161046+0000","last_became_peered":"2024-03-14T12:17:00.161046+0000","last_unstale":"2024-03-14T12:17:01.146041+0000","last_undegraded":"2024-03-14T12:17:01.146041+0000","last_fullsized":"2024-03-14T12:17:01.146041+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146258+0000","last_change":"2024-03-14T12:16:59.157167+0000","last_active":"2024-03-14T12:17:01.146258+0000","last_peered":"2024-03-14T12:17:01.146258+0000","last_clean":"2024-03-14T12:17:01.146258+0000","last_became_active":"2024-03-14T12:16:59.156847+0000","last_became_peered":"2024-03-14T12:16:59.156847+0000","last_unstale":"2024-03-14T12:17:01.146258+0000","last_undegraded":"2024-03-14T12:17:01.146258+0000","last_fullsized":"2024-03-14T12:17:01.146258+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:03:03.012491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197493+0000","last_change":"2024-03-14T12:17:00.168815+0000","last_active":"2024-03-14T12:17:02.197493+0000","last_peered":"2024-03-14T12:17:02.197493+0000","last_clean":"2024-03-14T12:17:02.197493+0000","last_became_active":"2024-03-14T12:17:00.168561+0000","last_became_peered":"2024-03-14T12:17:00.168561+0000","last_unstale":"2024-03-14T12:17:02.197493+0000","last_undegraded":"2024-03-14T12:17:02.197493+0000","last_fullsized":"2024-03-14T12:17:02.197493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:03:50.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144705+0000","last_change":"2024-03-14T12:16:59.157608+0000","last_active":"2024-03-14T12:17:01.144705+0000","last_peered":"2024-03-14T12:17:01.144705+0000","last_clean":"2024-03-14T12:17:01.144705+0000","last_became_active":"2024-03-14T12:16:59.157339+0000","last_became_peered":"2024-03-14T12:16:59.157339+0000","last_unstale":"2024-03-14T12:17:01.144705+0000","last_undegraded":"2024-03-14T12:17:01.144705+0000","last_fullsized":"2024-03-14T12:17:01.144705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:28.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197678+0000","last_change":"2024-03-14T12:17:00.163234+0000","last_active":"2024-03-14T12:17:02.197678+0000","last_peered":"2024-03-14T12:17:02.197678+0000","last_clean":"2024-03-14T12:17:02.197678+0000","last_became_active":"2024-03-14T12:17:00.162922+0000","last_became_peered":"2024-03-14T12:17:00.162922+0000","last_unstale":"2024-03-14T12:17:02.197678+0000","last_undegraded":"2024-03-14T12:17:02.197678+0000","last_fullsized":"2024-03-14T12:17:02.197678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:54:53.421821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159555+0000","last_change":"2024-03-14T12:16:59.161879+0000","last_active":"2024-03-14T12:17:02.159555+0000","last_peered":"2024-03-14T12:17:02.159555+0000","last_clean":"2024-03-14T12:17:02.159555+0000","last_became_active":"2024-03-14T12:16:59.161612+0000","last_became_peered":"2024-03-14T12:16:59.161612+0000","last_unstale":"2024-03-14T12:17:02.159555+0000","last_undegraded":"2024-03-14T12:17:02.159555+0000","last_fullsized":"2024-03-14T12:17:02.159555+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:24.152502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144466+0000","last_change":"2024-03-14T12:17:00.152780+0000","last_active":"2024-03-14T12:17:01.144466+0000","last_peered":"2024-03-14T12:17:01.144466+0000","last_clean":"2024-03-14T12:17:01.144466+0000","last_became_active":"2024-03-14T12:17:00.152426+0000","last_became_peered":"2024-03-14T12:17:00.152426+0000","last_unstale":"2024-03-14T12:17:01.144466+0000","last_undegraded":"2024-03-14T12:17:01.144466+0000","last_fullsized":"2024-03-14T12:17:01.144466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:49.941611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144339+0000","last_change":"2024-03-14T12:16:59.150881+0000","last_active":"2024-03-14T12:17:01.144339+0000","last_peered":"2024-03-14T12:17:01.144339+0000","last_clean":"2024-03-14T12:17:01.144339+0000","last_became_active":"2024-03-14T12:16:59.150578+0000","last_became_peered":"2024-03-14T12:16:59.150578+0000","last_unstale":"2024-03-14T12:17:01.144339+0000","last_undegraded":"2024-03-14T12:17:01.144339+0000","last_fullsized":"2024-03-14T12:17:01.144339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:02.510738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198277+0000","last_change":"2024-03-14T12:17:00.159754+0000","last_active":"2024-03-14T12:17:02.198277+0000","last_peered":"2024-03-14T12:17:02.198277+0000","last_clean":"2024-03-14T12:17:02.198277+0000","last_became_active":"2024-03-14T12:17:00.158664+0000","last_became_peered":"2024-03-14T12:17:00.158664+0000","last_unstale":"2024-03-14T12:17:02.198277+0000","last_undegraded":"2024-03-14T12:17:02.198277+0000","last_fullsized":"2024-03-14T12:17:02.198277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:39.201880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196679+0000","last_change":"2024-03-14T12:16:59.153605+0000","last_active":"2024-03-14T12:17:02.196679+0000","last_peered":"2024-03-14T12:17:02.196679+0000","last_clean":"2024-03-14T12:17:02.196679+0000","last_became_active":"2024-03-14T12:16:59.153330+0000","last_became_peered":"2024-03-14T12:16:59.153330+0000","last_unstale":"2024-03-14T12:17:02.196679+0000","last_undegraded":"2024-03-14T12:17:02.196679+0000","last_fullsized":"2024-03-14T12:17:02.196679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:45.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196750+0000","last_change":"2024-03-14T12:17:00.154881+0000","last_active":"2024-03-14T12:17:02.196750+0000","last_peered":"2024-03-14T12:17:02.196750+0000","last_clean":"2024-03-14T12:17:02.196750+0000","last_became_active":"2024-03-14T12:17:00.154597+0000","last_became_peered":"2024-03-14T12:17:00.154597+0000","last_unstale":"2024-03-14T12:17:02.196750+0000","last_undegraded":"2024-03-14T12:17:02.196750+0000","last_fullsized":"2024-03-14T12:17:02.196750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:45.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145806+0000","last_change":"2024-03-14T12:16:59.158773+0000","last_active":"2024-03-14T12:17:01.145806+0000","last_peered":"2024-03-14T12:17:01.145806+0000","last_clean":"2024-03-14T12:17:01.145806+0000","last_became_active":"2024-03-14T12:16:59.158497+0000","last_became_peered":"2024-03-14T12:16:59.158497+0000","last_unstale":"2024-03-14T12:17:01.145806+0000","last_undegraded":"2024-03-14T12:17:01.145806+0000","last_fullsized":"2024-03-14T12:17:01.145806+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:07:36.819982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197762+0000","last_change":"2024-03-14T12:17:00.155232+0000","last_active":"2024-03-14T12:17:02.197762+0000","last_peered":"2024-03-14T12:17:02.197762+0000","last_clean":"2024-03-14T12:17:02.197762+0000","last_became_active":"2024-03-14T12:17:00.154850+0000","last_became_peered":"2024-03-14T12:17:00.154850+0000","last_unstale":"2024-03-14T12:17:02.197762+0000","last_undegraded":"2024-03-14T12:17:02.197762+0000","last_fullsized":"2024-03-14T12:17:02.197762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:24.931278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198093+0000","last_change":"2024-03-14T12:16:59.159327+0000","last_active":"2024-03-14T12:17:02.198093+0000","last_peered":"2024-03-14T12:17:02.198093+0000","last_clean":"2024-03-14T12:17:02.198093+0000","last_became_active":"2024-03-14T12:16:59.159038+0000","last_became_peered":"2024-03-14T12:16:59.159038+0000","last_unstale":"2024-03-14T12:17:02.198093+0000","last_undegraded":"2024-03-14T12:17:02.198093+0000","last_fullsized":"2024-03-14T12:17:02.198093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:47.565994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190927+0000","last_change":"2024-03-14T12:17:00.159112+0000","last_active":"2024-03-14T12:17:02.190927+0000","last_peered":"2024-03-14T12:17:02.190927+0000","last_clean":"2024-03-14T12:17:02.190927+0000","last_became_active":"2024-03-14T12:17:00.158781+0000","last_became_peered":"2024-03-14T12:17:00.158781+0000","last_unstale":"2024-03-14T12:17:02.190927+0000","last_undegraded":"2024-03-14T12:17:02.190927+0000","last_fullsized":"2024-03-14T12:17:02.190927+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:17.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198174+0000","last_change":"2024-03-14T12:16:59.160492+0000","last_active":"2024-03-14T12:17:02.198174+0000","last_peered":"2024-03-14T12:17:02.198174+0000","last_clean":"2024-03-14T12:17:02.198174+0000","last_became_active":"2024-03-14T12:16:59.160171+0000","last_became_peered":"2024-03-14T12:16:59.160171+0000","last_unstale":"2024-03-14T12:17:02.198174+0000","last_undegraded":"2024-03-14T12:17:02.198174+0000","last_fullsized":"2024-03-14T12:17:02.198174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:38.375005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197292+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:17:02.197292+0000","last_peered":"2024-03-14T12:17:02.197292+0000","last_clean":"2024-03-14T12:17:02.197292+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:17:02.197292+0000","last_undegraded":"2024-03-14T12:17:02.197292+0000","last_fullsized":"2024-03-14T12:17:02.197292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":941994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":351102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":351100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":351080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":847872,"data_stored":351122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":352622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36180,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335660,"statfs":{"total":96636764160,"available":96599715840,"internally_reserved":0,"allocated":860160,"data_stored":353238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:17:03.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-14T12:17:03.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.931+0000 7fe0e2b3b640 1 Processor -- start 2024-03-14T12:17:03.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.931+0000 7fe0e2b3b640 1 -- start start 2024-03-14T12:17:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.932+0000 7fe0e2b3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 0x7fe0dc0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.932+0000 7fe0e2b3b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc0cf630 con 0x7fe0dc0b3270 2024-03-14T12:17:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.932+0000 7fe0e2b3b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc0cf910 con 0x7fe0dc161770 2024-03-14T12:17:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.932+0000 7fe0e2b3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0d2ea0 con 0x7fe0dc0d0ee0 2024-03-14T12:17:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.932+0000 7fe0e10d8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe0dc0b3270 0x7fe0dc0b13e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44484/0 (socket says 172.21.15.82:44484) 2024-03-14T12:17:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.932+0000 7fe0e10d8640 1 -- 172.21.15.82:0/1213457013 learned_addr learned my addr 172.21.15.82:0/1213457013 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0dbfff640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 0x7fe0dc0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822861270 0 0) 0x7fe0dc0cf630 con 0x7fe0dc0b3270 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0c4003520 con 0x7fe0dc0b3270 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3117975969 0 0) 0x7fe0c4003520 con 0x7fe0dc0b3270 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 msgr2=0x7fe0dc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 0x7fe0dc0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 >> v1:172.21.15.164:6789/0 conn(0x7fe0dc161770 legacy=0x7fe0dc0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0cfbf0 con 0x7fe0dc0b3270 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fe0d40019f0 con 0x7fe0dc0b3270 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 >> v1:172.21.15.82:6789/0 conn(0x7fe0dc0b3270 legacy=0x7fe0dc0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c40039e0 0x7fe0c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 0x7fe0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0d2ea0 con 0x7fe0c40039e0 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c400ba70 con 0x7fe0c4004800 2024-03-14T12:17:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c400bc70 con 0x7fe0dc0d0ee0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0e08d7640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c40039e0 0x7fe0c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0dbfff640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0e10d8640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 0x7fe0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0e08d7640 1 -- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 msgr2=0x7fe0c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0e08d7640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0dc0d0ee0 0x7fe0c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0e08d7640 1 -- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004800 msgr2=0x7fe0c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0e08d7640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0e08d7640 1 -- 172.21.15.82:0/1213457013 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4003520 con 0x7fe0c40039e0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.933+0000 7fe0dbfff640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004800 0x7fe0c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.934+0000 7fe0e08d7640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c40039e0 0x7fe0c4003e00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc001720 tx=0x7fe0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66338c99ec0e2bb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.934+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc001370 con 0x7fe0c40039e0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.934+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc0052d0 con 0x7fe0c40039e0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.934+0000 7fe0db7fe640 1 -- 172.21.15.82:0/1213457013 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc003830 con 0x7fe0c40039e0 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.935+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/1213457013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c40039e0 msgr2=0x7fe0c4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.935+0000 7fe0e2b3b640 1 --2- 172.21.15.82:0/1213457013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c40039e0 0x7fe0c4003e00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc001720 tx=0x7fe0c4005830 comp rx=0 tx=0).stop 2024-03-14T12:17:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.935+0000 7fe0e10d8640 1 -- 172.21.15.82:0/1213457013 reap_dead start 2024-03-14T12:17:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.935+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/1213457013 shutdown_connections 2024-03-14T12:17:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.935+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/1213457013 >> 172.21.15.82:0/1213457013 conn(0x7fe0dc09d1d0 msgr2=0x7fe0dc0d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.935+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/1213457013 shutdown_connections 2024-03-14T12:17:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.935+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/1213457013 wait complete. 2024-03-14T12:17:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.936+0000 7fe0e2b3b640 1 Processor -- start 2024-03-14T12:17:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.936+0000 7fe0e2b3b640 1 -- start start 2024-03-14T12:17:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c40039e0 0x7fe0dc09f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc0b4120 0x7fe0dc0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0dc154030 0x7fe0dc1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0d2c60 con 0x7fe0dc0b4120 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0cfed0 con 0x7fe0dc154030 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0b2c90 con 0x7fe0c40039e0 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc0b4120 0x7fe0dc0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc0b4120 0x7fe0dc0cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46308/0 (socket says 172.21.15.82:46308) 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 -- 172.21.15.82:0/3124198810 learned_addr learned my addr 172.21.15.82:0/3124198810 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e10d8640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0dc154030 0x7fe0dc1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e08d7640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c40039e0 0x7fe0dc09f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 -- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c40039e0 msgr2=0x7fe0dc09f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c40039e0 0x7fe0dc09f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 -- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0dc154030 msgr2=0x7fe0dc1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0dc154030 0x7fe0dc1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 -- 172.21.15.82:0/3124198810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0b4580 con 0x7fe0dc0b4120 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0dbfff640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc0b4120 0x7fe0dc0cd1c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0011700 tx=0x7fe0d0011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0d97fa640 1 -- 172.21.15.82:0/3124198810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000f9c0 con 0x7fe0dc0b4120 2024-03-14T12:17:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0d97fa640 1 -- 172.21.15.82:0/3124198810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d000fb60 con 0x7fe0dc0b4120 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc0b4820 con 0x7fe0dc0b4120 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0d97fa640 1 -- 172.21.15.82:0/3124198810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0015460 con 0x7fe0dc0b4120 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.937+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc0b4de0 con 0x7fe0dc0b4120 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.939+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc04f050 con 0x7fe0dc0b4120 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.939+0000 7fe0d97fa640 1 -- 172.21.15.82:0/3124198810 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe0d0015600 con 0x7fe0dc0b4120 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.939+0000 7fe0d97fa640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0c806bc00 0x7fe0c806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.940+0000 7fe0e08d7640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0c806bc00 0x7fe0c806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.940+0000 7fe0d97fa640 1 -- 172.21.15.82:0/3124198810 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0d008b3f0 con 0x7fe0dc0b4120 2024-03-14T12:17:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.940+0000 7fe0e08d7640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0c806bc00 0x7fe0c806e0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc033000 tx=0x7fe0cc009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:03.944+0000 7fe0d97fa640 1 -- 172.21.15.82:0/3124198810 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe0d0019030 con 0x7fe0dc0b4120 2024-03-14T12:17:04.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.184+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe0dc06c3c0 con 0x7fe0dc0b4120 2024-03-14T12:17:04.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.184+0000 7fe0d97fa640 1 -- 172.21.15.82:0/3124198810 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1174 (secure 0 0 0) 0x7fe0d0059e40 con 0x7fe0dc0b4120 2024-03-14T12:17:04.184 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:04.184 INFO:teuthology.orchestra.run.smithi082.stdout:{"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":216,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":69,"num_osds":8,"num_up_osds":8,"osd_up_since":1710418418,"num_in_osds":8,"osd_in_since":1710418409,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":267685888,"bytes_avail":772826427392,"bytes_total":773094113280,"write_bytes_sec":2651,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":34,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":6486}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-03-14T12:15:39.520695+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0c806bc00 msgr2=0x7fe0c806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0c806bc00 0x7fe0c806e0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc033000 tx=0x7fe0cc009030 comp rx=0 tx=0).stop 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc0b4120 msgr2=0x7fe0dc0cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc0b4120 0x7fe0dc0cd1c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0011700 tx=0x7fe0d0011bc0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 shutdown_connections 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0dc154030 0x7fe0dc1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc0b4120 0x7fe0dc0cd1c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe0c806bc00 0x7fe0c806e0c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 --2- 172.21.15.82:0/3124198810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c40039e0 0x7fe0dc09f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 >> 172.21.15.82:0/3124198810 conn(0x7fe0dc09d1d0 msgr2=0x7fe0dc0ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 shutdown_connections 2024-03-14T12:17:04.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.186+0000 7fe0e2b3b640 1 -- 172.21.15.82:0/3124198810 wait complete. 2024-03-14T12:17:04.196 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-14T12:17:04.196 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:17:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.297+0000 7fe4ed1c4640 1 Processor -- start 2024-03-14T12:17:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.297+0000 7fe4ed1c4640 1 -- start start 2024-03-14T12:17:04.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4ed1c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe4e80d0760 0x7fe4e80d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:04.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4ed1c4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe4e80b7720 0x7fe4e80b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:04.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4ed1c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e80d1080 con 0x7fe4e80cf370 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4ed1c4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e80cf8c0 con 0x7fe4e80b7720 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4ed1c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e80cfba0 con 0x7fe4e80d0760 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e759e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe4e80b7720 0x7fe4e80b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe4e80cf370 0x7fe4e80d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44496/0 (socket says 172.21.15.82:44496) 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e659c640 1 -- 172.21.15.82:0/116479778 learned_addr learned my addr 172.21.15.82:0/116479778 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e6d9d640 1 --2- 172.21.15.82:0/116479778 >> v2:172.21.15.164:3300/0 conn(0x7fe4e80d0760 0x7fe4e80d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e759e640 1 -- 172.21.15.82:0/116479778 >> v2:172.21.15.164:3300/0 conn(0x7fe4e80d0760 msgr2=0x7fe4e80d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e759e640 1 --2- 172.21.15.82:0/116479778 >> v2:172.21.15.164:3300/0 conn(0x7fe4e80d0760 0x7fe4e80d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e759e640 1 -- 172.21.15.82:0/116479778 >> v1:172.21.15.82:6789/0 conn(0x7fe4e80cf370 legacy=0x7fe4e80d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e759e640 1 -- 172.21.15.82:0/116479778 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80b6c40 con 0x7fe4e80b7720 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.298+0000 7fe4e6d9d640 1 --2- 172.21.15.82:0/116479778 >> v2:172.21.15.164:3300/0 conn(0x7fe4e80d0760 0x7fe4e80d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.299+0000 7fe4e759e640 1 --2- 172.21.15.82:0/116479778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80b7720 0x7fe4e80b7b50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe4e808ca90 tx=0x7fe4e002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e02d9d5625d29a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.299+0000 7fe4e5d9b640 1 -- 172.21.15.82:0/116479778 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0034070 con 0x7fe4e80b7720 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.299+0000 7fe4e5d9b640 1 -- 172.21.15.82:0/116479778 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0030610 con 0x7fe4e80b7720 2024-03-14T12:17:04.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.299+0000 7fe4e5d9b640 1 -- 172.21.15.82:0/116479778 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00309e0 con 0x7fe4e80b7720 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/116479778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80b7720 msgr2=0x7fe4e80b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/116479778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80b7720 0x7fe4e80b7b50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe4e808ca90 tx=0x7fe4e002b960 comp rx=0 tx=0).stop 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/116479778 shutdown_connections 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/116479778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80b7720 0x7fe4e80b7b50 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/116479778 >> v2:172.21.15.164:3300/0 conn(0x7fe4e80d0760 0x7fe4e80d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/116479778 >> 172.21.15.82:0/116479778 conn(0x7fe4e809d030 msgr2=0x7fe4e80d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/116479778 shutdown_connections 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.300+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/116479778 wait complete. 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.301+0000 7fe4ed1c4640 1 Processor -- start 2024-03-14T12:17:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.301+0000 7fe4ed1c4640 1 -- start start 2024-03-14T12:17:04.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4ed1c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e80b7720 0x7fe4e8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:04.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4ed1c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e80cf370 0x7fe4e80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:04.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4ed1c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80d0760 0x7fe4e815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4ed1c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e81617a0 con 0x7fe4e80d0760 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4ed1c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e80b7200 con 0x7fe4e80b7720 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4ed1c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e8161920 con 0x7fe4e80cf370 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4e759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80d0760 0x7fe4e815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4e759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80d0760 0x7fe4e815d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46316/0 (socket says 172.21.15.82:46316) 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4e759e640 1 -- 172.21.15.82:0/1398578821 learned_addr learned my addr 172.21.15.82:0/1398578821 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4e6d9d640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e80b7720 0x7fe4e8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.302+0000 7fe4e659c640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e80cf370 0x7fe4e80cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4e759e640 1 -- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e80cf370 msgr2=0x7fe4e80cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4e759e640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e80cf370 0x7fe4e80cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4e759e640 1 -- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e80b7720 msgr2=0x7fe4e8154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4e759e640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e80b7720 0x7fe4e8154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4e759e640 1 -- 172.21.15.82:0/1398578821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80c98f0 con 0x7fe4e80d0760 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4e759e640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80d0760 0x7fe4e815d270 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe4e002bf70 tx=0x7fe4e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4cf7fe640 1 -- 172.21.15.82:0/1398578821 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0046020 con 0x7fe4e80d0760 2024-03-14T12:17:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4cf7fe640 1 -- 172.21.15.82:0/1398578821 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0039990 con 0x7fe4e80d0760 2024-03-14T12:17:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e80c8850 con 0x7fe4e80d0760 2024-03-14T12:17:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4cf7fe640 1 -- 172.21.15.82:0/1398578821 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0034040 con 0x7fe4e80d0760 2024-03-14T12:17:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.303+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e80c90f0 con 0x7fe4e80d0760 2024-03-14T12:17:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.304+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e808d4f0 con 0x7fe4e80d0760 2024-03-14T12:17:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.304+0000 7fe4cf7fe640 1 -- 172.21.15.82:0/1398578821 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe4e0037070 con 0x7fe4e80d0760 2024-03-14T12:17:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.305+0000 7fe4cf7fe640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4d406bbd0 0x7fe4d406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.305+0000 7fe4cf7fe640 1 -- 172.21.15.82:0/1398578821 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e00ac790 con 0x7fe4e80d0760 2024-03-14T12:17:04.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.305+0000 7fe4e6d9d640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4d406bbd0 0x7fe4d406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:04.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.305+0000 7fe4e6d9d640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4d406bbd0 0x7fe4d406e090 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800f500 tx=0x7fe4d8007770 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:04.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.309+0000 7fe4cf7fe640 1 -- 172.21.15.82:0/1398578821 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe4e808d4f0 con 0x7fe4e80d0760 2024-03-14T12:17:04.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.478+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4e8073030 con 0x7fe4d406bbd0 2024-03-14T12:17:04.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.483+0000 7fe4cf7fe640 1 -- 172.21.15.82:0/1398578821 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252069 (secure 0 0 0) 0x7fe4e8073030 con 0x7fe4d406bbd0 2024-03-14T12:17:04.485 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:04.486 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4d406bbd0 msgr2=0x7fe4d406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4d406bbd0 0x7fe4d406e090 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800f500 tx=0x7fe4d8007770 comp rx=0 tx=0).stop 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80d0760 msgr2=0x7fe4e815d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80d0760 0x7fe4e815d270 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe4e002bf70 tx=0x7fe4e00046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 shutdown_connections 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e80d0760 0x7fe4e815d270 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe4d406bbd0 0x7fe4d406e090 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe4e80cf370 0x7fe4e80cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 --2- 172.21.15.82:0/1398578821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4e80b7720 0x7fe4e8154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:04.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 >> 172.21.15.82:0/1398578821 conn(0x7fe4e809d030 msgr2=0x7fe4e80d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:04.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 shutdown_connections 2024-03-14T12:17:04.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:04.487+0000 7fe4ed1c4640 1 -- 172.21.15.82:0/1398578821 wait complete. 2024-03-14T12:17:04.497 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-03-14T12:17:03.537053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261412,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713308,"statfs":{"total":773094113280,"available":772826427392,"internally_reserved":0,"allocated":7970816,"data_stored":3993636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":259575347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.407008"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197760+0000","last_change":"2024-03-14T12:17:00.165695+0000","last_active":"2024-03-14T12:17:02.197760+0000","last_peered":"2024-03-14T12:17:02.197760+0000","last_clean":"2024-03-14T12:17:02.197760+0000","last_became_active":"2024-03-14T12:17:00.165201+0000","last_became_peered":"2024-03-14T12:17:00.165201+0000","last_unstale":"2024-03-14T12:17:02.197760+0000","last_undegraded":"2024-03-14T12:17:02.197760+0000","last_fullsized":"2024-03-14T12:17:02.197760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:23:16.151346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198669+0000","last_change":"2024-03-14T12:17:00.158662+0000","last_active":"2024-03-14T12:17:02.198669+0000","last_peered":"2024-03-14T12:17:02.198669+0000","last_clean":"2024-03-14T12:17:02.198669+0000","last_became_active":"2024-03-14T12:17:00.158337+0000","last_became_peered":"2024-03-14T12:17:00.158337+0000","last_unstale":"2024-03-14T12:17:02.198669+0000","last_undegraded":"2024-03-14T12:17:02.198669+0000","last_fullsized":"2024-03-14T12:17:02.198669+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:54.810418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191019+0000","last_change":"2024-03-14T12:17:00.155441+0000","last_active":"2024-03-14T12:17:02.191019+0000","last_peered":"2024-03-14T12:17:02.191019+0000","last_clean":"2024-03-14T12:17:02.191019+0000","last_became_active":"2024-03-14T12:17:00.155153+0000","last_became_peered":"2024-03-14T12:17:00.155153+0000","last_unstale":"2024-03-14T12:17:02.191019+0000","last_undegraded":"2024-03-14T12:17:02.191019+0000","last_fullsized":"2024-03-14T12:17:02.191019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:46.598883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196932+0000","last_change":"2024-03-14T12:17:00.158103+0000","last_active":"2024-03-14T12:17:02.196932+0000","last_peered":"2024-03-14T12:17:02.196932+0000","last_clean":"2024-03-14T12:17:02.196932+0000","last_became_active":"2024-03-14T12:17:00.157827+0000","last_became_peered":"2024-03-14T12:17:00.157827+0000","last_unstale":"2024-03-14T12:17:02.196932+0000","last_undegraded":"2024-03-14T12:17:02.196932+0000","last_fullsized":"2024-03-14T12:17:02.196932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:48.140138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144471+0000","last_change":"2024-03-14T12:17:00.158865+0000","last_active":"2024-03-14T12:17:01.144471+0000","last_peered":"2024-03-14T12:17:01.144471+0000","last_clean":"2024-03-14T12:17:01.144471+0000","last_became_active":"2024-03-14T12:17:00.158562+0000","last_became_peered":"2024-03-14T12:17:00.158562+0000","last_unstale":"2024-03-14T12:17:01.144471+0000","last_undegraded":"2024-03-14T12:17:01.144471+0000","last_fullsized":"2024-03-14T12:17:01.144471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:51.747175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198057+0000","last_change":"2024-03-14T12:17:00.150382+0000","last_active":"2024-03-14T12:17:02.198057+0000","last_peered":"2024-03-14T12:17:02.198057+0000","last_clean":"2024-03-14T12:17:02.198057+0000","last_became_active":"2024-03-14T12:17:00.150078+0000","last_became_peered":"2024-03-14T12:17:00.150078+0000","last_unstale":"2024-03-14T12:17:02.198057+0000","last_undegraded":"2024-03-14T12:17:02.198057+0000","last_fullsized":"2024-03-14T12:17:02.198057+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:19.263347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145699+0000","last_change":"2024-03-14T12:17:00.163727+0000","last_active":"2024-03-14T12:17:01.145699+0000","last_peered":"2024-03-14T12:17:01.145699+0000","last_clean":"2024-03-14T12:17:01.145699+0000","last_became_active":"2024-03-14T12:17:00.163450+0000","last_became_peered":"2024-03-14T12:17:00.163450+0000","last_unstale":"2024-03-14T12:17:01.145699+0000","last_undegraded":"2024-03-14T12:17:01.145699+0000","last_fullsized":"2024-03-14T12:17:01.145699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:00:17.473435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198050+0000","last_change":"2024-03-14T12:17:00.157697+0000","last_active":"2024-03-14T12:17:02.198050+0000","last_peered":"2024-03-14T12:17:02.198050+0000","last_clean":"2024-03-14T12:17:02.198050+0000","last_became_active":"2024-03-14T12:17:00.157426+0000","last_became_peered":"2024-03-14T12:17:00.157426+0000","last_unstale":"2024-03-14T12:17:02.198050+0000","last_undegraded":"2024-03-14T12:17:02.198050+0000","last_fullsized":"2024-03-14T12:17:02.198050+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:03:04.017798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144864+0000","last_change":"2024-03-14T12:17:00.155164+0000","last_active":"2024-03-14T12:17:01.144864+0000","last_peered":"2024-03-14T12:17:01.144864+0000","last_clean":"2024-03-14T12:17:01.144864+0000","last_became_active":"2024-03-14T12:17:00.154702+0000","last_became_peered":"2024-03-14T12:17:00.154702+0000","last_unstale":"2024-03-14T12:17:01.144864+0000","last_undegraded":"2024-03-14T12:17:01.144864+0000","last_fullsized":"2024-03-14T12:17:01.144864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:50.379295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146290+0000","last_change":"2024-03-14T12:17:00.156268+0000","last_active":"2024-03-14T12:17:01.146290+0000","last_peered":"2024-03-14T12:17:01.146290+0000","last_clean":"2024-03-14T12:17:01.146290+0000","last_became_active":"2024-03-14T12:17:00.149471+0000","last_became_peered":"2024-03-14T12:17:00.149471+0000","last_unstale":"2024-03-14T12:17:01.146290+0000","last_undegraded":"2024-03-14T12:17:01.146290+0000","last_fullsized":"2024-03-14T12:17:01.146290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:39.201880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197442+0000","last_change":"2024-03-14T12:17:00.160595+0000","last_active":"2024-03-14T12:17:02.197442+0000","last_peered":"2024-03-14T12:17:02.197442+0000","last_clean":"2024-03-14T12:17:02.197442+0000","last_became_active":"2024-03-14T12:17:00.160248+0000","last_became_peered":"2024-03-14T12:17:00.160248+0000","last_unstale":"2024-03-14T12:17:02.197442+0000","last_undegraded":"2024-03-14T12:17:02.197442+0000","last_fullsized":"2024-03-14T12:17:02.197442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196931+0000","last_change":"2024-03-14T12:17:00.161414+0000","last_active":"2024-03-14T12:17:02.196931+0000","last_peered":"2024-03-14T12:17:02.196931+0000","last_clean":"2024-03-14T12:17:02.196931+0000","last_became_active":"2024-03-14T12:17:00.161137+0000","last_became_peered":"2024-03-14T12:17:00.161137+0000","last_unstale":"2024-03-14T12:17:02.196931+0000","last_undegraded":"2024-03-14T12:17:02.196931+0000","last_fullsized":"2024-03-14T12:17:02.196931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:37.868936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159671+0000","last_change":"2024-03-14T12:17:00.157788+0000","last_active":"2024-03-14T12:17:02.159671+0000","last_peered":"2024-03-14T12:17:02.159671+0000","last_clean":"2024-03-14T12:17:02.159671+0000","last_became_active":"2024-03-14T12:17:00.157422+0000","last_became_peered":"2024-03-14T12:17:00.157422+0000","last_unstale":"2024-03-14T12:17:02.159671+0000","last_undegraded":"2024-03-14T12:17:02.159671+0000","last_fullsized":"2024-03-14T12:17:02.159671+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144432+0000","last_change":"2024-03-14T12:17:00.158475+0000","last_active":"2024-03-14T12:17:01.144432+0000","last_peered":"2024-03-14T12:17:01.144432+0000","last_clean":"2024-03-14T12:17:01.144432+0000","last_became_active":"2024-03-14T12:17:00.157438+0000","last_became_peered":"2024-03-14T12:17:00.157438+0000","last_unstale":"2024-03-14T12:17:01.144432+0000","last_undegraded":"2024-03-14T12:17:01.144432+0000","last_fullsized":"2024-03-14T12:17:01.144432+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:35.836879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159452+0000","last_change":"2024-03-14T12:17:00.160877+0000","last_active":"2024-03-14T12:17:02.159452+0000","last_peered":"2024-03-14T12:17:02.159452+0000","last_clean":"2024-03-14T12:17:02.159452+0000","last_became_active":"2024-03-14T12:17:00.160612+0000","last_became_peered":"2024-03-14T12:17:00.160612+0000","last_unstale":"2024-03-14T12:17:02.159452+0000","last_undegraded":"2024-03-14T12:17:02.159452+0000","last_fullsized":"2024-03-14T12:17:02.159452+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147120+0000","last_change":"2024-03-14T12:17:00.156510+0000","last_active":"2024-03-14T12:17:01.147120+0000","last_peered":"2024-03-14T12:17:01.147120+0000","last_clean":"2024-03-14T12:17:01.147120+0000","last_became_active":"2024-03-14T12:17:00.155713+0000","last_became_peered":"2024-03-14T12:17:00.155713+0000","last_unstale":"2024-03-14T12:17:01.147120+0000","last_undegraded":"2024-03-14T12:17:01.147120+0000","last_fullsized":"2024-03-14T12:17:01.147120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:54.810418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144864+0000","last_change":"2024-03-14T12:17:00.154954+0000","last_active":"2024-03-14T12:17:01.144864+0000","last_peered":"2024-03-14T12:17:01.144864+0000","last_clean":"2024-03-14T12:17:01.144864+0000","last_became_active":"2024-03-14T12:17:00.152674+0000","last_became_peered":"2024-03-14T12:17:00.152674+0000","last_unstale":"2024-03-14T12:17:01.144864+0000","last_undegraded":"2024-03-14T12:17:01.144864+0000","last_fullsized":"2024-03-14T12:17:01.144864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:24.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196995+0000","last_change":"2024-03-14T12:17:00.163547+0000","last_active":"2024-03-14T12:17:02.196995+0000","last_peered":"2024-03-14T12:17:02.196995+0000","last_clean":"2024-03-14T12:17:02.196995+0000","last_became_active":"2024-03-14T12:17:00.163294+0000","last_became_peered":"2024-03-14T12:17:00.163294+0000","last_unstale":"2024-03-14T12:17:02.196995+0000","last_undegraded":"2024-03-14T12:17:02.196995+0000","last_fullsized":"2024-03-14T12:17:02.196995+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:48.571300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198420+0000","last_change":"2024-03-14T12:17:00.158303+0000","last_active":"2024-03-14T12:17:02.198420+0000","last_peered":"2024-03-14T12:17:02.198420+0000","last_clean":"2024-03-14T12:17:02.198420+0000","last_became_active":"2024-03-14T12:17:00.158026+0000","last_became_peered":"2024-03-14T12:17:00.158026+0000","last_unstale":"2024-03-14T12:17:02.198420+0000","last_undegraded":"2024-03-14T12:17:02.198420+0000","last_fullsized":"2024-03-14T12:17:02.198420+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:07.658231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159693+0000","last_change":"2024-03-14T12:17:00.163989+0000","last_active":"2024-03-14T12:17:02.159693+0000","last_peered":"2024-03-14T12:17:02.159693+0000","last_clean":"2024-03-14T12:17:02.159693+0000","last_became_active":"2024-03-14T12:17:00.163708+0000","last_became_peered":"2024-03-14T12:17:00.163708+0000","last_unstale":"2024-03-14T12:17:02.159693+0000","last_undegraded":"2024-03-14T12:17:02.159693+0000","last_fullsized":"2024-03-14T12:17:02.159693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:03:50.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146748+0000","last_change":"2024-03-14T12:17:00.154763+0000","last_active":"2024-03-14T12:17:01.146748+0000","last_peered":"2024-03-14T12:17:01.146748+0000","last_clean":"2024-03-14T12:17:01.146748+0000","last_became_active":"2024-03-14T12:17:00.154477+0000","last_became_peered":"2024-03-14T12:17:00.154477+0000","last_unstale":"2024-03-14T12:17:01.146748+0000","last_undegraded":"2024-03-14T12:17:01.146748+0000","last_fullsized":"2024-03-14T12:17:01.146748+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:48.450402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146147+0000","last_change":"2024-03-14T12:17:00.150815+0000","last_active":"2024-03-14T12:17:01.146147+0000","last_peered":"2024-03-14T12:17:01.146147+0000","last_clean":"2024-03-14T12:17:01.146147+0000","last_became_active":"2024-03-14T12:17:00.149442+0000","last_became_peered":"2024-03-14T12:17:00.149442+0000","last_unstale":"2024-03-14T12:17:01.146147+0000","last_undegraded":"2024-03-14T12:17:01.146147+0000","last_fullsized":"2024-03-14T12:17:01.146147+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:05.828371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197316+0000","last_change":"2024-03-14T12:17:00.158417+0000","last_active":"2024-03-14T12:17:02.197316+0000","last_peered":"2024-03-14T12:17:02.197316+0000","last_clean":"2024-03-14T12:17:02.197316+0000","last_became_active":"2024-03-14T12:17:00.158167+0000","last_became_peered":"2024-03-14T12:17:00.158167+0000","last_unstale":"2024-03-14T12:17:02.197316+0000","last_undegraded":"2024-03-14T12:17:02.197316+0000","last_fullsized":"2024-03-14T12:17:02.197316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197213+0000","last_change":"2024-03-14T12:17:00.171366+0000","last_active":"2024-03-14T12:17:02.197213+0000","last_peered":"2024-03-14T12:17:02.197213+0000","last_clean":"2024-03-14T12:17:02.197213+0000","last_became_active":"2024-03-14T12:17:00.170528+0000","last_became_peered":"2024-03-14T12:17:00.170528+0000","last_unstale":"2024-03-14T12:17:02.197213+0000","last_undegraded":"2024-03-14T12:17:02.197213+0000","last_fullsized":"2024-03-14T12:17:02.197213+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145897+0000","last_change":"2024-03-14T12:17:00.164124+0000","last_active":"2024-03-14T12:17:01.145897+0000","last_peered":"2024-03-14T12:17:01.145897+0000","last_clean":"2024-03-14T12:17:01.145897+0000","last_became_active":"2024-03-14T12:17:00.163838+0000","last_became_peered":"2024-03-14T12:17:00.163838+0000","last_unstale":"2024-03-14T12:17:01.145897+0000","last_undegraded":"2024-03-14T12:17:01.145897+0000","last_fullsized":"2024-03-14T12:17:01.145897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:19.263347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146009+0000","last_change":"2024-03-14T12:17:00.169710+0000","last_active":"2024-03-14T12:17:01.146009+0000","last_peered":"2024-03-14T12:17:01.146009+0000","last_clean":"2024-03-14T12:17:01.146009+0000","last_became_active":"2024-03-14T12:17:00.169437+0000","last_became_peered":"2024-03-14T12:17:00.169437+0000","last_unstale":"2024-03-14T12:17:01.146009+0000","last_undegraded":"2024-03-14T12:17:01.146009+0000","last_fullsized":"2024-03-14T12:17:01.146009+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191164+0000","last_change":"2024-03-14T12:17:00.161546+0000","last_active":"2024-03-14T12:17:02.191164+0000","last_peered":"2024-03-14T12:17:02.191164+0000","last_clean":"2024-03-14T12:17:02.191164+0000","last_became_active":"2024-03-14T12:17:00.161174+0000","last_became_peered":"2024-03-14T12:17:00.161174+0000","last_unstale":"2024-03-14T12:17:02.191164+0000","last_undegraded":"2024-03-14T12:17:02.191164+0000","last_fullsized":"2024-03-14T12:17:02.191164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:05.720182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147357+0000","last_change":"2024-03-14T12:17:00.150641+0000","last_active":"2024-03-14T12:17:01.147357+0000","last_peered":"2024-03-14T12:17:01.147357+0000","last_clean":"2024-03-14T12:17:01.147357+0000","last_became_active":"2024-03-14T12:17:00.149567+0000","last_became_peered":"2024-03-14T12:17:00.149567+0000","last_unstale":"2024-03-14T12:17:01.147357+0000","last_undegraded":"2024-03-14T12:17:01.147357+0000","last_fullsized":"2024-03-14T12:17:01.147357+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:25.157808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191238+0000","last_change":"2024-03-14T12:17:00.160024+0000","last_active":"2024-03-14T12:17:02.191238+0000","last_peered":"2024-03-14T12:17:02.191238+0000","last_clean":"2024-03-14T12:17:02.191238+0000","last_became_active":"2024-03-14T12:17:00.159513+0000","last_became_peered":"2024-03-14T12:17:00.159513+0000","last_unstale":"2024-03-14T12:17:02.191238+0000","last_undegraded":"2024-03-14T12:17:02.191238+0000","last_fullsized":"2024-03-14T12:17:02.191238+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:02.639220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145867+0000","last_change":"2024-03-14T12:17:00.171649+0000","last_active":"2024-03-14T12:17:01.145867+0000","last_peered":"2024-03-14T12:17:01.145867+0000","last_clean":"2024-03-14T12:17:01.145867+0000","last_became_active":"2024-03-14T12:17:00.171384+0000","last_became_peered":"2024-03-14T12:17:00.171384+0000","last_unstale":"2024-03-14T12:17:01.145867+0000","last_undegraded":"2024-03-14T12:17:01.145867+0000","last_fullsized":"2024-03-14T12:17:01.145867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:50:38.677924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146359+0000","last_change":"2024-03-14T12:17:00.164315+0000","last_active":"2024-03-14T12:17:01.146359+0000","last_peered":"2024-03-14T12:17:01.146359+0000","last_clean":"2024-03-14T12:17:01.146359+0000","last_became_active":"2024-03-14T12:17:00.164057+0000","last_became_peered":"2024-03-14T12:17:00.164057+0000","last_unstale":"2024-03-14T12:17:01.146359+0000","last_undegraded":"2024-03-14T12:17:01.146359+0000","last_fullsized":"2024-03-14T12:17:01.146359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198081+0000","last_change":"2024-03-14T12:17:00.156339+0000","last_active":"2024-03-14T12:17:02.198081+0000","last_peered":"2024-03-14T12:17:02.198081+0000","last_clean":"2024-03-14T12:17:02.198081+0000","last_became_active":"2024-03-14T12:17:00.156068+0000","last_became_peered":"2024-03-14T12:17:00.156068+0000","last_unstale":"2024-03-14T12:17:02.198081+0000","last_undegraded":"2024-03-14T12:17:02.198081+0000","last_fullsized":"2024-03-14T12:17:02.198081+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:05.828371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197795+0000","last_change":"2024-03-14T12:17:00.164288+0000","last_active":"2024-03-14T12:17:02.197795+0000","last_peered":"2024-03-14T12:17:02.197795+0000","last_clean":"2024-03-14T12:17:02.197795+0000","last_became_active":"2024-03-14T12:17:00.163969+0000","last_became_peered":"2024-03-14T12:17:00.163969+0000","last_unstale":"2024-03-14T12:17:02.197795+0000","last_undegraded":"2024-03-14T12:17:02.197795+0000","last_fullsized":"2024-03-14T12:17:02.197795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:10:36.197658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197849+0000","last_change":"2024-03-14T12:16:59.160707+0000","last_active":"2024-03-14T12:17:02.197849+0000","last_peered":"2024-03-14T12:17:02.197849+0000","last_clean":"2024-03-14T12:17:02.197849+0000","last_became_active":"2024-03-14T12:16:59.160442+0000","last_became_peered":"2024-03-14T12:16:59.160442+0000","last_unstale":"2024-03-14T12:17:02.197849+0000","last_undegraded":"2024-03-14T12:17:02.197849+0000","last_fullsized":"2024-03-14T12:17:02.197849+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:06.652925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144948+0000","last_change":"2024-03-14T12:17:00.152459+0000","last_active":"2024-03-14T12:17:01.144948+0000","last_peered":"2024-03-14T12:17:01.144948+0000","last_clean":"2024-03-14T12:17:01.144948+0000","last_became_active":"2024-03-14T12:17:00.152163+0000","last_became_peered":"2024-03-14T12:17:00.152163+0000","last_unstale":"2024-03-14T12:17:01.144948+0000","last_undegraded":"2024-03-14T12:17:01.144948+0000","last_fullsized":"2024-03-14T12:17:01.144948+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:52.251106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159748+0000","last_change":"2024-03-14T12:16:59.160021+0000","last_active":"2024-03-14T12:17:02.159748+0000","last_peered":"2024-03-14T12:17:02.159748+0000","last_clean":"2024-03-14T12:17:02.159748+0000","last_became_active":"2024-03-14T12:16:59.159736+0000","last_became_peered":"2024-03-14T12:16:59.159736+0000","last_unstale":"2024-03-14T12:17:02.159748+0000","last_undegraded":"2024-03-14T12:17:02.159748+0000","last_fullsized":"2024-03-14T12:17:02.159748+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:53.805112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198429+0000","last_change":"2024-03-14T12:17:00.162771+0000","last_active":"2024-03-14T12:17:02.198429+0000","last_peered":"2024-03-14T12:17:02.198429+0000","last_clean":"2024-03-14T12:17:02.198429+0000","last_became_active":"2024-03-14T12:17:00.162509+0000","last_became_peered":"2024-03-14T12:17:00.162509+0000","last_unstale":"2024-03-14T12:17:02.198429+0000","last_undegraded":"2024-03-14T12:17:02.198429+0000","last_fullsized":"2024-03-14T12:17:02.198429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:48.450402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146397+0000","last_change":"2024-03-14T12:16:59.150884+0000","last_active":"2024-03-14T12:17:01.146397+0000","last_peered":"2024-03-14T12:17:01.146397+0000","last_clean":"2024-03-14T12:17:01.146397+0000","last_became_active":"2024-03-14T12:16:59.150534+0000","last_became_peered":"2024-03-14T12:16:59.150534+0000","last_unstale":"2024-03-14T12:17:01.146397+0000","last_undegraded":"2024-03-14T12:17:01.146397+0000","last_fullsized":"2024-03-14T12:17:01.146397+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:39.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197636+0000","last_change":"2024-03-14T12:17:00.161330+0000","last_active":"2024-03-14T12:17:02.197636+0000","last_peered":"2024-03-14T12:17:02.197636+0000","last_clean":"2024-03-14T12:17:02.197636+0000","last_became_active":"2024-03-14T12:17:00.161060+0000","last_became_peered":"2024-03-14T12:17:00.161060+0000","last_unstale":"2024-03-14T12:17:02.197636+0000","last_undegraded":"2024-03-14T12:17:02.197636+0000","last_fullsized":"2024-03-14T12:17:02.197636+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:53:48.285207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198508+0000","last_change":"2024-03-14T12:16:59.160120+0000","last_active":"2024-03-14T12:17:02.198508+0000","last_peered":"2024-03-14T12:17:02.198508+0000","last_clean":"2024-03-14T12:17:02.198508+0000","last_became_active":"2024-03-14T12:16:59.159810+0000","last_became_peered":"2024-03-14T12:16:59.159810+0000","last_unstale":"2024-03-14T12:17:02.198508+0000","last_undegraded":"2024-03-14T12:17:02.198508+0000","last_fullsized":"2024-03-14T12:17:02.198508+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:23.661196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145933+0000","last_change":"2024-03-14T12:17:00.160986+0000","last_active":"2024-03-14T12:17:01.145933+0000","last_peered":"2024-03-14T12:17:01.145933+0000","last_clean":"2024-03-14T12:17:01.145933+0000","last_became_active":"2024-03-14T12:17:00.160699+0000","last_became_peered":"2024-03-14T12:17:00.160699+0000","last_unstale":"2024-03-14T12:17:01.145933+0000","last_undegraded":"2024-03-14T12:17:01.145933+0000","last_fullsized":"2024-03-14T12:17:01.145933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:04:30.945677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190985+0000","last_change":"2024-03-14T12:16:59.158332+0000","last_active":"2024-03-14T12:17:02.190985+0000","last_peered":"2024-03-14T12:17:02.190985+0000","last_clean":"2024-03-14T12:17:02.190985+0000","last_became_active":"2024-03-14T12:16:59.158049+0000","last_became_peered":"2024-03-14T12:16:59.158049+0000","last_unstale":"2024-03-14T12:17:02.190985+0000","last_undegraded":"2024-03-14T12:17:02.190985+0000","last_fullsized":"2024-03-14T12:17:02.190985+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:16:10.934601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146888+0000","last_change":"2024-03-14T12:17:00.159212+0000","last_active":"2024-03-14T12:17:01.146888+0000","last_peered":"2024-03-14T12:17:01.146888+0000","last_clean":"2024-03-14T12:17:01.146888+0000","last_became_active":"2024-03-14T12:17:00.158951+0000","last_became_peered":"2024-03-14T12:17:00.158951+0000","last_unstale":"2024-03-14T12:17:01.146888+0000","last_undegraded":"2024-03-14T12:17:01.146888+0000","last_fullsized":"2024-03-14T12:17:01.146888+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:03.354492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197981+0000","last_change":"2024-03-14T12:16:59.158539+0000","last_active":"2024-03-14T12:17:02.197981+0000","last_peered":"2024-03-14T12:17:02.197981+0000","last_clean":"2024-03-14T12:17:02.197981+0000","last_became_active":"2024-03-14T12:16:59.158223+0000","last_became_peered":"2024-03-14T12:16:59.158223+0000","last_unstale":"2024-03-14T12:17:02.197981+0000","last_undegraded":"2024-03-14T12:17:02.197981+0000","last_fullsized":"2024-03-14T12:17:02.197981+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:19:21.430991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144255+0000","last_change":"2024-03-14T12:17:00.146758+0000","last_active":"2024-03-14T12:17:01.144255+0000","last_peered":"2024-03-14T12:17:01.144255+0000","last_clean":"2024-03-14T12:17:01.144255+0000","last_became_active":"2024-03-14T12:17:00.146485+0000","last_became_peered":"2024-03-14T12:17:00.146485+0000","last_unstale":"2024-03-14T12:17:01.144255+0000","last_undegraded":"2024-03-14T12:17:01.144255+0000","last_fullsized":"2024-03-14T12:17:01.144255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:57:23.282105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196659+0000","last_change":"2024-03-14T12:16:59.153534+0000","last_active":"2024-03-14T12:17:02.196659+0000","last_peered":"2024-03-14T12:17:02.196659+0000","last_clean":"2024-03-14T12:17:02.196659+0000","last_became_active":"2024-03-14T12:16:59.153249+0000","last_became_peered":"2024-03-14T12:16:59.153249+0000","last_unstale":"2024-03-14T12:17:02.196659+0000","last_undegraded":"2024-03-14T12:17:02.196659+0000","last_fullsized":"2024-03-14T12:17:02.196659+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:57:22.276798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159422+0000","last_change":"2024-03-14T12:17:00.161414+0000","last_active":"2024-03-14T12:17:02.159422+0000","last_peered":"2024-03-14T12:17:02.159422+0000","last_clean":"2024-03-14T12:17:02.159422+0000","last_became_active":"2024-03-14T12:17:00.161178+0000","last_became_peered":"2024-03-14T12:17:00.161178+0000","last_unstale":"2024-03-14T12:17:02.159422+0000","last_undegraded":"2024-03-14T12:17:02.159422+0000","last_fullsized":"2024-03-14T12:17:02.159422+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:05:10.595088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144186+0000","last_change":"2024-03-14T12:16:59.156881+0000","last_active":"2024-03-14T12:17:01.144186+0000","last_peered":"2024-03-14T12:17:01.144186+0000","last_clean":"2024-03-14T12:17:01.144186+0000","last_became_active":"2024-03-14T12:16:59.156609+0000","last_became_peered":"2024-03-14T12:16:59.156609+0000","last_unstale":"2024-03-14T12:17:01.144186+0000","last_undegraded":"2024-03-14T12:17:01.144186+0000","last_fullsized":"2024-03-14T12:17:01.144186+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:14:57.357163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146160+0000","last_change":"2024-03-14T12:17:00.163114+0000","last_active":"2024-03-14T12:17:01.146160+0000","last_peered":"2024-03-14T12:17:01.146160+0000","last_clean":"2024-03-14T12:17:01.146160+0000","last_became_active":"2024-03-14T12:17:00.162835+0000","last_became_peered":"2024-03-14T12:17:00.162835+0000","last_unstale":"2024-03-14T12:17:01.146160+0000","last_undegraded":"2024-03-14T12:17:01.146160+0000","last_fullsized":"2024-03-14T12:17:01.146160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145963+0000","last_change":"2024-03-14T12:16:59.157422+0000","last_active":"2024-03-14T12:17:01.145963+0000","last_peered":"2024-03-14T12:17:01.145963+0000","last_clean":"2024-03-14T12:17:01.145963+0000","last_became_active":"2024-03-14T12:16:59.155980+0000","last_became_peered":"2024-03-14T12:16:59.155980+0000","last_unstale":"2024-03-14T12:17:01.145963+0000","last_undegraded":"2024-03-14T12:17:01.145963+0000","last_fullsized":"2024-03-14T12:17:01.145963+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:46.803639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197812+0000","last_change":"2024-03-14T12:17:00.167346+0000","last_active":"2024-03-14T12:17:02.197812+0000","last_peered":"2024-03-14T12:17:02.197812+0000","last_clean":"2024-03-14T12:17:02.197812+0000","last_became_active":"2024-03-14T12:17:00.167075+0000","last_became_peered":"2024-03-14T12:17:00.167075+0000","last_unstale":"2024-03-14T12:17:02.197812+0000","last_undegraded":"2024-03-14T12:17:02.197812+0000","last_fullsized":"2024-03-14T12:17:02.197812+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:36.504209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198643+0000","last_change":"2024-03-14T12:16:59.154772+0000","last_active":"2024-03-14T12:17:02.198643+0000","last_peered":"2024-03-14T12:17:02.198643+0000","last_clean":"2024-03-14T12:17:02.198643+0000","last_became_active":"2024-03-14T12:16:59.154363+0000","last_became_peered":"2024-03-14T12:16:59.154363+0000","last_unstale":"2024-03-14T12:17:02.198643+0000","last_undegraded":"2024-03-14T12:17:02.198643+0000","last_fullsized":"2024-03-14T12:17:02.198643+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:46.803639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198434+0000","last_change":"2024-03-14T12:17:00.156703+0000","last_active":"2024-03-14T12:17:02.198434+0000","last_peered":"2024-03-14T12:17:02.198434+0000","last_clean":"2024-03-14T12:17:02.198434+0000","last_became_active":"2024-03-14T12:17:00.156256+0000","last_became_peered":"2024-03-14T12:17:00.156256+0000","last_unstale":"2024-03-14T12:17:02.198434+0000","last_undegraded":"2024-03-14T12:17:02.198434+0000","last_fullsized":"2024-03-14T12:17:02.198434+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:03.354492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191221+0000","last_change":"2024-03-14T12:16:59.160192+0000","last_active":"2024-03-14T12:17:02.191221+0000","last_peered":"2024-03-14T12:17:02.191221+0000","last_clean":"2024-03-14T12:17:02.191221+0000","last_became_active":"2024-03-14T12:16:59.159901+0000","last_became_peered":"2024-03-14T12:16:59.159901+0000","last_unstale":"2024-03-14T12:17:02.191221+0000","last_undegraded":"2024-03-14T12:17:02.191221+0000","last_fullsized":"2024-03-14T12:17:02.191221+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:28:07.504769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144649+0000","last_change":"2024-03-14T12:17:00.150822+0000","last_active":"2024-03-14T12:17:01.144649+0000","last_peered":"2024-03-14T12:17:01.144649+0000","last_clean":"2024-03-14T12:17:01.144649+0000","last_became_active":"2024-03-14T12:17:00.150463+0000","last_became_peered":"2024-03-14T12:17:00.150463+0000","last_unstale":"2024-03-14T12:17:01.144649+0000","last_undegraded":"2024-03-14T12:17:01.144649+0000","last_fullsized":"2024-03-14T12:17:01.144649+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:39.380311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197081+0000","last_change":"2024-03-14T12:16:59.147830+0000","last_active":"2024-03-14T12:17:02.197081+0000","last_peered":"2024-03-14T12:17:02.197081+0000","last_clean":"2024-03-14T12:17:02.197081+0000","last_became_active":"2024-03-14T12:16:59.147503+0000","last_became_peered":"2024-03-14T12:16:59.147503+0000","last_unstale":"2024-03-14T12:17:02.197081+0000","last_undegraded":"2024-03-14T12:17:02.197081+0000","last_fullsized":"2024-03-14T12:17:02.197081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:48.936304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197326+0000","last_change":"2024-03-14T12:17:00.161960+0000","last_active":"2024-03-14T12:17:02.197326+0000","last_peered":"2024-03-14T12:17:02.197326+0000","last_clean":"2024-03-14T12:17:02.197326+0000","last_became_active":"2024-03-14T12:17:00.161650+0000","last_became_peered":"2024-03-14T12:17:00.161650+0000","last_unstale":"2024-03-14T12:17:02.197326+0000","last_undegraded":"2024-03-14T12:17:02.197326+0000","last_fullsized":"2024-03-14T12:17:02.197326+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196793+0000","last_change":"2024-03-14T12:16:59.156977+0000","last_active":"2024-03-14T12:17:02.196793+0000","last_peered":"2024-03-14T12:17:02.196793+0000","last_clean":"2024-03-14T12:17:02.196793+0000","last_became_active":"2024-03-14T12:16:59.156715+0000","last_became_peered":"2024-03-14T12:16:59.156715+0000","last_unstale":"2024-03-14T12:17:02.196793+0000","last_undegraded":"2024-03-14T12:17:02.196793+0000","last_fullsized":"2024-03-14T12:17:02.196793+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:34.831573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145991+0000","last_change":"2024-03-14T12:17:00.170808+0000","last_active":"2024-03-14T12:17:01.145991+0000","last_peered":"2024-03-14T12:17:01.145991+0000","last_clean":"2024-03-14T12:17:01.145991+0000","last_became_active":"2024-03-14T12:17:00.170529+0000","last_became_peered":"2024-03-14T12:17:00.170529+0000","last_unstale":"2024-03-14T12:17:01.145991+0000","last_undegraded":"2024-03-14T12:17:01.145991+0000","last_fullsized":"2024-03-14T12:17:01.145991+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:05:10.595088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198068+0000","last_change":"2024-03-14T12:16:59.162979+0000","last_active":"2024-03-14T12:17:02.198068+0000","last_peered":"2024-03-14T12:17:02.198068+0000","last_clean":"2024-03-14T12:17:02.198068+0000","last_became_active":"2024-03-14T12:16:59.162698+0000","last_became_peered":"2024-03-14T12:16:59.162698+0000","last_unstale":"2024-03-14T12:17:02.198068+0000","last_undegraded":"2024-03-14T12:17:02.198068+0000","last_fullsized":"2024-03-14T12:17:02.198068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:51.245799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159543+0000","last_change":"2024-03-14T12:17:00.157109+0000","last_active":"2024-03-14T12:17:02.159543+0000","last_peered":"2024-03-14T12:17:02.159543+0000","last_clean":"2024-03-14T12:17:02.159543+0000","last_became_active":"2024-03-14T12:17:00.156852+0000","last_became_peered":"2024-03-14T12:17:00.156852+0000","last_unstale":"2024-03-14T12:17:02.159543+0000","last_undegraded":"2024-03-14T12:17:02.159543+0000","last_fullsized":"2024-03-14T12:17:02.159543+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146075+0000","last_change":"2024-03-14T12:16:59.157658+0000","last_active":"2024-03-14T12:17:01.146075+0000","last_peered":"2024-03-14T12:17:01.146075+0000","last_clean":"2024-03-14T12:17:01.146075+0000","last_became_active":"2024-03-14T12:16:59.156521+0000","last_became_peered":"2024-03-14T12:16:59.156521+0000","last_unstale":"2024-03-14T12:17:01.146075+0000","last_undegraded":"2024-03-14T12:17:01.146075+0000","last_fullsized":"2024-03-14T12:17:01.146075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:18.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144334+0000","last_change":"2024-03-14T12:17:00.152560+0000","last_active":"2024-03-14T12:17:01.144334+0000","last_peered":"2024-03-14T12:17:01.144334+0000","last_clean":"2024-03-14T12:17:01.144334+0000","last_became_active":"2024-03-14T12:17:00.152302+0000","last_became_peered":"2024-03-14T12:17:00.152302+0000","last_unstale":"2024-03-14T12:17:01.144334+0000","last_undegraded":"2024-03-14T12:17:01.144334+0000","last_fullsized":"2024-03-14T12:17:01.144334+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:46.281680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147065+0000","last_change":"2024-03-14T12:16:59.157970+0000","last_active":"2024-03-14T12:17:01.147065+0000","last_peered":"2024-03-14T12:17:01.147065+0000","last_clean":"2024-03-14T12:17:01.147065+0000","last_became_active":"2024-03-14T12:16:59.157694+0000","last_became_peered":"2024-03-14T12:16:59.157694+0000","last_unstale":"2024-03-14T12:17:01.147065+0000","last_undegraded":"2024-03-14T12:17:01.147065+0000","last_fullsized":"2024-03-14T12:17:01.147065+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:47.565994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146444+0000","last_change":"2024-03-14T12:17:00.151385+0000","last_active":"2024-03-14T12:17:01.146444+0000","last_peered":"2024-03-14T12:17:01.146444+0000","last_clean":"2024-03-14T12:17:01.146444+0000","last_became_active":"2024-03-14T12:17:00.151140+0000","last_became_peered":"2024-03-14T12:17:00.151140+0000","last_unstale":"2024-03-14T12:17:01.146444+0000","last_undegraded":"2024-03-14T12:17:01.146444+0000","last_fullsized":"2024-03-14T12:17:01.146444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:07.658231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198497+0000","last_change":"2024-03-14T12:16:59.161389+0000","last_active":"2024-03-14T12:17:02.198497+0000","last_peered":"2024-03-14T12:17:02.198497+0000","last_clean":"2024-03-14T12:17:02.198497+0000","last_became_active":"2024-03-14T12:16:59.161130+0000","last_became_peered":"2024-03-14T12:16:59.161130+0000","last_unstale":"2024-03-14T12:17:02.198497+0000","last_undegraded":"2024-03-14T12:17:02.198497+0000","last_fullsized":"2024-03-14T12:17:02.198497+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:36.863630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197907+0000","last_change":"2024-03-14T12:17:00.168001+0000","last_active":"2024-03-14T12:17:02.197907+0000","last_peered":"2024-03-14T12:17:02.197907+0000","last_clean":"2024-03-14T12:17:02.197907+0000","last_became_active":"2024-03-14T12:17:00.167735+0000","last_became_peered":"2024-03-14T12:17:00.167735+0000","last_unstale":"2024-03-14T12:17:02.197907+0000","last_undegraded":"2024-03-14T12:17:02.197907+0000","last_fullsized":"2024-03-14T12:17:02.197907+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:16:11.939907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146483+0000","last_change":"2024-03-14T12:16:59.153745+0000","last_active":"2024-03-14T12:17:01.146483+0000","last_peered":"2024-03-14T12:17:01.146483+0000","last_clean":"2024-03-14T12:17:01.146483+0000","last_became_active":"2024-03-14T12:16:59.153456+0000","last_became_peered":"2024-03-14T12:16:59.153456+0000","last_unstale":"2024-03-14T12:17:01.146483+0000","last_undegraded":"2024-03-14T12:17:01.146483+0000","last_fullsized":"2024-03-14T12:17:01.146483+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:36:23.425887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197129+0000","last_change":"2024-03-14T12:17:00.161390+0000","last_active":"2024-03-14T12:17:02.197129+0000","last_peered":"2024-03-14T12:17:02.197129+0000","last_clean":"2024-03-14T12:17:02.197129+0000","last_became_active":"2024-03-14T12:17:00.161087+0000","last_became_peered":"2024-03-14T12:17:00.161087+0000","last_unstale":"2024-03-14T12:17:02.197129+0000","last_undegraded":"2024-03-14T12:17:02.197129+0000","last_fullsized":"2024-03-14T12:17:02.197129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:40.293976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198465+0000","last_change":"2024-03-14T12:16:59.151077+0000","last_active":"2024-03-14T12:17:02.198465+0000","last_peered":"2024-03-14T12:17:02.198465+0000","last_clean":"2024-03-14T12:17:02.198465+0000","last_became_active":"2024-03-14T12:16:59.150728+0000","last_became_peered":"2024-03-14T12:16:59.150728+0000","last_unstale":"2024-03-14T12:17:02.198465+0000","last_undegraded":"2024-03-14T12:17:02.198465+0000","last_fullsized":"2024-03-14T12:17:02.198465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:44.730420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144710+0000","last_change":"2024-03-14T12:17:00.148497+0000","last_active":"2024-03-14T12:17:01.144710+0000","last_peered":"2024-03-14T12:17:01.144710+0000","last_clean":"2024-03-14T12:17:01.144710+0000","last_became_active":"2024-03-14T12:17:00.148125+0000","last_became_peered":"2024-03-14T12:17:00.148125+0000","last_unstale":"2024-03-14T12:17:01.144710+0000","last_undegraded":"2024-03-14T12:17:01.144710+0000","last_fullsized":"2024-03-14T12:17:01.144710+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:19:22.436297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197406+0000","last_change":"2024-03-14T12:16:59.160922+0000","last_active":"2024-03-14T12:17:02.197406+0000","last_peered":"2024-03-14T12:17:02.197406+0000","last_clean":"2024-03-14T12:17:02.197406+0000","last_became_active":"2024-03-14T12:16:59.160607+0000","last_became_peered":"2024-03-14T12:16:59.160607+0000","last_unstale":"2024-03-14T12:17:02.197406+0000","last_undegraded":"2024-03-14T12:17:02.197406+0000","last_fullsized":"2024-03-14T12:17:02.197406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:38.196573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190932+0000","last_change":"2024-03-14T12:17:00.156143+0000","last_active":"2024-03-14T12:17:02.190932+0000","last_peered":"2024-03-14T12:17:02.190932+0000","last_clean":"2024-03-14T12:17:02.190932+0000","last_became_active":"2024-03-14T12:17:00.155844+0000","last_became_peered":"2024-03-14T12:17:00.155844+0000","last_unstale":"2024-03-14T12:17:02.190932+0000","last_undegraded":"2024-03-14T12:17:02.190932+0000","last_fullsized":"2024-03-14T12:17:02.190932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:49:56.360025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198120+0000","last_change":"2024-03-14T12:16:59.158103+0000","last_active":"2024-03-14T12:17:02.198120+0000","last_peered":"2024-03-14T12:17:02.198120+0000","last_clean":"2024-03-14T12:17:02.198120+0000","last_became_active":"2024-03-14T12:16:59.157770+0000","last_became_peered":"2024-03-14T12:16:59.157770+0000","last_unstale":"2024-03-14T12:17:02.198120+0000","last_undegraded":"2024-03-14T12:17:02.198120+0000","last_fullsized":"2024-03-14T12:17:02.198120+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:49.373989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190827+0000","last_change":"2024-03-14T12:17:00.159001+0000","last_active":"2024-03-14T12:17:02.190827+0000","last_peered":"2024-03-14T12:17:02.190827+0000","last_clean":"2024-03-14T12:17:02.190827+0000","last_became_active":"2024-03-14T12:17:00.158720+0000","last_became_peered":"2024-03-14T12:17:00.158720+0000","last_unstale":"2024-03-14T12:17:02.190827+0000","last_undegraded":"2024-03-14T12:17:02.190827+0000","last_fullsized":"2024-03-14T12:17:02.190827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:00:30.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146166+0000","last_change":"2024-03-14T12:16:59.149014+0000","last_active":"2024-03-14T12:17:01.146166+0000","last_peered":"2024-03-14T12:17:01.146166+0000","last_clean":"2024-03-14T12:17:01.146166+0000","last_became_active":"2024-03-14T12:16:59.148708+0000","last_became_peered":"2024-03-14T12:16:59.148708+0000","last_unstale":"2024-03-14T12:17:01.146166+0000","last_undegraded":"2024-03-14T12:17:01.146166+0000","last_fullsized":"2024-03-14T12:17:01.146166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:04.823064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190842+0000","last_change":"2024-03-14T12:17:00.158300+0000","last_active":"2024-03-14T12:17:02.190842+0000","last_peered":"2024-03-14T12:17:02.190842+0000","last_clean":"2024-03-14T12:17:02.190842+0000","last_became_active":"2024-03-14T12:17:00.158049+0000","last_became_peered":"2024-03-14T12:17:00.158049+0000","last_unstale":"2024-03-14T12:17:02.190842+0000","last_undegraded":"2024-03-14T12:17:02.190842+0000","last_fullsized":"2024-03-14T12:17:02.190842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:33:51.561745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159206+0000","last_change":"2024-03-14T12:16:59.163118+0000","last_active":"2024-03-14T12:17:02.159206+0000","last_peered":"2024-03-14T12:17:02.159206+0000","last_clean":"2024-03-14T12:17:02.159206+0000","last_became_active":"2024-03-14T12:16:59.162808+0000","last_became_peered":"2024-03-14T12:16:59.162808+0000","last_unstale":"2024-03-14T12:17:02.159206+0000","last_undegraded":"2024-03-14T12:17:02.159206+0000","last_fullsized":"2024-03-14T12:17:02.159206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:02.349185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159686+0000","last_change":"2024-03-14T12:17:00.161657+0000","last_active":"2024-03-14T12:17:02.159686+0000","last_peered":"2024-03-14T12:17:02.159686+0000","last_clean":"2024-03-14T12:17:02.159686+0000","last_became_active":"2024-03-14T12:17:00.160635+0000","last_became_peered":"2024-03-14T12:17:00.160635+0000","last_unstale":"2024-03-14T12:17:02.159686+0000","last_undegraded":"2024-03-14T12:17:02.159686+0000","last_fullsized":"2024-03-14T12:17:02.159686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198729+0000","last_change":"2024-03-14T12:16:59.160742+0000","last_active":"2024-03-14T12:17:02.198729+0000","last_peered":"2024-03-14T12:17:02.198729+0000","last_clean":"2024-03-14T12:17:02.198729+0000","last_became_active":"2024-03-14T12:16:59.160452+0000","last_became_peered":"2024-03-14T12:16:59.160452+0000","last_unstale":"2024-03-14T12:17:02.198729+0000","last_undegraded":"2024-03-14T12:17:02.198729+0000","last_fullsized":"2024-03-14T12:17:02.198729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:47.134831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146041+0000","last_change":"2024-03-14T12:17:00.161317+0000","last_active":"2024-03-14T12:17:01.146041+0000","last_peered":"2024-03-14T12:17:01.146041+0000","last_clean":"2024-03-14T12:17:01.146041+0000","last_became_active":"2024-03-14T12:17:00.161046+0000","last_became_peered":"2024-03-14T12:17:00.161046+0000","last_unstale":"2024-03-14T12:17:01.146041+0000","last_undegraded":"2024-03-14T12:17:01.146041+0000","last_fullsized":"2024-03-14T12:17:01.146041+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146258+0000","last_change":"2024-03-14T12:16:59.157167+0000","last_active":"2024-03-14T12:17:01.146258+0000","last_peered":"2024-03-14T12:17:01.146258+0000","last_clean":"2024-03-14T12:17:01.146258+0000","last_became_active":"2024-03-14T12:16:59.156847+0000","last_became_peered":"2024-03-14T12:16:59.156847+0000","last_unstale":"2024-03-14T12:17:01.146258+0000","last_undegraded":"2024-03-14T12:17:01.146258+0000","last_fullsized":"2024-03-14T12:17:01.146258+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:03:03.012491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197493+0000","last_change":"2024-03-14T12:17:00.168815+0000","last_active":"2024-03-14T12:17:02.197493+0000","last_peered":"2024-03-14T12:17:02.197493+0000","last_clean":"2024-03-14T12:17:02.197493+0000","last_became_active":"2024-03-14T12:17:00.168561+0000","last_became_peered":"2024-03-14T12:17:00.168561+0000","last_unstale":"2024-03-14T12:17:02.197493+0000","last_undegraded":"2024-03-14T12:17:02.197493+0000","last_fullsized":"2024-03-14T12:17:02.197493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:03:50.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144705+0000","last_change":"2024-03-14T12:16:59.157608+0000","last_active":"2024-03-14T12:17:01.144705+0000","last_peered":"2024-03-14T12:17:01.144705+0000","last_clean":"2024-03-14T12:17:01.144705+0000","last_became_active":"2024-03-14T12:16:59.157339+0000","last_became_peered":"2024-03-14T12:16:59.157339+0000","last_unstale":"2024-03-14T12:17:01.144705+0000","last_undegraded":"2024-03-14T12:17:01.144705+0000","last_fullsized":"2024-03-14T12:17:01.144705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:28.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197678+0000","last_change":"2024-03-14T12:17:00.163234+0000","last_active":"2024-03-14T12:17:02.197678+0000","last_peered":"2024-03-14T12:17:02.197678+0000","last_clean":"2024-03-14T12:17:02.197678+0000","last_became_active":"2024-03-14T12:17:00.162922+0000","last_became_peered":"2024-03-14T12:17:00.162922+0000","last_unstale":"2024-03-14T12:17:02.197678+0000","last_undegraded":"2024-03-14T12:17:02.197678+0000","last_fullsized":"2024-03-14T12:17:02.197678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:54:53.421821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159555+0000","last_change":"2024-03-14T12:16:59.161879+0000","last_active":"2024-03-14T12:17:02.159555+0000","last_peered":"2024-03-14T12:17:02.159555+0000","last_clean":"2024-03-14T12:17:02.159555+0000","last_became_active":"2024-03-14T12:16:59.161612+0000","last_became_peered":"2024-03-14T12:16:59.161612+0000","last_unstale":"2024-03-14T12:17:02.159555+0000","last_undegraded":"2024-03-14T12:17:02.159555+0000","last_fullsized":"2024-03-14T12:17:02.159555+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:24.152502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144466+0000","last_change":"2024-03-14T12:17:00.152780+0000","last_active":"2024-03-14T12:17:01.144466+0000","last_peered":"2024-03-14T12:17:01.144466+0000","last_clean":"2024-03-14T12:17:01.144466+0000","last_became_active":"2024-03-14T12:17:00.152426+0000","last_became_peered":"2024-03-14T12:17:00.152426+0000","last_unstale":"2024-03-14T12:17:01.144466+0000","last_undegraded":"2024-03-14T12:17:01.144466+0000","last_fullsized":"2024-03-14T12:17:01.144466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:49.941611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144339+0000","last_change":"2024-03-14T12:16:59.150881+0000","last_active":"2024-03-14T12:17:01.144339+0000","last_peered":"2024-03-14T12:17:01.144339+0000","last_clean":"2024-03-14T12:17:01.144339+0000","last_became_active":"2024-03-14T12:16:59.150578+0000","last_became_peered":"2024-03-14T12:16:59.150578+0000","last_unstale":"2024-03-14T12:17:01.144339+0000","last_undegraded":"2024-03-14T12:17:01.144339+0000","last_fullsized":"2024-03-14T12:17:01.144339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:02.510738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198277+0000","last_change":"2024-03-14T12:17:00.159754+0000","last_active":"2024-03-14T12:17:02.198277+0000","last_peered":"2024-03-14T12:17:02.198277+0000","last_clean":"2024-03-14T12:17:02.198277+0000","last_became_active":"2024-03-14T12:17:00.158664+0000","last_became_peered":"2024-03-14T12:17:00.158664+0000","last_unstale":"2024-03-14T12:17:02.198277+0000","last_undegraded":"2024-03-14T12:17:02.198277+0000","last_fullsized":"2024-03-14T12:17:02.198277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:39.201880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196679+0000","last_change":"2024-03-14T12:16:59.153605+0000","last_active":"2024-03-14T12:17:02.196679+0000","last_peered":"2024-03-14T12:17:02.196679+0000","last_clean":"2024-03-14T12:17:02.196679+0000","last_became_active":"2024-03-14T12:16:59.153330+0000","last_became_peered":"2024-03-14T12:16:59.153330+0000","last_unstale":"2024-03-14T12:17:02.196679+0000","last_undegraded":"2024-03-14T12:17:02.196679+0000","last_fullsized":"2024-03-14T12:17:02.196679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:45.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196750+0000","last_change":"2024-03-14T12:17:00.154881+0000","last_active":"2024-03-14T12:17:02.196750+0000","last_peered":"2024-03-14T12:17:02.196750+0000","last_clean":"2024-03-14T12:17:02.196750+0000","last_became_active":"2024-03-14T12:17:00.154597+0000","last_became_peered":"2024-03-14T12:17:00.154597+0000","last_unstale":"2024-03-14T12:17:02.196750+0000","last_undegraded":"2024-03-14T12:17:02.196750+0000","last_fullsized":"2024-03-14T12:17:02.196750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:45.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145806+0000","last_change":"2024-03-14T12:16:59.158773+0000","last_active":"2024-03-14T12:17:01.145806+0000","last_peered":"2024-03-14T12:17:01.145806+0000","last_clean":"2024-03-14T12:17:01.145806+0000","last_became_active":"2024-03-14T12:16:59.158497+0000","last_became_peered":"2024-03-14T12:16:59.158497+0000","last_unstale":"2024-03-14T12:17:01.145806+0000","last_undegraded":"2024-03-14T12:17:01.145806+0000","last_fullsized":"2024-03-14T12:17:01.145806+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:07:36.819982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197762+0000","last_change":"2024-03-14T12:17:00.155232+0000","last_active":"2024-03-14T12:17:02.197762+0000","last_peered":"2024-03-14T12:17:02.197762+0000","last_clean":"2024-03-14T12:17:02.197762+0000","last_became_active":"2024-03-14T12:17:00.154850+0000","last_became_peered":"2024-03-14T12:17:00.154850+0000","last_unstale":"2024-03-14T12:17:02.197762+0000","last_undegraded":"2024-03-14T12:17:02.197762+0000","last_fullsized":"2024-03-14T12:17:02.197762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:24.931278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198093+0000","last_change":"2024-03-14T12:16:59.159327+0000","last_active":"2024-03-14T12:17:02.198093+0000","last_peered":"2024-03-14T12:17:02.198093+0000","last_clean":"2024-03-14T12:17:02.198093+0000","last_became_active":"2024-03-14T12:16:59.159038+0000","last_became_peered":"2024-03-14T12:16:59.159038+0000","last_unstale":"2024-03-14T12:17:02.198093+0000","last_undegraded":"2024-03-14T12:17:02.198093+0000","last_fullsized":"2024-03-14T12:17:02.198093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:47.565994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190927+0000","last_change":"2024-03-14T12:17:00.159112+0000","last_active":"2024-03-14T12:17:02.190927+0000","last_peered":"2024-03-14T12:17:02.190927+0000","last_clean":"2024-03-14T12:17:02.190927+0000","last_became_active":"2024-03-14T12:17:00.158781+0000","last_became_peered":"2024-03-14T12:17:00.158781+0000","last_unstale":"2024-03-14T12:17:02.190927+0000","last_undegraded":"2024-03-14T12:17:02.190927+0000","last_fullsized":"2024-03-14T12:17:02.190927+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:17.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198174+0000","last_change":"2024-03-14T12:16:59.160492+0000","last_active":"2024-03-14T12:17:02.198174+0000","last_peered":"2024-03-14T12:17:02.198174+0000","last_clean":"2024-03-14T12:17:02.198174+0000","last_became_active":"2024-03-14T12:16:59.160171+0000","last_became_peered":"2024-03-14T12:16:59.160171+0000","last_unstale":"2024-03-14T12:17:02.198174+0000","last_undegraded":"2024-03-14T12:17:02.198174+0000","last_fullsized":"2024-03-14T12:17:02.198174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:38.375005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197292+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:17:02.197292+0000","last_peered":"2024-03-14T12:17:02.197292+0000","last_clean":"2024-03-14T12:17:02.197292+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:17:02.197292+0000","last_undegraded":"2024-03-14T12:17:02.197292+0000","last_fullsized":"2024-03-14T12:17:02.197292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":941994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":351102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":351100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":351080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":847872,"data_stored":351122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":352622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36180,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335660,"statfs":{"total":96636764160,"available":96599715840,"internally_reserved":0,"allocated":860160,"data_stored":353238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:17:07.502 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:17:07.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.614+0000 7f929f4f8640 1 Processor -- start 2024-03-14T12:17:07.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.614+0000 7f929f4f8640 1 -- start start 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929f4f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9298071740 0x7f9298071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929f4f8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f929815e670 0x7f92980b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929f4f8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9298072060 con 0x7f9298143f60 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929f4f8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929815ebe0 con 0x7f9298071740 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929f4f8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929815eec0 con 0x7f929815e670 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929d294640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9298071740 0x7f9298071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929d294640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9298071740 0x7f9298071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46318/0 (socket says 172.21.15.82:46318) 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929d294640 1 -- 172.21.15.82:0/2395962783 learned_addr learned my addr 172.21.15.82:0/2395962783 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929ca93640 1 --2- 172.21.15.82:0/2395962783 >> v2:172.21.15.164:3300/0 conn(0x7f929815e670 0x7f92980b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929d294640 1 -- 172.21.15.82:0/2395962783 >> v2:172.21.15.164:3300/0 conn(0x7f929815e670 msgr2=0x7f92980b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:07.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929d294640 1 --2- 172.21.15.82:0/2395962783 >> v2:172.21.15.164:3300/0 conn(0x7f929815e670 0x7f92980b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929d294640 1 -- 172.21.15.82:0/2395962783 >> v1:172.21.15.164:6789/0 conn(0x7f9298143f60 legacy=0x7f929815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:07.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.615+0000 7f929d294640 1 -- 172.21.15.82:0/2395962783 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929815f1a0 con 0x7f9298071740 2024-03-14T12:17:07.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.616+0000 7f929d294640 1 --2- 172.21.15.82:0/2395962783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9298071740 0x7f9298071b20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9298072ac0 tx=0x7f928c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b488859f8bda04fa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:07.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.616+0000 7f928bfff640 1 -- 172.21.15.82:0/2395962783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c03b070 con 0x7f9298071740 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.616+0000 7f928bfff640 1 -- 172.21.15.82:0/2395962783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c036440 con 0x7f9298071740 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.616+0000 7f928bfff640 1 -- 172.21.15.82:0/2395962783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c036810 con 0x7f9298071740 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 -- 172.21.15.82:0/2395962783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9298071740 msgr2=0x7f9298071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2395962783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9298071740 0x7f9298071b20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9298072ac0 tx=0x7f928c02e0e0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 -- 172.21.15.82:0/2395962783 shutdown_connections 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2395962783 >> v2:172.21.15.164:3300/0 conn(0x7f929815e670 0x7f92980b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2395962783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9298071740 0x7f9298071b20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 -- 172.21.15.82:0/2395962783 >> 172.21.15.82:0/2395962783 conn(0x7f92980b1c70 msgr2=0x7f92980cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 -- 172.21.15.82:0/2395962783 shutdown_connections 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.617+0000 7f929f4f8640 1 -- 172.21.15.82:0/2395962783 wait complete. 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.618+0000 7f929f4f8640 1 Processor -- start 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.618+0000 7f929f4f8640 1 -- start start 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929f4f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9298071740 0x7f929815bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929f4f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9298143f60 0x7f9298141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929f4f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929815e670 0x7f9298159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929f4f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929815b080 con 0x7f929815e670 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929f4f8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9298143ce0 con 0x7f9298143f60 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929f4f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9298144670 con 0x7f9298071740 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929815e670 0x7f9298159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929815e670 0x7f9298159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46326/0 (socket says 172.21.15.82:46326) 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 -- 172.21.15.82:0/2324418156 learned_addr learned my addr 172.21.15.82:0/2324418156 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929ca93640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9298143f60 0x7f9298141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929d294640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9298071740 0x7f929815bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 -- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9298071740 msgr2=0x7f929815bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9298071740 0x7f929815bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 -- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9298143f60 msgr2=0x7f9298141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9298143f60 0x7f9298141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 -- 172.21.15.82:0/2324418156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92981f49b0 con 0x7f929815e670 2024-03-14T12:17:07.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.619+0000 7f929da95640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929815e670 0x7f9298159550 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f929000c780 tx=0x7f929000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:07.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.620+0000 7f9289ffb640 1 -- 172.21.15.82:0/2324418156 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290004b60 con 0x7f929815e670 2024-03-14T12:17:07.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.620+0000 7f9289ffb640 1 -- 172.21.15.82:0/2324418156 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9290004d00 con 0x7f929815e670 2024-03-14T12:17:07.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.620+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92981f4c90 con 0x7f929815e670 2024-03-14T12:17:07.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.620+0000 7f9289ffb640 1 -- 172.21.15.82:0/2324418156 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000aa90 con 0x7f929815e670 2024-03-14T12:17:07.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.620+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92981f5530 con 0x7f929815e670 2024-03-14T12:17:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.621+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92981f5080 con 0x7f929815e670 2024-03-14T12:17:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.621+0000 7f9289ffb640 1 -- 172.21.15.82:0/2324418156 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f929000ac30 con 0x7f929815e670 2024-03-14T12:17:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.622+0000 7f9289ffb640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f927c06bbd0 0x7f927c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.622+0000 7f9289ffb640 1 -- 172.21.15.82:0/2324418156 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f929008ab70 con 0x7f929815e670 2024-03-14T12:17:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.622+0000 7f929d294640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f927c06bbd0 0x7f927c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.622+0000 7f929d294640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f927c06bbd0 0x7f927c06e090 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f928c038000 tx=0x7f928c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:07.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.626+0000 7f9289ffb640 1 -- 172.21.15.82:0/2324418156 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9290020070 con 0x7f929815e670 2024-03-14T12:17:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.795+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f929804f050 con 0x7f927c06bbd0 2024-03-14T12:17:07.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.801+0000 7f9289ffb640 1 -- 172.21.15.82:0/2324418156 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252069 (secure 0 0 0) 0x7f929804f050 con 0x7f927c06bbd0 2024-03-14T12:17:07.803 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:07.804 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:17:07.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f927c06bbd0 msgr2=0x7f927c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f927c06bbd0 0x7f927c06e090 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f928c038000 tx=0x7f928c008040 comp rx=0 tx=0).stop 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929815e670 msgr2=0x7f9298159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929815e670 0x7f9298159550 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f929000c780 tx=0x7f929000cc40 comp rx=0 tx=0).stop 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 shutdown_connections 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929815e670 0x7f9298159550 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9298143f60 0x7f9298141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f927c06bbd0 0x7f927c06e090 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 --2- 172.21.15.82:0/2324418156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9298071740 0x7f929815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 >> 172.21.15.82:0/2324418156 conn(0x7f92980b1c70 msgr2=0x7f929806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 shutdown_connections 2024-03-14T12:17:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.804+0000 7f929f4f8640 1 -- 172.21.15.82:0/2324418156 wait complete. 2024-03-14T12:17:07.814 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-03-14T12:17:07.538119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261412,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713308,"statfs":{"total":773094113280,"available":772826427392,"internally_reserved":0,"allocated":7970816,"data_stored":3993636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":259575347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001839"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197760+0000","last_change":"2024-03-14T12:17:00.165695+0000","last_active":"2024-03-14T12:17:02.197760+0000","last_peered":"2024-03-14T12:17:02.197760+0000","last_clean":"2024-03-14T12:17:02.197760+0000","last_became_active":"2024-03-14T12:17:00.165201+0000","last_became_peered":"2024-03-14T12:17:00.165201+0000","last_unstale":"2024-03-14T12:17:02.197760+0000","last_undegraded":"2024-03-14T12:17:02.197760+0000","last_fullsized":"2024-03-14T12:17:02.197760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:23:16.151346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198669+0000","last_change":"2024-03-14T12:17:00.158662+0000","last_active":"2024-03-14T12:17:02.198669+0000","last_peered":"2024-03-14T12:17:02.198669+0000","last_clean":"2024-03-14T12:17:02.198669+0000","last_became_active":"2024-03-14T12:17:00.158337+0000","last_became_peered":"2024-03-14T12:17:00.158337+0000","last_unstale":"2024-03-14T12:17:02.198669+0000","last_undegraded":"2024-03-14T12:17:02.198669+0000","last_fullsized":"2024-03-14T12:17:02.198669+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:54.810418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191019+0000","last_change":"2024-03-14T12:17:00.155441+0000","last_active":"2024-03-14T12:17:02.191019+0000","last_peered":"2024-03-14T12:17:02.191019+0000","last_clean":"2024-03-14T12:17:02.191019+0000","last_became_active":"2024-03-14T12:17:00.155153+0000","last_became_peered":"2024-03-14T12:17:00.155153+0000","last_unstale":"2024-03-14T12:17:02.191019+0000","last_undegraded":"2024-03-14T12:17:02.191019+0000","last_fullsized":"2024-03-14T12:17:02.191019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:46.598883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196932+0000","last_change":"2024-03-14T12:17:00.158103+0000","last_active":"2024-03-14T12:17:02.196932+0000","last_peered":"2024-03-14T12:17:02.196932+0000","last_clean":"2024-03-14T12:17:02.196932+0000","last_became_active":"2024-03-14T12:17:00.157827+0000","last_became_peered":"2024-03-14T12:17:00.157827+0000","last_unstale":"2024-03-14T12:17:02.196932+0000","last_undegraded":"2024-03-14T12:17:02.196932+0000","last_fullsized":"2024-03-14T12:17:02.196932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:48.140138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144471+0000","last_change":"2024-03-14T12:17:00.158865+0000","last_active":"2024-03-14T12:17:01.144471+0000","last_peered":"2024-03-14T12:17:01.144471+0000","last_clean":"2024-03-14T12:17:01.144471+0000","last_became_active":"2024-03-14T12:17:00.158562+0000","last_became_peered":"2024-03-14T12:17:00.158562+0000","last_unstale":"2024-03-14T12:17:01.144471+0000","last_undegraded":"2024-03-14T12:17:01.144471+0000","last_fullsized":"2024-03-14T12:17:01.144471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:59:51.747175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198057+0000","last_change":"2024-03-14T12:17:00.150382+0000","last_active":"2024-03-14T12:17:02.198057+0000","last_peered":"2024-03-14T12:17:02.198057+0000","last_clean":"2024-03-14T12:17:02.198057+0000","last_became_active":"2024-03-14T12:17:00.150078+0000","last_became_peered":"2024-03-14T12:17:00.150078+0000","last_unstale":"2024-03-14T12:17:02.198057+0000","last_undegraded":"2024-03-14T12:17:02.198057+0000","last_fullsized":"2024-03-14T12:17:02.198057+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:19.263347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145699+0000","last_change":"2024-03-14T12:17:00.163727+0000","last_active":"2024-03-14T12:17:01.145699+0000","last_peered":"2024-03-14T12:17:01.145699+0000","last_clean":"2024-03-14T12:17:01.145699+0000","last_became_active":"2024-03-14T12:17:00.163450+0000","last_became_peered":"2024-03-14T12:17:00.163450+0000","last_unstale":"2024-03-14T12:17:01.145699+0000","last_undegraded":"2024-03-14T12:17:01.145699+0000","last_fullsized":"2024-03-14T12:17:01.145699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:00:17.473435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198050+0000","last_change":"2024-03-14T12:17:00.157697+0000","last_active":"2024-03-14T12:17:02.198050+0000","last_peered":"2024-03-14T12:17:02.198050+0000","last_clean":"2024-03-14T12:17:02.198050+0000","last_became_active":"2024-03-14T12:17:00.157426+0000","last_became_peered":"2024-03-14T12:17:00.157426+0000","last_unstale":"2024-03-14T12:17:02.198050+0000","last_undegraded":"2024-03-14T12:17:02.198050+0000","last_fullsized":"2024-03-14T12:17:02.198050+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:03:04.017798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144864+0000","last_change":"2024-03-14T12:17:00.155164+0000","last_active":"2024-03-14T12:17:01.144864+0000","last_peered":"2024-03-14T12:17:01.144864+0000","last_clean":"2024-03-14T12:17:01.144864+0000","last_became_active":"2024-03-14T12:17:00.154702+0000","last_became_peered":"2024-03-14T12:17:00.154702+0000","last_unstale":"2024-03-14T12:17:01.144864+0000","last_undegraded":"2024-03-14T12:17:01.144864+0000","last_fullsized":"2024-03-14T12:17:01.144864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:50.379295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146290+0000","last_change":"2024-03-14T12:17:00.156268+0000","last_active":"2024-03-14T12:17:01.146290+0000","last_peered":"2024-03-14T12:17:01.146290+0000","last_clean":"2024-03-14T12:17:01.146290+0000","last_became_active":"2024-03-14T12:17:00.149471+0000","last_became_peered":"2024-03-14T12:17:00.149471+0000","last_unstale":"2024-03-14T12:17:01.146290+0000","last_undegraded":"2024-03-14T12:17:01.146290+0000","last_fullsized":"2024-03-14T12:17:01.146290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:39.201880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197442+0000","last_change":"2024-03-14T12:17:00.160595+0000","last_active":"2024-03-14T12:17:02.197442+0000","last_peered":"2024-03-14T12:17:02.197442+0000","last_clean":"2024-03-14T12:17:02.197442+0000","last_became_active":"2024-03-14T12:17:00.160248+0000","last_became_peered":"2024-03-14T12:17:00.160248+0000","last_unstale":"2024-03-14T12:17:02.197442+0000","last_undegraded":"2024-03-14T12:17:02.197442+0000","last_fullsized":"2024-03-14T12:17:02.197442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196931+0000","last_change":"2024-03-14T12:17:00.161414+0000","last_active":"2024-03-14T12:17:02.196931+0000","last_peered":"2024-03-14T12:17:02.196931+0000","last_clean":"2024-03-14T12:17:02.196931+0000","last_became_active":"2024-03-14T12:17:00.161137+0000","last_became_peered":"2024-03-14T12:17:00.161137+0000","last_unstale":"2024-03-14T12:17:02.196931+0000","last_undegraded":"2024-03-14T12:17:02.196931+0000","last_fullsized":"2024-03-14T12:17:02.196931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:37.868936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159671+0000","last_change":"2024-03-14T12:17:00.157788+0000","last_active":"2024-03-14T12:17:02.159671+0000","last_peered":"2024-03-14T12:17:02.159671+0000","last_clean":"2024-03-14T12:17:02.159671+0000","last_became_active":"2024-03-14T12:17:00.157422+0000","last_became_peered":"2024-03-14T12:17:00.157422+0000","last_unstale":"2024-03-14T12:17:02.159671+0000","last_undegraded":"2024-03-14T12:17:02.159671+0000","last_fullsized":"2024-03-14T12:17:02.159671+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144432+0000","last_change":"2024-03-14T12:17:00.158475+0000","last_active":"2024-03-14T12:17:01.144432+0000","last_peered":"2024-03-14T12:17:01.144432+0000","last_clean":"2024-03-14T12:17:01.144432+0000","last_became_active":"2024-03-14T12:17:00.157438+0000","last_became_peered":"2024-03-14T12:17:00.157438+0000","last_unstale":"2024-03-14T12:17:01.144432+0000","last_undegraded":"2024-03-14T12:17:01.144432+0000","last_fullsized":"2024-03-14T12:17:01.144432+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:35.836879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159452+0000","last_change":"2024-03-14T12:17:00.160877+0000","last_active":"2024-03-14T12:17:02.159452+0000","last_peered":"2024-03-14T12:17:02.159452+0000","last_clean":"2024-03-14T12:17:02.159452+0000","last_became_active":"2024-03-14T12:17:00.160612+0000","last_became_peered":"2024-03-14T12:17:00.160612+0000","last_unstale":"2024-03-14T12:17:02.159452+0000","last_undegraded":"2024-03-14T12:17:02.159452+0000","last_fullsized":"2024-03-14T12:17:02.159452+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147120+0000","last_change":"2024-03-14T12:17:00.156510+0000","last_active":"2024-03-14T12:17:01.147120+0000","last_peered":"2024-03-14T12:17:01.147120+0000","last_clean":"2024-03-14T12:17:01.147120+0000","last_became_active":"2024-03-14T12:17:00.155713+0000","last_became_peered":"2024-03-14T12:17:00.155713+0000","last_unstale":"2024-03-14T12:17:01.147120+0000","last_undegraded":"2024-03-14T12:17:01.147120+0000","last_fullsized":"2024-03-14T12:17:01.147120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:54.810418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144864+0000","last_change":"2024-03-14T12:17:00.154954+0000","last_active":"2024-03-14T12:17:01.144864+0000","last_peered":"2024-03-14T12:17:01.144864+0000","last_clean":"2024-03-14T12:17:01.144864+0000","last_became_active":"2024-03-14T12:17:00.152674+0000","last_became_peered":"2024-03-14T12:17:00.152674+0000","last_unstale":"2024-03-14T12:17:01.144864+0000","last_undegraded":"2024-03-14T12:17:01.144864+0000","last_fullsized":"2024-03-14T12:17:01.144864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:24.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196995+0000","last_change":"2024-03-14T12:17:00.163547+0000","last_active":"2024-03-14T12:17:02.196995+0000","last_peered":"2024-03-14T12:17:02.196995+0000","last_clean":"2024-03-14T12:17:02.196995+0000","last_became_active":"2024-03-14T12:17:00.163294+0000","last_became_peered":"2024-03-14T12:17:00.163294+0000","last_unstale":"2024-03-14T12:17:02.196995+0000","last_undegraded":"2024-03-14T12:17:02.196995+0000","last_fullsized":"2024-03-14T12:17:02.196995+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:48.571300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198420+0000","last_change":"2024-03-14T12:17:00.158303+0000","last_active":"2024-03-14T12:17:02.198420+0000","last_peered":"2024-03-14T12:17:02.198420+0000","last_clean":"2024-03-14T12:17:02.198420+0000","last_became_active":"2024-03-14T12:17:00.158026+0000","last_became_peered":"2024-03-14T12:17:00.158026+0000","last_unstale":"2024-03-14T12:17:02.198420+0000","last_undegraded":"2024-03-14T12:17:02.198420+0000","last_fullsized":"2024-03-14T12:17:02.198420+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:07.658231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159693+0000","last_change":"2024-03-14T12:17:00.163989+0000","last_active":"2024-03-14T12:17:02.159693+0000","last_peered":"2024-03-14T12:17:02.159693+0000","last_clean":"2024-03-14T12:17:02.159693+0000","last_became_active":"2024-03-14T12:17:00.163708+0000","last_became_peered":"2024-03-14T12:17:00.163708+0000","last_unstale":"2024-03-14T12:17:02.159693+0000","last_undegraded":"2024-03-14T12:17:02.159693+0000","last_fullsized":"2024-03-14T12:17:02.159693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:03:50.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146748+0000","last_change":"2024-03-14T12:17:00.154763+0000","last_active":"2024-03-14T12:17:01.146748+0000","last_peered":"2024-03-14T12:17:01.146748+0000","last_clean":"2024-03-14T12:17:01.146748+0000","last_became_active":"2024-03-14T12:17:00.154477+0000","last_became_peered":"2024-03-14T12:17:00.154477+0000","last_unstale":"2024-03-14T12:17:01.146748+0000","last_undegraded":"2024-03-14T12:17:01.146748+0000","last_fullsized":"2024-03-14T12:17:01.146748+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:48.450402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146147+0000","last_change":"2024-03-14T12:17:00.150815+0000","last_active":"2024-03-14T12:17:01.146147+0000","last_peered":"2024-03-14T12:17:01.146147+0000","last_clean":"2024-03-14T12:17:01.146147+0000","last_became_active":"2024-03-14T12:17:00.149442+0000","last_became_peered":"2024-03-14T12:17:00.149442+0000","last_unstale":"2024-03-14T12:17:01.146147+0000","last_undegraded":"2024-03-14T12:17:01.146147+0000","last_fullsized":"2024-03-14T12:17:01.146147+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:05.828371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197316+0000","last_change":"2024-03-14T12:17:00.158417+0000","last_active":"2024-03-14T12:17:02.197316+0000","last_peered":"2024-03-14T12:17:02.197316+0000","last_clean":"2024-03-14T12:17:02.197316+0000","last_became_active":"2024-03-14T12:17:00.158167+0000","last_became_peered":"2024-03-14T12:17:00.158167+0000","last_unstale":"2024-03-14T12:17:02.197316+0000","last_undegraded":"2024-03-14T12:17:02.197316+0000","last_fullsized":"2024-03-14T12:17:02.197316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197213+0000","last_change":"2024-03-14T12:17:00.171366+0000","last_active":"2024-03-14T12:17:02.197213+0000","last_peered":"2024-03-14T12:17:02.197213+0000","last_clean":"2024-03-14T12:17:02.197213+0000","last_became_active":"2024-03-14T12:17:00.170528+0000","last_became_peered":"2024-03-14T12:17:00.170528+0000","last_unstale":"2024-03-14T12:17:02.197213+0000","last_undegraded":"2024-03-14T12:17:02.197213+0000","last_fullsized":"2024-03-14T12:17:02.197213+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145897+0000","last_change":"2024-03-14T12:17:00.164124+0000","last_active":"2024-03-14T12:17:01.145897+0000","last_peered":"2024-03-14T12:17:01.145897+0000","last_clean":"2024-03-14T12:17:01.145897+0000","last_became_active":"2024-03-14T12:17:00.163838+0000","last_became_peered":"2024-03-14T12:17:00.163838+0000","last_unstale":"2024-03-14T12:17:01.145897+0000","last_undegraded":"2024-03-14T12:17:01.145897+0000","last_fullsized":"2024-03-14T12:17:01.145897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:19.263347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146009+0000","last_change":"2024-03-14T12:17:00.169710+0000","last_active":"2024-03-14T12:17:01.146009+0000","last_peered":"2024-03-14T12:17:01.146009+0000","last_clean":"2024-03-14T12:17:01.146009+0000","last_became_active":"2024-03-14T12:17:00.169437+0000","last_became_peered":"2024-03-14T12:17:00.169437+0000","last_unstale":"2024-03-14T12:17:01.146009+0000","last_undegraded":"2024-03-14T12:17:01.146009+0000","last_fullsized":"2024-03-14T12:17:01.146009+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:25.284177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191164+0000","last_change":"2024-03-14T12:17:00.161546+0000","last_active":"2024-03-14T12:17:02.191164+0000","last_peered":"2024-03-14T12:17:02.191164+0000","last_clean":"2024-03-14T12:17:02.191164+0000","last_became_active":"2024-03-14T12:17:00.161174+0000","last_became_peered":"2024-03-14T12:17:00.161174+0000","last_unstale":"2024-03-14T12:17:02.191164+0000","last_undegraded":"2024-03-14T12:17:02.191164+0000","last_fullsized":"2024-03-14T12:17:02.191164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:38:05.720182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147357+0000","last_change":"2024-03-14T12:17:00.150641+0000","last_active":"2024-03-14T12:17:01.147357+0000","last_peered":"2024-03-14T12:17:01.147357+0000","last_clean":"2024-03-14T12:17:01.147357+0000","last_became_active":"2024-03-14T12:17:00.149567+0000","last_became_peered":"2024-03-14T12:17:00.149567+0000","last_unstale":"2024-03-14T12:17:01.147357+0000","last_undegraded":"2024-03-14T12:17:01.147357+0000","last_fullsized":"2024-03-14T12:17:01.147357+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:25.157808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191238+0000","last_change":"2024-03-14T12:17:00.160024+0000","last_active":"2024-03-14T12:17:02.191238+0000","last_peered":"2024-03-14T12:17:02.191238+0000","last_clean":"2024-03-14T12:17:02.191238+0000","last_became_active":"2024-03-14T12:17:00.159513+0000","last_became_peered":"2024-03-14T12:17:00.159513+0000","last_unstale":"2024-03-14T12:17:02.191238+0000","last_undegraded":"2024-03-14T12:17:02.191238+0000","last_fullsized":"2024-03-14T12:17:02.191238+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:02.639220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145867+0000","last_change":"2024-03-14T12:17:00.171649+0000","last_active":"2024-03-14T12:17:01.145867+0000","last_peered":"2024-03-14T12:17:01.145867+0000","last_clean":"2024-03-14T12:17:01.145867+0000","last_became_active":"2024-03-14T12:17:00.171384+0000","last_became_peered":"2024-03-14T12:17:00.171384+0000","last_unstale":"2024-03-14T12:17:01.145867+0000","last_undegraded":"2024-03-14T12:17:01.145867+0000","last_fullsized":"2024-03-14T12:17:01.145867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:50:38.677924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146359+0000","last_change":"2024-03-14T12:17:00.164315+0000","last_active":"2024-03-14T12:17:01.146359+0000","last_peered":"2024-03-14T12:17:01.146359+0000","last_clean":"2024-03-14T12:17:01.146359+0000","last_became_active":"2024-03-14T12:17:00.164057+0000","last_became_peered":"2024-03-14T12:17:00.164057+0000","last_unstale":"2024-03-14T12:17:01.146359+0000","last_undegraded":"2024-03-14T12:17:01.146359+0000","last_fullsized":"2024-03-14T12:17:01.146359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198081+0000","last_change":"2024-03-14T12:17:00.156339+0000","last_active":"2024-03-14T12:17:02.198081+0000","last_peered":"2024-03-14T12:17:02.198081+0000","last_clean":"2024-03-14T12:17:02.198081+0000","last_became_active":"2024-03-14T12:17:00.156068+0000","last_became_peered":"2024-03-14T12:17:00.156068+0000","last_unstale":"2024-03-14T12:17:02.198081+0000","last_undegraded":"2024-03-14T12:17:02.198081+0000","last_fullsized":"2024-03-14T12:17:02.198081+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:05.828371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197795+0000","last_change":"2024-03-14T12:17:00.164288+0000","last_active":"2024-03-14T12:17:02.197795+0000","last_peered":"2024-03-14T12:17:02.197795+0000","last_clean":"2024-03-14T12:17:02.197795+0000","last_became_active":"2024-03-14T12:17:00.163969+0000","last_became_peered":"2024-03-14T12:17:00.163969+0000","last_unstale":"2024-03-14T12:17:02.197795+0000","last_undegraded":"2024-03-14T12:17:02.197795+0000","last_fullsized":"2024-03-14T12:17:02.197795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:10:36.197658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197849+0000","last_change":"2024-03-14T12:16:59.160707+0000","last_active":"2024-03-14T12:17:02.197849+0000","last_peered":"2024-03-14T12:17:02.197849+0000","last_clean":"2024-03-14T12:17:02.197849+0000","last_became_active":"2024-03-14T12:16:59.160442+0000","last_became_peered":"2024-03-14T12:16:59.160442+0000","last_unstale":"2024-03-14T12:17:02.197849+0000","last_undegraded":"2024-03-14T12:17:02.197849+0000","last_fullsized":"2024-03-14T12:17:02.197849+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:06.652925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144948+0000","last_change":"2024-03-14T12:17:00.152459+0000","last_active":"2024-03-14T12:17:01.144948+0000","last_peered":"2024-03-14T12:17:01.144948+0000","last_clean":"2024-03-14T12:17:01.144948+0000","last_became_active":"2024-03-14T12:17:00.152163+0000","last_became_peered":"2024-03-14T12:17:00.152163+0000","last_unstale":"2024-03-14T12:17:01.144948+0000","last_undegraded":"2024-03-14T12:17:01.144948+0000","last_fullsized":"2024-03-14T12:17:01.144948+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:52.251106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159748+0000","last_change":"2024-03-14T12:16:59.160021+0000","last_active":"2024-03-14T12:17:02.159748+0000","last_peered":"2024-03-14T12:17:02.159748+0000","last_clean":"2024-03-14T12:17:02.159748+0000","last_became_active":"2024-03-14T12:16:59.159736+0000","last_became_peered":"2024-03-14T12:16:59.159736+0000","last_unstale":"2024-03-14T12:17:02.159748+0000","last_undegraded":"2024-03-14T12:17:02.159748+0000","last_fullsized":"2024-03-14T12:17:02.159748+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:25:53.805112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198429+0000","last_change":"2024-03-14T12:17:00.162771+0000","last_active":"2024-03-14T12:17:02.198429+0000","last_peered":"2024-03-14T12:17:02.198429+0000","last_clean":"2024-03-14T12:17:02.198429+0000","last_became_active":"2024-03-14T12:17:00.162509+0000","last_became_peered":"2024-03-14T12:17:00.162509+0000","last_unstale":"2024-03-14T12:17:02.198429+0000","last_undegraded":"2024-03-14T12:17:02.198429+0000","last_fullsized":"2024-03-14T12:17:02.198429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:48.450402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146397+0000","last_change":"2024-03-14T12:16:59.150884+0000","last_active":"2024-03-14T12:17:01.146397+0000","last_peered":"2024-03-14T12:17:01.146397+0000","last_clean":"2024-03-14T12:17:01.146397+0000","last_became_active":"2024-03-14T12:16:59.150534+0000","last_became_peered":"2024-03-14T12:16:59.150534+0000","last_unstale":"2024-03-14T12:17:01.146397+0000","last_undegraded":"2024-03-14T12:17:01.146397+0000","last_fullsized":"2024-03-14T12:17:01.146397+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:39.288670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197636+0000","last_change":"2024-03-14T12:17:00.161330+0000","last_active":"2024-03-14T12:17:02.197636+0000","last_peered":"2024-03-14T12:17:02.197636+0000","last_clean":"2024-03-14T12:17:02.197636+0000","last_became_active":"2024-03-14T12:17:00.161060+0000","last_became_peered":"2024-03-14T12:17:00.161060+0000","last_unstale":"2024-03-14T12:17:02.197636+0000","last_undegraded":"2024-03-14T12:17:02.197636+0000","last_fullsized":"2024-03-14T12:17:02.197636+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:53:48.285207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198508+0000","last_change":"2024-03-14T12:16:59.160120+0000","last_active":"2024-03-14T12:17:02.198508+0000","last_peered":"2024-03-14T12:17:02.198508+0000","last_clean":"2024-03-14T12:17:02.198508+0000","last_became_active":"2024-03-14T12:16:59.159810+0000","last_became_peered":"2024-03-14T12:16:59.159810+0000","last_unstale":"2024-03-14T12:17:02.198508+0000","last_undegraded":"2024-03-14T12:17:02.198508+0000","last_fullsized":"2024-03-14T12:17:02.198508+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:55:23.661196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145933+0000","last_change":"2024-03-14T12:17:00.160986+0000","last_active":"2024-03-14T12:17:01.145933+0000","last_peered":"2024-03-14T12:17:01.145933+0000","last_clean":"2024-03-14T12:17:01.145933+0000","last_became_active":"2024-03-14T12:17:00.160699+0000","last_became_peered":"2024-03-14T12:17:00.160699+0000","last_unstale":"2024-03-14T12:17:01.145933+0000","last_undegraded":"2024-03-14T12:17:01.145933+0000","last_fullsized":"2024-03-14T12:17:01.145933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:04:30.945677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190985+0000","last_change":"2024-03-14T12:16:59.158332+0000","last_active":"2024-03-14T12:17:02.190985+0000","last_peered":"2024-03-14T12:17:02.190985+0000","last_clean":"2024-03-14T12:17:02.190985+0000","last_became_active":"2024-03-14T12:16:59.158049+0000","last_became_peered":"2024-03-14T12:16:59.158049+0000","last_unstale":"2024-03-14T12:17:02.190985+0000","last_undegraded":"2024-03-14T12:17:02.190985+0000","last_fullsized":"2024-03-14T12:17:02.190985+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:16:10.934601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146888+0000","last_change":"2024-03-14T12:17:00.159212+0000","last_active":"2024-03-14T12:17:01.146888+0000","last_peered":"2024-03-14T12:17:01.146888+0000","last_clean":"2024-03-14T12:17:01.146888+0000","last_became_active":"2024-03-14T12:17:00.158951+0000","last_became_peered":"2024-03-14T12:17:00.158951+0000","last_unstale":"2024-03-14T12:17:01.146888+0000","last_undegraded":"2024-03-14T12:17:01.146888+0000","last_fullsized":"2024-03-14T12:17:01.146888+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:03.354492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197981+0000","last_change":"2024-03-14T12:16:59.158539+0000","last_active":"2024-03-14T12:17:02.197981+0000","last_peered":"2024-03-14T12:17:02.197981+0000","last_clean":"2024-03-14T12:17:02.197981+0000","last_became_active":"2024-03-14T12:16:59.158223+0000","last_became_peered":"2024-03-14T12:16:59.158223+0000","last_unstale":"2024-03-14T12:17:02.197981+0000","last_undegraded":"2024-03-14T12:17:02.197981+0000","last_fullsized":"2024-03-14T12:17:02.197981+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:19:21.430991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144255+0000","last_change":"2024-03-14T12:17:00.146758+0000","last_active":"2024-03-14T12:17:01.144255+0000","last_peered":"2024-03-14T12:17:01.144255+0000","last_clean":"2024-03-14T12:17:01.144255+0000","last_became_active":"2024-03-14T12:17:00.146485+0000","last_became_peered":"2024-03-14T12:17:00.146485+0000","last_unstale":"2024-03-14T12:17:01.144255+0000","last_undegraded":"2024-03-14T12:17:01.144255+0000","last_fullsized":"2024-03-14T12:17:01.144255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:57:23.282105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196659+0000","last_change":"2024-03-14T12:16:59.153534+0000","last_active":"2024-03-14T12:17:02.196659+0000","last_peered":"2024-03-14T12:17:02.196659+0000","last_clean":"2024-03-14T12:17:02.196659+0000","last_became_active":"2024-03-14T12:16:59.153249+0000","last_became_peered":"2024-03-14T12:16:59.153249+0000","last_unstale":"2024-03-14T12:17:02.196659+0000","last_undegraded":"2024-03-14T12:17:02.196659+0000","last_fullsized":"2024-03-14T12:17:02.196659+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:57:22.276798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159422+0000","last_change":"2024-03-14T12:17:00.161414+0000","last_active":"2024-03-14T12:17:02.159422+0000","last_peered":"2024-03-14T12:17:02.159422+0000","last_clean":"2024-03-14T12:17:02.159422+0000","last_became_active":"2024-03-14T12:17:00.161178+0000","last_became_peered":"2024-03-14T12:17:00.161178+0000","last_unstale":"2024-03-14T12:17:02.159422+0000","last_undegraded":"2024-03-14T12:17:02.159422+0000","last_fullsized":"2024-03-14T12:17:02.159422+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:05:10.595088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144186+0000","last_change":"2024-03-14T12:16:59.156881+0000","last_active":"2024-03-14T12:17:01.144186+0000","last_peered":"2024-03-14T12:17:01.144186+0000","last_clean":"2024-03-14T12:17:01.144186+0000","last_became_active":"2024-03-14T12:16:59.156609+0000","last_became_peered":"2024-03-14T12:16:59.156609+0000","last_unstale":"2024-03-14T12:17:01.144186+0000","last_undegraded":"2024-03-14T12:17:01.144186+0000","last_fullsized":"2024-03-14T12:17:01.144186+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:14:57.357163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146160+0000","last_change":"2024-03-14T12:17:00.163114+0000","last_active":"2024-03-14T12:17:01.146160+0000","last_peered":"2024-03-14T12:17:01.146160+0000","last_clean":"2024-03-14T12:17:01.146160+0000","last_became_active":"2024-03-14T12:17:00.162835+0000","last_became_peered":"2024-03-14T12:17:00.162835+0000","last_unstale":"2024-03-14T12:17:01.146160+0000","last_undegraded":"2024-03-14T12:17:01.146160+0000","last_fullsized":"2024-03-14T12:17:01.146160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145963+0000","last_change":"2024-03-14T12:16:59.157422+0000","last_active":"2024-03-14T12:17:01.145963+0000","last_peered":"2024-03-14T12:17:01.145963+0000","last_clean":"2024-03-14T12:17:01.145963+0000","last_became_active":"2024-03-14T12:16:59.155980+0000","last_became_peered":"2024-03-14T12:16:59.155980+0000","last_unstale":"2024-03-14T12:17:01.145963+0000","last_undegraded":"2024-03-14T12:17:01.145963+0000","last_fullsized":"2024-03-14T12:17:01.145963+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:46.803639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197812+0000","last_change":"2024-03-14T12:17:00.167346+0000","last_active":"2024-03-14T12:17:02.197812+0000","last_peered":"2024-03-14T12:17:02.197812+0000","last_clean":"2024-03-14T12:17:02.197812+0000","last_became_active":"2024-03-14T12:17:00.167075+0000","last_became_peered":"2024-03-14T12:17:00.167075+0000","last_unstale":"2024-03-14T12:17:02.197812+0000","last_undegraded":"2024-03-14T12:17:02.197812+0000","last_fullsized":"2024-03-14T12:17:02.197812+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:36.504209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198643+0000","last_change":"2024-03-14T12:16:59.154772+0000","last_active":"2024-03-14T12:17:02.198643+0000","last_peered":"2024-03-14T12:17:02.198643+0000","last_clean":"2024-03-14T12:17:02.198643+0000","last_became_active":"2024-03-14T12:16:59.154363+0000","last_became_peered":"2024-03-14T12:16:59.154363+0000","last_unstale":"2024-03-14T12:17:02.198643+0000","last_undegraded":"2024-03-14T12:17:02.198643+0000","last_fullsized":"2024-03-14T12:17:02.198643+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:35:46.803639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198434+0000","last_change":"2024-03-14T12:17:00.156703+0000","last_active":"2024-03-14T12:17:02.198434+0000","last_peered":"2024-03-14T12:17:02.198434+0000","last_clean":"2024-03-14T12:17:02.198434+0000","last_became_active":"2024-03-14T12:17:00.156256+0000","last_became_peered":"2024-03-14T12:17:00.156256+0000","last_unstale":"2024-03-14T12:17:02.198434+0000","last_undegraded":"2024-03-14T12:17:02.198434+0000","last_fullsized":"2024-03-14T12:17:02.198434+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:03.354492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.191221+0000","last_change":"2024-03-14T12:16:59.160192+0000","last_active":"2024-03-14T12:17:02.191221+0000","last_peered":"2024-03-14T12:17:02.191221+0000","last_clean":"2024-03-14T12:17:02.191221+0000","last_became_active":"2024-03-14T12:16:59.159901+0000","last_became_peered":"2024-03-14T12:16:59.159901+0000","last_unstale":"2024-03-14T12:17:02.191221+0000","last_undegraded":"2024-03-14T12:17:02.191221+0000","last_fullsized":"2024-03-14T12:17:02.191221+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:28:07.504769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144649+0000","last_change":"2024-03-14T12:17:00.150822+0000","last_active":"2024-03-14T12:17:01.144649+0000","last_peered":"2024-03-14T12:17:01.144649+0000","last_clean":"2024-03-14T12:17:01.144649+0000","last_became_active":"2024-03-14T12:17:00.150463+0000","last_became_peered":"2024-03-14T12:17:00.150463+0000","last_unstale":"2024-03-14T12:17:01.144649+0000","last_undegraded":"2024-03-14T12:17:01.144649+0000","last_fullsized":"2024-03-14T12:17:01.144649+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:39.380311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197081+0000","last_change":"2024-03-14T12:16:59.147830+0000","last_active":"2024-03-14T12:17:02.197081+0000","last_peered":"2024-03-14T12:17:02.197081+0000","last_clean":"2024-03-14T12:17:02.197081+0000","last_became_active":"2024-03-14T12:16:59.147503+0000","last_became_peered":"2024-03-14T12:16:59.147503+0000","last_unstale":"2024-03-14T12:17:02.197081+0000","last_undegraded":"2024-03-14T12:17:02.197081+0000","last_fullsized":"2024-03-14T12:17:02.197081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:48.936304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197326+0000","last_change":"2024-03-14T12:17:00.161960+0000","last_active":"2024-03-14T12:17:02.197326+0000","last_peered":"2024-03-14T12:17:02.197326+0000","last_clean":"2024-03-14T12:17:02.197326+0000","last_became_active":"2024-03-14T12:17:00.161650+0000","last_became_peered":"2024-03-14T12:17:00.161650+0000","last_unstale":"2024-03-14T12:17:02.197326+0000","last_undegraded":"2024-03-14T12:17:02.197326+0000","last_fullsized":"2024-03-14T12:17:02.197326+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:59.605734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196793+0000","last_change":"2024-03-14T12:16:59.156977+0000","last_active":"2024-03-14T12:17:02.196793+0000","last_peered":"2024-03-14T12:17:02.196793+0000","last_clean":"2024-03-14T12:17:02.196793+0000","last_became_active":"2024-03-14T12:16:59.156715+0000","last_became_peered":"2024-03-14T12:16:59.156715+0000","last_unstale":"2024-03-14T12:17:02.196793+0000","last_undegraded":"2024-03-14T12:17:02.196793+0000","last_fullsized":"2024-03-14T12:17:02.196793+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:51:34.831573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145991+0000","last_change":"2024-03-14T12:17:00.170808+0000","last_active":"2024-03-14T12:17:01.145991+0000","last_peered":"2024-03-14T12:17:01.145991+0000","last_clean":"2024-03-14T12:17:01.145991+0000","last_became_active":"2024-03-14T12:17:00.170529+0000","last_became_peered":"2024-03-14T12:17:00.170529+0000","last_unstale":"2024-03-14T12:17:01.145991+0000","last_undegraded":"2024-03-14T12:17:01.145991+0000","last_fullsized":"2024-03-14T12:17:01.145991+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:05:10.595088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198068+0000","last_change":"2024-03-14T12:16:59.162979+0000","last_active":"2024-03-14T12:17:02.198068+0000","last_peered":"2024-03-14T12:17:02.198068+0000","last_clean":"2024-03-14T12:17:02.198068+0000","last_became_active":"2024-03-14T12:16:59.162698+0000","last_became_peered":"2024-03-14T12:16:59.162698+0000","last_unstale":"2024-03-14T12:17:02.198068+0000","last_undegraded":"2024-03-14T12:17:02.198068+0000","last_fullsized":"2024-03-14T12:17:02.198068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:51.245799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159543+0000","last_change":"2024-03-14T12:17:00.157109+0000","last_active":"2024-03-14T12:17:02.159543+0000","last_peered":"2024-03-14T12:17:02.159543+0000","last_clean":"2024-03-14T12:17:02.159543+0000","last_became_active":"2024-03-14T12:17:00.156852+0000","last_became_peered":"2024-03-14T12:17:00.156852+0000","last_unstale":"2024-03-14T12:17:02.159543+0000","last_undegraded":"2024-03-14T12:17:02.159543+0000","last_fullsized":"2024-03-14T12:17:02.159543+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146075+0000","last_change":"2024-03-14T12:16:59.157658+0000","last_active":"2024-03-14T12:17:01.146075+0000","last_peered":"2024-03-14T12:17:01.146075+0000","last_clean":"2024-03-14T12:17:01.146075+0000","last_became_active":"2024-03-14T12:16:59.156521+0000","last_became_peered":"2024-03-14T12:16:59.156521+0000","last_unstale":"2024-03-14T12:17:01.146075+0000","last_undegraded":"2024-03-14T12:17:01.146075+0000","last_fullsized":"2024-03-14T12:17:01.146075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:59:18.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144334+0000","last_change":"2024-03-14T12:17:00.152560+0000","last_active":"2024-03-14T12:17:01.144334+0000","last_peered":"2024-03-14T12:17:01.144334+0000","last_clean":"2024-03-14T12:17:01.144334+0000","last_became_active":"2024-03-14T12:17:00.152302+0000","last_became_peered":"2024-03-14T12:17:00.152302+0000","last_unstale":"2024-03-14T12:17:01.144334+0000","last_undegraded":"2024-03-14T12:17:01.144334+0000","last_fullsized":"2024-03-14T12:17:01.144334+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:46.281680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.147065+0000","last_change":"2024-03-14T12:16:59.157970+0000","last_active":"2024-03-14T12:17:01.147065+0000","last_peered":"2024-03-14T12:17:01.147065+0000","last_clean":"2024-03-14T12:17:01.147065+0000","last_became_active":"2024-03-14T12:16:59.157694+0000","last_became_peered":"2024-03-14T12:16:59.157694+0000","last_unstale":"2024-03-14T12:17:01.147065+0000","last_undegraded":"2024-03-14T12:17:01.147065+0000","last_fullsized":"2024-03-14T12:17:01.147065+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:47.565994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146444+0000","last_change":"2024-03-14T12:17:00.151385+0000","last_active":"2024-03-14T12:17:01.146444+0000","last_peered":"2024-03-14T12:17:01.146444+0000","last_clean":"2024-03-14T12:17:01.146444+0000","last_became_active":"2024-03-14T12:17:00.151140+0000","last_became_peered":"2024-03-14T12:17:00.151140+0000","last_unstale":"2024-03-14T12:17:01.146444+0000","last_undegraded":"2024-03-14T12:17:01.146444+0000","last_fullsized":"2024-03-14T12:17:01.146444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:36:07.658231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198497+0000","last_change":"2024-03-14T12:16:59.161389+0000","last_active":"2024-03-14T12:17:02.198497+0000","last_peered":"2024-03-14T12:17:02.198497+0000","last_clean":"2024-03-14T12:17:02.198497+0000","last_became_active":"2024-03-14T12:16:59.161130+0000","last_became_peered":"2024-03-14T12:16:59.161130+0000","last_unstale":"2024-03-14T12:17:02.198497+0000","last_undegraded":"2024-03-14T12:17:02.198497+0000","last_fullsized":"2024-03-14T12:17:02.198497+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:36.863630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197907+0000","last_change":"2024-03-14T12:17:00.168001+0000","last_active":"2024-03-14T12:17:02.197907+0000","last_peered":"2024-03-14T12:17:02.197907+0000","last_clean":"2024-03-14T12:17:02.197907+0000","last_became_active":"2024-03-14T12:17:00.167735+0000","last_became_peered":"2024-03-14T12:17:00.167735+0000","last_unstale":"2024-03-14T12:17:02.197907+0000","last_undegraded":"2024-03-14T12:17:02.197907+0000","last_fullsized":"2024-03-14T12:17:02.197907+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:16:11.939907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146483+0000","last_change":"2024-03-14T12:16:59.153745+0000","last_active":"2024-03-14T12:17:01.146483+0000","last_peered":"2024-03-14T12:17:01.146483+0000","last_clean":"2024-03-14T12:17:01.146483+0000","last_became_active":"2024-03-14T12:16:59.153456+0000","last_became_peered":"2024-03-14T12:16:59.153456+0000","last_unstale":"2024-03-14T12:17:01.146483+0000","last_undegraded":"2024-03-14T12:17:01.146483+0000","last_fullsized":"2024-03-14T12:17:01.146483+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:36:23.425887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197129+0000","last_change":"2024-03-14T12:17:00.161390+0000","last_active":"2024-03-14T12:17:02.197129+0000","last_peered":"2024-03-14T12:17:02.197129+0000","last_clean":"2024-03-14T12:17:02.197129+0000","last_became_active":"2024-03-14T12:17:00.161087+0000","last_became_peered":"2024-03-14T12:17:00.161087+0000","last_unstale":"2024-03-14T12:17:02.197129+0000","last_undegraded":"2024-03-14T12:17:02.197129+0000","last_fullsized":"2024-03-14T12:17:02.197129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:40.293976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198465+0000","last_change":"2024-03-14T12:16:59.151077+0000","last_active":"2024-03-14T12:17:02.198465+0000","last_peered":"2024-03-14T12:17:02.198465+0000","last_clean":"2024-03-14T12:17:02.198465+0000","last_became_active":"2024-03-14T12:16:59.150728+0000","last_became_peered":"2024-03-14T12:16:59.150728+0000","last_unstale":"2024-03-14T12:17:02.198465+0000","last_undegraded":"2024-03-14T12:17:02.198465+0000","last_fullsized":"2024-03-14T12:17:02.198465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:44.730420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144710+0000","last_change":"2024-03-14T12:17:00.148497+0000","last_active":"2024-03-14T12:17:01.144710+0000","last_peered":"2024-03-14T12:17:01.144710+0000","last_clean":"2024-03-14T12:17:01.144710+0000","last_became_active":"2024-03-14T12:17:00.148125+0000","last_became_peered":"2024-03-14T12:17:00.148125+0000","last_unstale":"2024-03-14T12:17:01.144710+0000","last_undegraded":"2024-03-14T12:17:01.144710+0000","last_fullsized":"2024-03-14T12:17:01.144710+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:19:22.436297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197406+0000","last_change":"2024-03-14T12:16:59.160922+0000","last_active":"2024-03-14T12:17:02.197406+0000","last_peered":"2024-03-14T12:17:02.197406+0000","last_clean":"2024-03-14T12:17:02.197406+0000","last_became_active":"2024-03-14T12:16:59.160607+0000","last_became_peered":"2024-03-14T12:16:59.160607+0000","last_unstale":"2024-03-14T12:17:02.197406+0000","last_undegraded":"2024-03-14T12:17:02.197406+0000","last_fullsized":"2024-03-14T12:17:02.197406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:38.196573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190932+0000","last_change":"2024-03-14T12:17:00.156143+0000","last_active":"2024-03-14T12:17:02.190932+0000","last_peered":"2024-03-14T12:17:02.190932+0000","last_clean":"2024-03-14T12:17:02.190932+0000","last_became_active":"2024-03-14T12:17:00.155844+0000","last_became_peered":"2024-03-14T12:17:00.155844+0000","last_unstale":"2024-03-14T12:17:02.190932+0000","last_undegraded":"2024-03-14T12:17:02.190932+0000","last_fullsized":"2024-03-14T12:17:02.190932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:49:56.360025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198120+0000","last_change":"2024-03-14T12:16:59.158103+0000","last_active":"2024-03-14T12:17:02.198120+0000","last_peered":"2024-03-14T12:17:02.198120+0000","last_clean":"2024-03-14T12:17:02.198120+0000","last_became_active":"2024-03-14T12:16:59.157770+0000","last_became_peered":"2024-03-14T12:16:59.157770+0000","last_unstale":"2024-03-14T12:17:02.198120+0000","last_undegraded":"2024-03-14T12:17:02.198120+0000","last_fullsized":"2024-03-14T12:17:02.198120+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:58:49.373989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190827+0000","last_change":"2024-03-14T12:17:00.159001+0000","last_active":"2024-03-14T12:17:02.190827+0000","last_peered":"2024-03-14T12:17:02.190827+0000","last_clean":"2024-03-14T12:17:02.190827+0000","last_became_active":"2024-03-14T12:17:00.158720+0000","last_became_peered":"2024-03-14T12:17:00.158720+0000","last_unstale":"2024-03-14T12:17:02.190827+0000","last_undegraded":"2024-03-14T12:17:02.190827+0000","last_fullsized":"2024-03-14T12:17:02.190827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:00:30.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146166+0000","last_change":"2024-03-14T12:16:59.149014+0000","last_active":"2024-03-14T12:17:01.146166+0000","last_peered":"2024-03-14T12:17:01.146166+0000","last_clean":"2024-03-14T12:17:01.146166+0000","last_became_active":"2024-03-14T12:16:59.148708+0000","last_became_peered":"2024-03-14T12:16:59.148708+0000","last_unstale":"2024-03-14T12:17:01.146166+0000","last_undegraded":"2024-03-14T12:17:01.146166+0000","last_fullsized":"2024-03-14T12:17:01.146166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:04.823064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190842+0000","last_change":"2024-03-14T12:17:00.158300+0000","last_active":"2024-03-14T12:17:02.190842+0000","last_peered":"2024-03-14T12:17:02.190842+0000","last_clean":"2024-03-14T12:17:02.190842+0000","last_became_active":"2024-03-14T12:17:00.158049+0000","last_became_peered":"2024-03-14T12:17:00.158049+0000","last_unstale":"2024-03-14T12:17:02.190842+0000","last_undegraded":"2024-03-14T12:17:02.190842+0000","last_fullsized":"2024-03-14T12:17:02.190842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:33:51.561745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159206+0000","last_change":"2024-03-14T12:16:59.163118+0000","last_active":"2024-03-14T12:17:02.159206+0000","last_peered":"2024-03-14T12:17:02.159206+0000","last_clean":"2024-03-14T12:17:02.159206+0000","last_became_active":"2024-03-14T12:16:59.162808+0000","last_became_peered":"2024-03-14T12:16:59.162808+0000","last_unstale":"2024-03-14T12:17:02.159206+0000","last_undegraded":"2024-03-14T12:17:02.159206+0000","last_fullsized":"2024-03-14T12:17:02.159206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:02.349185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159686+0000","last_change":"2024-03-14T12:17:00.161657+0000","last_active":"2024-03-14T12:17:02.159686+0000","last_peered":"2024-03-14T12:17:02.159686+0000","last_clean":"2024-03-14T12:17:02.159686+0000","last_became_active":"2024-03-14T12:17:00.160635+0000","last_became_peered":"2024-03-14T12:17:00.160635+0000","last_unstale":"2024-03-14T12:17:02.159686+0000","last_undegraded":"2024-03-14T12:17:02.159686+0000","last_fullsized":"2024-03-14T12:17:02.159686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:04:14.726006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198729+0000","last_change":"2024-03-14T12:16:59.160742+0000","last_active":"2024-03-14T12:17:02.198729+0000","last_peered":"2024-03-14T12:17:02.198729+0000","last_clean":"2024-03-14T12:17:02.198729+0000","last_became_active":"2024-03-14T12:16:59.160452+0000","last_became_peered":"2024-03-14T12:16:59.160452+0000","last_unstale":"2024-03-14T12:17:02.198729+0000","last_undegraded":"2024-03-14T12:17:02.198729+0000","last_fullsized":"2024-03-14T12:17:02.198729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:47.134831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146041+0000","last_change":"2024-03-14T12:17:00.161317+0000","last_active":"2024-03-14T12:17:01.146041+0000","last_peered":"2024-03-14T12:17:01.146041+0000","last_clean":"2024-03-14T12:17:01.146041+0000","last_became_active":"2024-03-14T12:17:00.161046+0000","last_became_peered":"2024-03-14T12:17:00.161046+0000","last_unstale":"2024-03-14T12:17:01.146041+0000","last_undegraded":"2024-03-14T12:17:01.146041+0000","last_fullsized":"2024-03-14T12:17:01.146041+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:09.685141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.146258+0000","last_change":"2024-03-14T12:16:59.157167+0000","last_active":"2024-03-14T12:17:01.146258+0000","last_peered":"2024-03-14T12:17:01.146258+0000","last_clean":"2024-03-14T12:17:01.146258+0000","last_became_active":"2024-03-14T12:16:59.156847+0000","last_became_peered":"2024-03-14T12:16:59.156847+0000","last_unstale":"2024-03-14T12:17:01.146258+0000","last_undegraded":"2024-03-14T12:17:01.146258+0000","last_fullsized":"2024-03-14T12:17:01.146258+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:03:03.012491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197493+0000","last_change":"2024-03-14T12:17:00.168815+0000","last_active":"2024-03-14T12:17:02.197493+0000","last_peered":"2024-03-14T12:17:02.197493+0000","last_clean":"2024-03-14T12:17:02.197493+0000","last_became_active":"2024-03-14T12:17:00.168561+0000","last_became_peered":"2024-03-14T12:17:00.168561+0000","last_unstale":"2024-03-14T12:17:02.197493+0000","last_undegraded":"2024-03-14T12:17:02.197493+0000","last_fullsized":"2024-03-14T12:17:02.197493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:03:50.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144705+0000","last_change":"2024-03-14T12:16:59.157608+0000","last_active":"2024-03-14T12:17:01.144705+0000","last_peered":"2024-03-14T12:17:01.144705+0000","last_clean":"2024-03-14T12:17:01.144705+0000","last_became_active":"2024-03-14T12:16:59.157339+0000","last_became_peered":"2024-03-14T12:16:59.157339+0000","last_unstale":"2024-03-14T12:17:01.144705+0000","last_undegraded":"2024-03-14T12:17:01.144705+0000","last_fullsized":"2024-03-14T12:17:01.144705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:28.610985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197678+0000","last_change":"2024-03-14T12:17:00.163234+0000","last_active":"2024-03-14T12:17:02.197678+0000","last_peered":"2024-03-14T12:17:02.197678+0000","last_clean":"2024-03-14T12:17:02.197678+0000","last_became_active":"2024-03-14T12:17:00.162922+0000","last_became_peered":"2024-03-14T12:17:00.162922+0000","last_unstale":"2024-03-14T12:17:02.197678+0000","last_undegraded":"2024-03-14T12:17:02.197678+0000","last_fullsized":"2024-03-14T12:17:02.197678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:54:53.421821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.159555+0000","last_change":"2024-03-14T12:16:59.161879+0000","last_active":"2024-03-14T12:17:02.159555+0000","last_peered":"2024-03-14T12:17:02.159555+0000","last_clean":"2024-03-14T12:17:02.159555+0000","last_became_active":"2024-03-14T12:16:59.161612+0000","last_became_peered":"2024-03-14T12:16:59.161612+0000","last_unstale":"2024-03-14T12:17:02.159555+0000","last_undegraded":"2024-03-14T12:17:02.159555+0000","last_fullsized":"2024-03-14T12:17:02.159555+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:24.152502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144466+0000","last_change":"2024-03-14T12:17:00.152780+0000","last_active":"2024-03-14T12:17:01.144466+0000","last_peered":"2024-03-14T12:17:01.144466+0000","last_clean":"2024-03-14T12:17:01.144466+0000","last_became_active":"2024-03-14T12:17:00.152426+0000","last_became_peered":"2024-03-14T12:17:00.152426+0000","last_unstale":"2024-03-14T12:17:01.144466+0000","last_undegraded":"2024-03-14T12:17:01.144466+0000","last_fullsized":"2024-03-14T12:17:01.144466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:49.941611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.144339+0000","last_change":"2024-03-14T12:16:59.150881+0000","last_active":"2024-03-14T12:17:01.144339+0000","last_peered":"2024-03-14T12:17:01.144339+0000","last_clean":"2024-03-14T12:17:01.144339+0000","last_became_active":"2024-03-14T12:16:59.150578+0000","last_became_peered":"2024-03-14T12:16:59.150578+0000","last_unstale":"2024-03-14T12:17:01.144339+0000","last_undegraded":"2024-03-14T12:17:01.144339+0000","last_fullsized":"2024-03-14T12:17:01.144339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:02.510738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198277+0000","last_change":"2024-03-14T12:17:00.159754+0000","last_active":"2024-03-14T12:17:02.198277+0000","last_peered":"2024-03-14T12:17:02.198277+0000","last_clean":"2024-03-14T12:17:02.198277+0000","last_became_active":"2024-03-14T12:17:00.158664+0000","last_became_peered":"2024-03-14T12:17:00.158664+0000","last_unstale":"2024-03-14T12:17:02.198277+0000","last_undegraded":"2024-03-14T12:17:02.198277+0000","last_fullsized":"2024-03-14T12:17:02.198277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:34:39.201880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196679+0000","last_change":"2024-03-14T12:16:59.153605+0000","last_active":"2024-03-14T12:17:02.196679+0000","last_peered":"2024-03-14T12:17:02.196679+0000","last_clean":"2024-03-14T12:17:02.196679+0000","last_became_active":"2024-03-14T12:16:59.153330+0000","last_became_peered":"2024-03-14T12:16:59.153330+0000","last_unstale":"2024-03-14T12:17:02.196679+0000","last_undegraded":"2024-03-14T12:17:02.196679+0000","last_fullsized":"2024-03-14T12:17:02.196679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:45.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.196750+0000","last_change":"2024-03-14T12:17:00.154881+0000","last_active":"2024-03-14T12:17:02.196750+0000","last_peered":"2024-03-14T12:17:02.196750+0000","last_clean":"2024-03-14T12:17:02.196750+0000","last_became_active":"2024-03-14T12:17:00.154597+0000","last_became_peered":"2024-03-14T12:17:00.154597+0000","last_unstale":"2024-03-14T12:17:02.196750+0000","last_undegraded":"2024-03-14T12:17:02.196750+0000","last_fullsized":"2024-03-14T12:17:02.196750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:40:45.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:01.145806+0000","last_change":"2024-03-14T12:16:59.158773+0000","last_active":"2024-03-14T12:17:01.145806+0000","last_peered":"2024-03-14T12:17:01.145806+0000","last_clean":"2024-03-14T12:17:01.145806+0000","last_became_active":"2024-03-14T12:16:59.158497+0000","last_became_peered":"2024-03-14T12:16:59.158497+0000","last_unstale":"2024-03-14T12:17:01.145806+0000","last_undegraded":"2024-03-14T12:17:01.145806+0000","last_fullsized":"2024-03-14T12:17:01.145806+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:07:36.819982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197762+0000","last_change":"2024-03-14T12:17:00.155232+0000","last_active":"2024-03-14T12:17:02.197762+0000","last_peered":"2024-03-14T12:17:02.197762+0000","last_clean":"2024-03-14T12:17:02.197762+0000","last_became_active":"2024-03-14T12:17:00.154850+0000","last_became_peered":"2024-03-14T12:17:00.154850+0000","last_unstale":"2024-03-14T12:17:02.197762+0000","last_undegraded":"2024-03-14T12:17:02.197762+0000","last_fullsized":"2024-03-14T12:17:02.197762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:21:24.931278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198093+0000","last_change":"2024-03-14T12:16:59.159327+0000","last_active":"2024-03-14T12:17:02.198093+0000","last_peered":"2024-03-14T12:17:02.198093+0000","last_clean":"2024-03-14T12:17:02.198093+0000","last_became_active":"2024-03-14T12:16:59.159038+0000","last_became_peered":"2024-03-14T12:16:59.159038+0000","last_unstale":"2024-03-14T12:17:02.198093+0000","last_undegraded":"2024-03-14T12:17:02.198093+0000","last_fullsized":"2024-03-14T12:17:02.198093+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:06:47.565994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.190927+0000","last_change":"2024-03-14T12:17:00.159112+0000","last_active":"2024-03-14T12:17:02.190927+0000","last_peered":"2024-03-14T12:17:02.190927+0000","last_clean":"2024-03-14T12:17:02.190927+0000","last_became_active":"2024-03-14T12:17:00.158781+0000","last_became_peered":"2024-03-14T12:17:00.158781+0000","last_unstale":"2024-03-14T12:17:02.190927+0000","last_undegraded":"2024-03-14T12:17:02.190927+0000","last_fullsized":"2024-03-14T12:17:02.190927+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:59.129472+0000","last_clean_scrub_stamp":"2024-03-14T12:16:59.129472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:17.421534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.198174+0000","last_change":"2024-03-14T12:16:59.160492+0000","last_active":"2024-03-14T12:17:02.198174+0000","last_peered":"2024-03-14T12:17:02.198174+0000","last_clean":"2024-03-14T12:17:02.198174+0000","last_became_active":"2024-03-14T12:16:59.160171+0000","last_became_peered":"2024-03-14T12:16:59.160171+0000","last_unstale":"2024-03-14T12:17:02.198174+0000","last_undegraded":"2024-03-14T12:17:02.198174+0000","last_fullsized":"2024-03-14T12:17:02.198174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:16:58.124166+0000","last_clean_scrub_stamp":"2024-03-14T12:16:58.124166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:36:38.375005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T12:17:02.197292+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:17:02.197292+0000","last_peered":"2024-03-14T12:17:02.197292+0000","last_clean":"2024-03-14T12:17:02.197292+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:17:02.197292+0000","last_undegraded":"2024-03-14T12:17:02.197292+0000","last_fullsized":"2024-03-14T12:17:02.197292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":941994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":351102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":351100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":351080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":847872,"data_stored":351122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":352622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36180,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335660,"statfs":{"total":96636764160,"available":96599715840,"internally_reserved":0,"allocated":860160,"data_stored":353238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:17:07.818 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T12:17:07.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-14T12:17:07.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.962+0000 7fb4e8f7e640 1 Processor -- start 2024-03-14T12:17:07.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.962+0000 7fb4e8f7e640 1 -- start start 2024-03-14T12:17:07.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e8f7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4143c60 0x7fb4e40b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e8f7e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e40b41c0 con 0x7fb4e4140e70 2024-03-14T12:17:07.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e8f7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e4161c10 con 0x7fb4e41626f0 2024-03-14T12:17:07.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e8f7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4161ef0 con 0x7fb4e4143c60 2024-03-14T12:17:07.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb4e4140e70 0x7fb4e4143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44500/0 (socket says 172.21.15.82:44500) 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e259c640 1 -- 172.21.15.82:0/2047483968 learned_addr learned my addr 172.21.15.82:0/2047483968 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4dbfff640 1 --2- 172.21.15.82:0/2047483968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4143c60 0x7fb4e40b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230281861 0 0) 0x7fb4e40b41c0 con 0x7fb4e4140e70 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d0003520 con 0x7fb4e4140e70 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512505379 0 0) 0x7fb4e4161c10 con 0x7fb4e41626f0 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e40b41c0 con 0x7fb4e41626f0 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4dbfff640 1 -- 172.21.15.82:0/2047483968 >> v1:172.21.15.164:6789/0 conn(0x7fb4e41626f0 legacy=0x7fb4e40af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789519919 0 0) 0x7fb4d0003520 con 0x7fb4e4140e70 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4dbfff640 1 -- 172.21.15.82:0/2047483968 >> v1:172.21.15.82:6789/0 conn(0x7fb4e4140e70 legacy=0x7fb4e4143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.963+0000 7fb4dbfff640 1 -- 172.21.15.82:0/2047483968 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e41621d0 con 0x7fb4e4143c60 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.964+0000 7fb4dbfff640 1 --2- 172.21.15.82:0/2047483968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4143c60 0x7fb4e40b3b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097d0 tx=0x7fb4cc02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3e3e8701fee260 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.964+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037070 con 0x7fb4e4143c60 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.964+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc030510 con 0x7fb4e4143c60 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.964+0000 7fb4e1d9b640 1 -- 172.21.15.82:0/2047483968 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4001670 con 0x7fb4e4143c60 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.965+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/2047483968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4143c60 msgr2=0x7fb4e40b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.965+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/2047483968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4143c60 0x7fb4e40b3b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097d0 tx=0x7fb4cc02b800 comp rx=0 tx=0).stop 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.965+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/2047483968 shutdown_connections 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.965+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/2047483968 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4143c60 0x7fb4e40b3b30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.965+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/2047483968 >> 172.21.15.82:0/2047483968 conn(0x7fb4e40b2830 msgr2=0x7fb4e40b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.965+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/2047483968 shutdown_connections 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.965+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/2047483968 wait complete. 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.966+0000 7fb4e8f7e640 1 Processor -- start 2024-03-14T12:17:07.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.966+0000 7fb4e8f7e640 1 -- start start 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e8f7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4140e70 0x7fb4e40cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e8f7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e4143c60 0x7fb4e41301d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e8f7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4e41626f0 0x7fb4e40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e8f7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415f240 con 0x7fb4e4143c60 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e8f7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415f0c0 con 0x7fb4e41626f0 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e8f7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415f3c0 con 0x7fb4e4140e70 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e4143c60 0x7fb4e41301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e4143c60 0x7fb4e41301d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46340/0 (socket says 172.21.15.82:46340) 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 -- 172.21.15.82:0/1613470173 learned_addr learned my addr 172.21.15.82:0/1613470173 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e259c640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4140e70 0x7fb4e40cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4e2d9d640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4e41626f0 0x7fb4e40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 -- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4140e70 msgr2=0x7fb4e40cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4140e70 0x7fb4e40cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 -- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4e41626f0 msgr2=0x7fb4e40cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4e41626f0 0x7fb4e40cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 -- 172.21.15.82:0/1613470173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40cf5e0 con 0x7fb4e4143c60 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.967+0000 7fb4dbfff640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e4143c60 0x7fb4e41301d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02b840 tx=0x7fb4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.968+0000 7fb4daffd640 1 -- 172.21.15.82:0/1613470173 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc042070 con 0x7fb4e4143c60 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.968+0000 7fb4daffd640 1 -- 172.21.15.82:0/1613470173 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc03e870 con 0x7fb4e4143c60 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.968+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e4130c90 con 0x7fb4e4143c60 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.968+0000 7fb4daffd640 1 -- 172.21.15.82:0/1613470173 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037040 con 0x7fb4e4143c60 2024-03-14T12:17:07.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.968+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e4131250 con 0x7fb4e4143c60 2024-03-14T12:17:07.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.969+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e404f050 con 0x7fb4e4143c60 2024-03-14T12:17:07.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.969+0000 7fb4daffd640 1 -- 172.21.15.82:0/1613470173 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb4cc045070 con 0x7fb4e4143c60 2024-03-14T12:17:07.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.970+0000 7fb4daffd640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb4c406bc00 0x7fb4c406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:07.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.970+0000 7fb4e259c640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb4c406bc00 0x7fb4c406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:07.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.970+0000 7fb4daffd640 1 -- 172.21.15.82:0/1613470173 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4cc0ac1f0 con 0x7fb4e4143c60 2024-03-14T12:17:07.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.970+0000 7fb4e259c640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb4c406bc00 0x7fb4c406e0c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4000fd0 tx=0x7fb4d4001150 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:07.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:07.974+0000 7fb4daffd640 1 -- 172.21.15.82:0/1613470173 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb4cc04c070 con 0x7fb4e4143c60 2024-03-14T12:17:08.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.212+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb4e40d1630 con 0x7fb4e4143c60 2024-03-14T12:17:08.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.214+0000 7fb4daffd640 1 -- 172.21.15.82:0/1613470173 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb4e40cf5e0 con 0x7fb4e4143c60 2024-03-14T12:17:08.213 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-03-14T12:17:08.214 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDF6fJlMYxvOBAASnH9eTjNpCbjk2Yk3dhJEg== 2024-03-14T12:17:08.214 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-03-14T12:17:08.214 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-03-14T12:17:08.214 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-14T12:17:08.214 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb4c406bc00 msgr2=0x7fb4c406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb4c406bc00 0x7fb4c406e0c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4000fd0 tx=0x7fb4d4001150 comp rx=0 tx=0).stop 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e4143c60 msgr2=0x7fb4e41301d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e4143c60 0x7fb4e41301d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc02b840 tx=0x7fb4cc0046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 shutdown_connections 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4e41626f0 0x7fb4e40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e4143c60 0x7fb4e41301d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb4c406bc00 0x7fb4c406e0c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 --2- 172.21.15.82:0/1613470173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4e4140e70 0x7fb4e40cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 >> 172.21.15.82:0/1613470173 conn(0x7fb4e40b2830 msgr2=0x7fb4e4143260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:08.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 shutdown_connections 2024-03-14T12:17:08.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.216+0000 7fb4e8f7e640 1 -- 172.21.15.82:0/1613470173 wait complete. 2024-03-14T12:17:08.226 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:08.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.327+0000 7f92f2fd4640 1 Processor -- start 2024-03-14T12:17:08.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.327+0000 7f92f2fd4640 1 -- start start 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f2fd4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f92ec151230 0x7f92ec0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f2fd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 0x7f92ec070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f2fd4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ec0b3da0 con 0x7f92ec15dde0 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f2fd4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec06c3b0 con 0x7f92ec151230 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f2fd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec06c690 con 0x7f92ec0b4b70 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f1571640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f92ec15dde0 0x7f92ec06cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44516/0 (socket says 172.21.15.82:44516) 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f1571640 1 -- 172.21.15.82:0/2335901507 learned_addr learned my addr 172.21.15.82:0/2335901507 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854687668 0 0) 0x7f92ec0b3da0 con 0x7f92ec15dde0 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d4003520 con 0x7f92ec15dde0 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f0d70640 1 --2- 172.21.15.82:0/2335901507 >> v2:172.21.15.164:3300/0 conn(0x7f92ec151230 0x7f92ec0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e3fff640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 0x7f92ec070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3444218073 0 0) 0x7f92d4003520 con 0x7f92ec15dde0 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 msgr2=0x7f92ec070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 0x7f92ec070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 >> v2:172.21.15.164:3300/0 conn(0x7f92ec151230 msgr2=0x7f92ec0cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 --2- 172.21.15.82:0/2335901507 >> v2:172.21.15.164:3300/0 conn(0x7f92ec151230 0x7f92ec0cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec0b4080 con 0x7f92ec15dde0 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f92e4001670 con 0x7f92ec15dde0 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 >> v1:172.21.15.82:6789/0 conn(0x7f92ec15dde0 legacy=0x7f92ec06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 0x7f92d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92d4004d40 0x7f92d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec06c3b0 con 0x7f92d40039b0 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec06c690 con 0x7f92d4004d40 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400bb20 con 0x7f92ec0b4b70 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f0d70640 1 --2- 172.21.15.82:0/2335901507 >> v2:172.21.15.164:3300/0 conn(0x7f92ec151230 0x7f92ec0cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:08.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.328+0000 7f92f0d70640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92e3fff640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 0x7f92d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92f1571640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92d4004d40 0x7f92d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92f0d70640 1 -- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 msgr2=0x7f92d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92f0d70640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec0b4b70 0x7f92d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92f0d70640 1 -- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92d4004d40 msgr2=0x7f92d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92f0d70640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92d4004d40 0x7f92d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92f0d70640 1 -- 172.21.15.82:0/2335901507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4003520 con 0x7f92d40039b0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.329+0000 7f92f0d70640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92d4003dd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f92e80073e0 tx=0x7f92d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528239e1fd44bc8f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8039070 con 0x7f92d40039b0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e802cdc0 con 0x7f92d40039b0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92e37fe640 1 -- 172.21.15.82:0/2335901507 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8009c70 con 0x7f92d40039b0 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92f2fd4640 1 -- 172.21.15.82:0/2335901507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 msgr2=0x7f92d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92f2fd4640 1 --2- 172.21.15.82:0/2335901507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92d4003dd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f92e80073e0 tx=0x7f92d4005830 comp rx=0 tx=0).stop 2024-03-14T12:17:08.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92f1571640 1 -- 172.21.15.82:0/2335901507 reap_dead start 2024-03-14T12:17:08.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92f2fd4640 1 -- 172.21.15.82:0/2335901507 shutdown_connections 2024-03-14T12:17:08.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92f2fd4640 1 -- 172.21.15.82:0/2335901507 >> 172.21.15.82:0/2335901507 conn(0x7f92ec0b1c70 msgr2=0x7f92ec0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:08.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92f2fd4640 1 -- 172.21.15.82:0/2335901507 shutdown_connections 2024-03-14T12:17:08.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.330+0000 7f92f2fd4640 1 -- 172.21.15.82:0/2335901507 wait complete. 2024-03-14T12:17:08.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.331+0000 7f92f2fd4640 1 Processor -- start 2024-03-14T12:17:08.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.331+0000 7f92f2fd4640 1 -- start start 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f2fd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92ec146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f2fd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec148600 0x7f92ec149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f2fd4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92ec142960 0x7f92ec0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f2fd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0cf5e0 con 0x7f92d40039b0 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f2fd4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec151610 con 0x7f92ec142960 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f2fd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec06d230 con 0x7f92ec148600 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f0d70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92ec146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f0d70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92ec146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46358/0 (socket says 172.21.15.82:46358) 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f0d70640 1 -- 172.21.15.82:0/502777790 learned_addr learned my addr 172.21.15.82:0/502777790 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92f1571640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92ec142960 0x7f92ec0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.332+0000 7f92e3fff640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec148600 0x7f92ec149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f0d70640 1 -- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec148600 msgr2=0x7f92ec149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f0d70640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec148600 0x7f92ec149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f0d70640 1 -- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92ec142960 msgr2=0x7f92ec0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f0d70640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92ec142960 0x7f92ec0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f0d70640 1 -- 172.21.15.82:0/502777790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec149030 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92e3fff640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec148600 0x7f92ec149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f0d70640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92ec146690 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f92e80073e0 tx=0x7f92e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92e17fa640 1 -- 172.21.15.82:0/502777790 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8009c70 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92e17fa640 1 -- 172.21.15.82:0/502777790 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e8040070 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec1492d0 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92e17fa640 1 -- 172.21.15.82:0/502777790 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e803c850 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.333+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec149890 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.334+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec04f050 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.334+0000 7f92e17fa640 1 -- 172.21.15.82:0/502777790 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f92e803c9f0 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.335+0000 7f92e17fa640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92d806bbd0 0x7f92d806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.335+0000 7f92e17fa640 1 -- 172.21.15.82:0/502777790 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92e80aecc0 con 0x7f92d40039b0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.335+0000 7f92e3fff640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92d806bbd0 0x7f92d806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:08.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.335+0000 7f92e3fff640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92d806bbd0 0x7f92d806e090 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f92ec146bf0 tx=0x7f92dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:08.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.339+0000 7f92e17fa640 1 -- 172.21.15.82:0/502777790 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f92e807d870 con 0x7f92d40039b0 2024-03-14T12:17:08.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.546+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92ec0b2f40 con 0x7f92d40039b0 2024-03-14T12:17:08.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.547+0000 7f92e17fa640 1 -- 172.21.15.82:0/502777790 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f92e807d170 con 0x7f92d40039b0 2024-03-14T12:17:08.547 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:08.547 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6471,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:01.490623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6834/4033072140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4033072140},{"type":"v1","addr":"172.21.15.164:6834","nonce":4033072140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:08.547 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 34 2024-03-14T12:17:08.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92d806bbd0 msgr2=0x7f92d806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92d806bbd0 0x7f92d806e090 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f92ec146bf0 tx=0x7f92dc008040 comp rx=0 tx=0).stop 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 msgr2=0x7f92ec146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92ec146690 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f92e80073e0 tx=0x7f92e8004ab0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 shutdown_connections 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92ec142960 0x7f92ec0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f92d806bbd0 0x7f92d806e090 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92ec148600 0x7f92ec149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 --2- 172.21.15.82:0/502777790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92d40039b0 0x7f92ec146690 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 >> 172.21.15.82:0/502777790 conn(0x7f92ec0b1c70 msgr2=0x7f92ec071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 shutdown_connections 2024-03-14T12:17:08.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:08.549+0000 7f92f2fd4640 1 -- 172.21.15.82:0/502777790 wait complete. 2024-03-14T12:17:08.962 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T12:16:59.493058+0000', 'modified': '2024-03-14T12:17:01.490623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'b', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6834/4033072140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 4033072140}, {'type': 'v1', 'addr': '172.21.15.164:6834', 'nonce': 4033072140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T12:17:08.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T12:17:08.963 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-14T12:17:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.067+0000 7fc074eb1640 1 Processor -- start 2024-03-14T12:17:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.067+0000 7fc074eb1640 1 -- start start 2024-03-14T12:17:09.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc074eb1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc070071740 0x7fc070071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc074eb1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc07015e670 0x7fc0700b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc074eb1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc070143f60 0x7fc07015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc074eb1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc070072060 con 0x7fc070071740 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc074eb1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07015ebe0 con 0x7fc070143f60 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc074eb1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07015eec0 con 0x7fc07015e670 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc070071740 0x7fc070071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc070071740 0x7fc070071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46360/0 (socket says 172.21.15.82:46360) 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 -- 172.21.15.82:0/1137917702 learned_addr learned my addr 172.21.15.82:0/1137917702 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 -- 172.21.15.82:0/1137917702 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc07015e670 msgr2=0x7fc0700b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 --2- 172.21.15.82:0/1137917702 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc07015e670 0x7fc0700b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 -- 172.21.15.82:0/1137917702 >> v2:172.21.15.164:3300/0 conn(0x7fc070143f60 msgr2=0x7fc07015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/1137917702 >> v2:172.21.15.164:3300/0 conn(0x7fc070143f60 0x7fc07015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 --2- 172.21.15.82:0/1137917702 >> v2:172.21.15.164:3300/0 conn(0x7fc070143f60 0x7fc07015b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.068+0000 7fc06e59c640 1 -- 172.21.15.82:0/1137917702 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07015f1a0 con 0x7fc070071740 2024-03-14T12:17:09.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc06e59c640 1 --2- 172.21.15.82:0/1137917702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc070071740 0x7fc070071b20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc070072ac0 tx=0x7fc06802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffffbb635cd3edfd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc06d59a640 1 -- 172.21.15.82:0/1137917702 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06803b070 con 0x7fc070071740 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc06d59a640 1 -- 172.21.15.82:0/1137917702 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068036440 con 0x7fc070071740 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc06d59a640 1 -- 172.21.15.82:0/1137917702 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068036810 con 0x7fc070071740 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc074eb1640 1 -- 172.21.15.82:0/1137917702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc070071740 msgr2=0x7fc070071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc074eb1640 1 --2- 172.21.15.82:0/1137917702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc070071740 0x7fc070071b20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc070072ac0 tx=0x7fc06802e0e0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc074eb1640 1 -- 172.21.15.82:0/1137917702 shutdown_connections 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.069+0000 7fc074eb1640 1 --2- 172.21.15.82:0/1137917702 >> v2:172.21.15.164:3300/0 conn(0x7fc070143f60 0x7fc07015b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.070+0000 7fc074eb1640 1 --2- 172.21.15.82:0/1137917702 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc07015e670 0x7fc0700b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.070+0000 7fc074eb1640 1 --2- 172.21.15.82:0/1137917702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc070071740 0x7fc070071b20 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.070+0000 7fc074eb1640 1 -- 172.21.15.82:0/1137917702 >> 172.21.15.82:0/1137917702 conn(0x7fc0700b1c70 msgr2=0x7fc0700cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.070+0000 7fc074eb1640 1 -- 172.21.15.82:0/1137917702 shutdown_connections 2024-03-14T12:17:09.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.070+0000 7fc074eb1640 1 -- 172.21.15.82:0/1137917702 wait complete. 2024-03-14T12:17:09.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.071+0000 7fc074eb1640 1 Processor -- start 2024-03-14T12:17:09.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.071+0000 7fc074eb1640 1 -- start start 2024-03-14T12:17:09.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc070071740 0x7fc07015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc070143f60 0x7fc07015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07015e670 0x7fc07013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07015b080 con 0x7fc07015e670 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc070143ce0 con 0x7fc070143f60 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc070144670 con 0x7fc070071740 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07015e670 0x7fc07013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07015e670 0x7fc07013e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46364/0 (socket says 172.21.15.82:46364) 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 -- 172.21.15.82:0/2208198637 learned_addr learned my addr 172.21.15.82:0/2208198637 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc070143f60 0x7fc07015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06e59c640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc070071740 0x7fc07015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 -- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc070071740 msgr2=0x7fc07015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc070071740 0x7fc07015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 -- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc070143f60 msgr2=0x7fc07015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc070143f60 0x7fc07015def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 -- 172.21.15.82:0/2208198637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07015e430 con 0x7fc07015e670 2024-03-14T12:17:09.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc070143f60 0x7fc07015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:09.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07015e670 0x7fc07013e6a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cc30 tx=0x7fc06000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc066ffd640 1 -- 172.21.15.82:0/2208198637 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000ce40 con 0x7fc07015e670 2024-03-14T12:17:09.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc066ffd640 1 -- 172.21.15.82:0/2208198637 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc060003990 con 0x7fc07015e670 2024-03-14T12:17:09.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07013d600 con 0x7fc07015e670 2024-03-14T12:17:09.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc066ffd640 1 -- 172.21.15.82:0/2208198637 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060003d40 con 0x7fc07015e670 2024-03-14T12:17:09.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.072+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07013dbc0 con 0x7fc07015e670 2024-03-14T12:17:09.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.074+0000 7fc066ffd640 1 -- 172.21.15.82:0/2208198637 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc060003fa0 con 0x7fc07015e670 2024-03-14T12:17:09.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.074+0000 7fc066ffd640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc05006bbd0 0x7fc05006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.075+0000 7fc066ffd640 1 -- 172.21.15.82:0/2208198637 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc06008af60 con 0x7fc07015e670 2024-03-14T12:17:09.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.075+0000 7fc06e59c640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc05006bbd0 0x7fc05006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.075+0000 7fc06e59c640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc05006bbd0 0x7fc05006e090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc068038000 tx=0x7fc068032000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.075+0000 7fc077b68740 1 Processor -- start 2024-03-14T12:17:09.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.075+0000 7fc077b68740 1 -- start start 2024-03-14T12:17:09.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c11380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc077b68740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c52ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc077b68740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5b747c0 con 0x55c8b5c37580 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5b1db50 con 0x55c8b5c3a8d0 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5b70860 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c52ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c52ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46374/0 (socket says 172.21.15.82:46374) 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.076+0000 7fc06dd9b640 1 -- 172.21.15.82:0/774660458 learned_addr learned my addr 172.21.15.82:0/774660458 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06e59c640 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c11380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06dd9b640 1 -- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 msgr2=0x55c8b5c11380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c11380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06dd9b640 1 -- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 msgr2=0x55c8b5c33340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c33340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06dd9b640 1 -- 172.21.15.82:0/774660458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b5b319a0 con 0x55c8b5c37580 2024-03-14T12:17:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c52ed0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc058007760 tx=0x7fc058007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0723c1955d5f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc04ffff640 1 -- 172.21.15.82:0/774660458 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05800d950 con 0x55c8b5c37580 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc04ffff640 1 -- 172.21.15.82:0/774660458 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058018070 con 0x55c8b5c37580 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.077+0000 7fc04ffff640 1 -- 172.21.15.82:0/774660458 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05800dcf0 con 0x55c8b5c37580 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 -- 172.21.15.82:0/774660458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 msgr2=0x55c8b5c52ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c52ed0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc058007760 tx=0x7fc058007c20 comp rx=0 tx=0).stop 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 -- 172.21.15.82:0/774660458 shutdown_connections 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c33340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c52ed0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 --2- 172.21.15.82:0/774660458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c11380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 -- 172.21.15.82:0/774660458 >> 172.21.15.82:0/774660458 conn(0x55c8b5bd7400 msgr2=0x55c8b5c55300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 -- 172.21.15.82:0/774660458 shutdown_connections 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 -- 172.21.15.82:0/774660458 wait complete. 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 -- start start 2024-03-14T12:17:09.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.078+0000 7fc077b68740 1 Processor -- start 2024-03-14T12:17:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc077b68740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c34880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c34dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc077b68740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5c0a510 con 0x55c8b5c37580 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5b30f90 con 0x55c8b5c3a8d0 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5c0a7b0 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc077b68740 10 client.0 _ll_register_callbacks cb 0x55c8b5b6c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c4c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46390/0 (socket says 172.21.15.82:46390) 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc06e59c640 1 -- 172.21.15.82:0/2487632405 learned_addr learned my addr 172.21.15.82:0/2487632405 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c34880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.079+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c34dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06e59c640 1 -- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 msgr2=0x55c8b5c34880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06e59c640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c34880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06e59c640 1 -- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 msgr2=0x55c8b5c34dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06e59c640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c34dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06e59c640 1 -- 172.21.15.82:0/2487632405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8b58892d0 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c34880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c34dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc06e59c640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c4c380 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc06802e6f0 tx=0x7fc068030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de57a8e920f07d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc04d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc04d7fa640 1 -- 172.21.15.82:0/2487632405 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068051060 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc04d7fa640 1 -- 172.21.15.82:0/2487632405 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc068043ad0 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc058018070 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc04d7fa640 1 -- 172.21.15.82:0/2487632405 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc068057050 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 10 client.6636 fetch_fsmap learned FSMap version 34 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c8b59536b0 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc04d7fa640 1 -- 172.21.15.82:0/2487632405 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058018070 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc04d7fa640 1 -- 172.21.15.82:0/2487632405 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc06805a020 con 0x55c8b5c37580 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 10 client.6636 fetch_fsmap finished waiting for FSMap version 34 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 10 client.6636 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 20 client.6636 populate_metadata read hostname 'smithi082' 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5c4e210 0x55c8b5c4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 4 client.6636 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.080+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 --> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8b5b81d00 con 0x55c8b5c4e210 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.081+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5c4e210 0x55c8b5c4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.081+0000 7fc04d7fa640 10 client.6636 ms_handle_connect on v2:172.21.15.164:6832/4033072140 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.081+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5c4e210 0x55c8b5c4e5f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf57b10faef73016 server_cookie=bf2d6bf50162651c in_seq=0 out_seq=0 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc04d7fa640 1 -- 172.21.15.82:0/2487632405 <== mds.0 v2:172.21.15.164:6832/4033072140 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8b5b81d00 con 0x55c8b5c4e210 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc04d7fa640 10 client.6636 handle_command_reply: tid=0 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 client.6636 shutdown 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 msgr2=0x55c8b5c4c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c4c380 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc06802e6f0 tx=0x7fc068030070 comp rx=0 tx=0).stop 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 shutdown_connections 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5c4e210 0x55c8b5c4e5f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c34dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c34880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 --2- 172.21.15.82:0/2487632405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c4c380 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc04d7fa640 0 client.6636 ms_handle_reset on v2:172.21.15.164:6832/4033072140 2024-03-14T12:17:09.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 >> 172.21.15.82:0/2487632405 conn(0x55c8b5bd7400 msgr2=0x55c8b5c0fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 shutdown_connections 2024-03-14T12:17:09.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 1 -- 172.21.15.82:0/2487632405 wait complete. 2024-03-14T12:17:09.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.082+0000 7fc077b68740 20 client.6636 trim_cache size 0 max 16384 2024-03-14T12:17:09.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 Processor -- start 2024-03-14T12:17:09.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 -- start start 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c0f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c55540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07015bd70 con 0x55c8b5c37580 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07013b550 con 0x55c8b5c3a8d0 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07015b770 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c0f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c0f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46404/0 (socket says 172.21.15.82:46404) 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.098+0000 7fc06e59c640 1 -- 172.21.15.82:0/1274731555 learned_addr learned my addr 172.21.15.82:0/1274731555 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc06e59c640 1 -- 172.21.15.82:0/1274731555 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 msgr2=0x55c8b5c55000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc06e59c640 1 --2- 172.21.15.82:0/1274731555 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc06e59c640 1 -- 172.21.15.82:0/1274731555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 msgr2=0x55c8b5c55540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc06e59c640 1 --2- 172.21.15.82:0/1274731555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c55540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc06e59c640 1 -- 172.21.15.82:0/1274731555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b5c56730 con 0x55c8b5c37580 2024-03-14T12:17:09.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc06e59c640 1 --2- 172.21.15.82:0/1274731555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c0f590 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc068008010 tx=0x7fc06803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaa2ae10724eb7d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc04f7fe640 1 -- 172.21.15.82:0/1274731555 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06803b040 con 0x55c8b5c37580 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc04f7fe640 1 -- 172.21.15.82:0/1274731555 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068036620 con 0x55c8b5c37580 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc04f7fe640 1 -- 172.21.15.82:0/1274731555 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068005990 con 0x55c8b5c37580 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc077b68740 1 -- 172.21.15.82:0/1274731555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 msgr2=0x55c8b5c0f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc077b68740 1 --2- 172.21.15.82:0/1274731555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c0f590 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc068008010 tx=0x7fc06803f000 comp rx=0 tx=0).stop 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc077b68740 1 -- 172.21.15.82:0/1274731555 shutdown_connections 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc077b68740 1 --2- 172.21.15.82:0/1274731555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c55540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc077b68740 1 --2- 172.21.15.82:0/1274731555 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c3a4f0 0x55c8b5c55000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc077b68740 1 --2- 172.21.15.82:0/1274731555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c37580 0x55c8b5c0f590 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.099+0000 7fc077b68740 1 -- 172.21.15.82:0/1274731555 >> 172.21.15.82:0/1274731555 conn(0x55c8b5bd7400 msgr2=0x55c8b5c57210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.100+0000 7fc077b68740 1 -- 172.21.15.82:0/1274731555 shutdown_connections 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.100+0000 7fc077b68740 1 -- 172.21.15.82:0/1274731555 wait complete. 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.100+0000 7fc077b68740 1 -- start start 2024-03-14T12:17:09.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.100+0000 7fc077b68740 1 Processor -- start 2024-03-14T12:17:09.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c37580 0x55c8b5c09400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc077b68740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c3a4f0 0x55c8b5c09940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc077b68740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c4b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc077b68740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5c59620 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5c4d750 con 0x55c8b5c3a8d0 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc077b68740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5c3bda0 con 0x55c8b5c37580 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc077b68740 10 client.0 _ll_register_callbacks cb 0x7fc0580071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c3a4f0 0x55c8b5c09940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c3a4f0 0x55c8b5c09940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46420/0 (socket says 172.21.15.82:46420) 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 -- 172.21.15.82:0/3756971556 learned_addr learned my addr 172.21.15.82:0/3756971556 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06e59c640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c37580 0x55c8b5c09400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 -- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c37580 msgr2=0x55c8b5c09400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c37580 0x55c8b5c09400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 -- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 msgr2=0x55c8b5c4b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c4b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 -- 172.21.15.82:0/3756971556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8b5c52bb0 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06ed9d640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06e59c640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c37580 0x55c8b5c09400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc06dd9b640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c3a4f0 0x55c8b5c09940 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc058015040 tx=0x7fc0580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04b2a4bec6221b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.101+0000 7fc04d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc04d7fa640 1 -- 172.21.15.82:0/3756971556 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05802a060 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc04d7fa640 1 -- 172.21.15.82:0/3756971556 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc058025b70 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc068036620 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc04d7fa640 1 -- 172.21.15.82:0/3756971556 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc058030050 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 10 client.6642 fetch_fsmap learned FSMap version 34 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c8b5cad500 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc04d7fa640 1 -- 172.21.15.82:0/3756971556 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068036620 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc04d7fa640 1 -- 172.21.15.82:0/3756971556 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc058025070 con 0x55c8b5c3a4f0 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 10 client.6642 fetch_fsmap finished waiting for FSMap version 34 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 10 client.6642 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 20 client.6642 populate_metadata read hostname 'smithi082' 2024-03-14T12:17:09.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5ca8e50 0x55c8b5ca9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 4 client.6642 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 --> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c8b5b7c050 con 0x55c8b5ca8e50 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.102+0000 7fc06e59c640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5ca8e50 0x55c8b5ca9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc04d7fa640 10 client.6642 ms_handle_connect on v2:172.21.15.164:6832/4033072140 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc06e59c640 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5ca8e50 0x55c8b5ca9230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe1a424ce39161c server_cookie=f17f381ee881dfd0 in_seq=0 out_seq=0 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc04d7fa640 1 -- 172.21.15.82:0/3756971556 <== mds.0 v2:172.21.15.164:6832/4033072140 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c8b5b7c050 con 0x55c8b5ca8e50 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc04d7fa640 10 client.6642 handle_command_reply: tid=0 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 client.6642 shutdown 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c3a4f0 msgr2=0x55c8b5c09940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c3a4f0 0x55c8b5c09940 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc058015040 tx=0x7fc0580046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 shutdown_connections 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:6832/4033072140,v1:172.21.15.164:6834/4033072140] conn(0x55c8b5ca8e50 0x55c8b5ca9230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8b5c3a8d0 0x55c8b5c4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8b5c3a4f0 0x55c8b5c09940 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 --2- 172.21.15.82:0/3756971556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c8b5c37580 0x55c8b5c09400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc04d7fa640 0 client.6642 ms_handle_reset on v2:172.21.15.164:6832/4033072140 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 >> 172.21.15.82:0/3756971556 conn(0x55c8b5bd7400 msgr2=0x55c8b5c11050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 shutdown_connections 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.103+0000 7fc077b68740 1 -- 172.21.15.82:0/3756971556 wait complete. 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc077b68740 20 client.6642 trim_cache size 0 max 16384 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:17:09.109 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2", 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 6486, 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 8.9536664259999998, 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 34, 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 69, 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 69, 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 13.989154041000001 2024-03-14T12:17:09.110 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:17:09.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc05006bbd0 msgr2=0x7fc05006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc05006bbd0 0x7fc05006e090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc068038000 tx=0x7fc068032000 comp rx=0 tx=0).stop 2024-03-14T12:17:09.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07015e670 msgr2=0x7fc07013e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07015e670 0x7fc07013e6a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cc30 tx=0x7fc06000a5c0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 shutdown_connections 2024-03-14T12:17:09.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07015e670 0x7fc07013e6a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc070143f60 0x7fc07015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc05006bbd0 0x7fc05006e090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 --2- 172.21.15.82:0/2208198637 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc070071740 0x7fc07015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 >> 172.21.15.82:0/2208198637 conn(0x7fc0700b1c70 msgr2=0x7fc07015a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 shutdown_connections 2024-03-14T12:17:09.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.104+0000 7fc074eb1640 1 -- 172.21.15.82:0/2208198637 wait complete. 2024-03-14T12:17:09.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:17:09.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.216+0000 7f37604d7640 1 Processor -- start 2024-03-14T12:17:09.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.216+0000 7f37604d7640 1 -- start start 2024-03-14T12:17:09.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.217+0000 7f37604d7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 0x7f37580b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.217+0000 7f37604d7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37580b60f0 con 0x7f37580b5bb0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.217+0000 7f37604d7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37580b63d0 con 0x7f3758162190 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.217+0000 7f37604d7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580b6d00 con 0x7f37581551d0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.217+0000 7f375da72640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f37580b5bb0 0x7f3758161a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44530/0 (socket says 172.21.15.82:44530) 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.217+0000 7f375da72640 1 -- 172.21.15.82:0/2103936913 learned_addr learned my addr 172.21.15.82:0/2103936913 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375e273640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 0x7f37580b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500962631 0 0) 0x7f37580b60f0 con 0x7f37580b5bb0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3744003520 con 0x7f37580b5bb0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428602360 0 0) 0x7f3744003520 con 0x7f37580b5bb0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 msgr2=0x7f37580b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 0x7f37580b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 >> v1:172.21.15.164:6789/0 conn(0x7f3758162190 legacy=0x7f375806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37580b66b0 con 0x7f37580b5bb0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f3748001670 con 0x7f37580b5bb0 2024-03-14T12:17:09.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 >> v1:172.21.15.82:6789/0 conn(0x7f37580b5bb0 legacy=0x7f3758161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37440039e0 0x7f3744003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 0x7f3744004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3744004d40 0x7f37440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374400b9c0 con 0x7f3744004d40 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374400bb40 con 0x7f37440039e0 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374400bd40 con 0x7f37581551d0 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375ea74640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3744004d40 0x7f37440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375e273640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37440039e0 0x7f3744003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375da72640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 0x7f3744004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375ea74640 1 -- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 msgr2=0x7f3744004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375ea74640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37581551d0 0x7f3744004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375ea74640 1 -- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37440039e0 msgr2=0x7f3744003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375ea74640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37440039e0 0x7f3744003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.218+0000 7f375ea74640 1 -- 172.21.15.82:0/2103936913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744003520 con 0x7f3744004d40 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.219+0000 7f375ea74640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3744004d40 0x7f37440051c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3750001f60 tx=0x7f3744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e18bd05fbe26ee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.219+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750001b30 con 0x7f3744004d40 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.219+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375002d710 con 0x7f3744004d40 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.219+0000 7f375d271640 1 -- 172.21.15.82:0/2103936913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375002db00 con 0x7f3744004d40 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.220+0000 7f37604d7640 1 -- 172.21.15.82:0/2103936913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3744004d40 msgr2=0x7f37440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.220+0000 7f37604d7640 1 --2- 172.21.15.82:0/2103936913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3744004d40 0x7f37440051c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3750001f60 tx=0x7f3744005830 comp rx=0 tx=0).stop 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.220+0000 7f375ea74640 1 -- 172.21.15.82:0/2103936913 reap_dead start 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.220+0000 7f37604d7640 1 -- 172.21.15.82:0/2103936913 shutdown_connections 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.220+0000 7f37604d7640 1 -- 172.21.15.82:0/2103936913 >> 172.21.15.82:0/2103936913 conn(0x7f3758072ee0 msgr2=0x7f375809a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.220+0000 7f37604d7640 1 -- 172.21.15.82:0/2103936913 shutdown_connections 2024-03-14T12:17:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.220+0000 7f37604d7640 1 -- 172.21.15.82:0/2103936913 wait complete. 2024-03-14T12:17:09.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.221+0000 7f37604d7640 1 Processor -- start 2024-03-14T12:17:09.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.221+0000 7f37604d7640 1 -- start start 2024-03-14T12:17:09.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f37604d7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3744004d40 0x7f3758153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f37604d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758155a80 0x7f3758155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f37604d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3758156440 0x7f37581e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f37604d7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758160cc0 con 0x7f3758156440 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f37604d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758160b40 con 0x7f3758155a80 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f37604d7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3758160e40 con 0x7f3744004d40 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3758156440 0x7f37581e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3758156440 0x7f37581e7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46440/0 (socket says 172.21.15.82:46440) 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 -- 172.21.15.82:0/3107224338 learned_addr learned my addr 172.21.15.82:0/3107224338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375da72640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758155a80 0x7f3758155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375e273640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3744004d40 0x7f3758153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 -- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3744004d40 msgr2=0x7f3758153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3744004d40 0x7f3758153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 -- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758155a80 msgr2=0x7f3758155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758155a80 0x7f3758155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 -- 172.21.15.82:0/3107224338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37581568c0 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.222+0000 7f375ea74640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3758156440 0x7f37581e7d70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3750002080 tx=0x7f37500015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.223+0000 7f374effd640 1 -- 172.21.15.82:0/3107224338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750001b30 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.223+0000 7f374effd640 1 -- 172.21.15.82:0/3107224338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3750043070 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.223+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3758157960 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.223+0000 7f374effd640 1 -- 172.21.15.82:0/3107224338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750003b90 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.223+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3758157f20 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.224+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375804f050 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.224+0000 7f374effd640 1 -- 172.21.15.82:0/3107224338 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3750005070 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.224+0000 7f374effd640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f373c06bbd0 0x7f373c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.224+0000 7f374effd640 1 -- 172.21.15.82:0/3107224338 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37500ac890 con 0x7f3758156440 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.224+0000 7f375e273640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f373c06bbd0 0x7f373c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:09.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.225+0000 7f375e273640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f373c06bbd0 0x7f373c06e090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3754009b00 tx=0x7f3754008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:09.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.229+0000 7f374effd640 1 -- 172.21.15.82:0/3107224338 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f375007b380 con 0x7f3758156440 2024-03-14T12:17:09.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.438+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37580b6ff0 con 0x7f3758156440 2024-03-14T12:17:09.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.438+0000 7f374effd640 1 -- 172.21.15.82:0/3107224338 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f375007ac80 con 0x7f3758156440 2024-03-14T12:17:09.437 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f373c06bbd0 msgr2=0x7f373c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f373c06bbd0 0x7f373c06e090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3754009b00 tx=0x7f3754008040 comp rx=0 tx=0).stop 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3758156440 msgr2=0x7f37581e7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3758156440 0x7f37581e7d70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3750002080 tx=0x7f37500015b0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 shutdown_connections 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3758156440 0x7f37581e7d70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758155a80 0x7f3758155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f373c06bbd0 0x7f373c06e090 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 --2- 172.21.15.82:0/3107224338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3744004d40 0x7f3758153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 >> 172.21.15.82:0/3107224338 conn(0x7f3758072ee0 msgr2=0x7f375806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:09.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 shutdown_connections 2024-03-14T12:17:09.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:09.440+0000 7f37604d7640 1 -- 172.21.15.82:0/3107224338 wait complete. 2024-03-14T12:17:09.450 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:17:09.450 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:17:09.450 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:17:09.450 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T12:17:09.450 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:17:09.451 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:17:09.451 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:17:09.451 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:17:09.451 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:17:09.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.451 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:17:09.466 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:17:09.466 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:09.467 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c443:69ff:fec0:3488/64 scope link 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:17:09.468 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:17:09.469 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::887e:60ff:fe47:4333/64 scope link 2024-03-14T12:17:09.469 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:09.469 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:17:09.469 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a6:e3:19:81:dc:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:17:09.469 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a4e3:19ff:fe81:dc41/64 scope link 2024-03-14T12:17:09.469 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:09.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.469 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:17:09.532 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:17:09.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.534 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:17:09.589 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:17:09.590 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:17:09.590 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:17:09.590 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:17:09.590 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:17:09.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.590 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:17:09.590 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:17:09.590 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:17:09.590 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:17:09.591 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:17:09.591 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:17:09.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.685 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:17:09.699 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-14T12:17:09.700 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:17:09.700 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:17:09.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.700 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-14T12:17:09.767 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:17:09.767 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T12:17:09.821 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:17:09.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:17:09.822 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:17:09.876 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T12:17:09.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.876 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:17:09.960 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:17:09.960 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:17:09.967 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:17:09.967 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:17:09.968 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:17:09.970 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:17:09.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.970 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:17:09.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:09.997 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:17:10.061 INFO:teuthology.orchestra.run.smithi082.stdout:client6651 2024-03-14T12:17:10.061 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:17:10.061 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:17:10.116 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:17:10.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.257+0000 7fbe786c6640 1 Processor -- start 2024-03-14T12:17:10.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.257+0000 7fbe786c6640 1 -- start start 2024-03-14T12:17:10.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe786c6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe700b4b70 0x7fbe70070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:10.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe786c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe7015dde0 0x7fbe7015e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:10.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe786c6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe700b3da0 con 0x7fbe70151230 2024-03-14T12:17:10.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe786c6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7015e790 con 0x7fbe700b4b70 2024-03-14T12:17:10.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe786c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7006c3b0 con 0x7fbe7015dde0 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe75c61640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe700b4b70 0x7fbe70070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe75c61640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe700b4b70 0x7fbe70070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46446/0 (socket says 172.21.15.82:46446) 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe75c61640 1 -- 172.21.15.82:0/2063395847 learned_addr learned my addr 172.21.15.82:0/2063395847 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe75c61640 1 -- 172.21.15.82:0/2063395847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe7015dde0 msgr2=0x7fbe7015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe75c61640 1 --2- 172.21.15.82:0/2063395847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe7015dde0 0x7fbe7015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe75c61640 1 -- 172.21.15.82:0/2063395847 >> v1:172.21.15.164:6789/0 conn(0x7fbe70151230 legacy=0x7fbe700cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.258+0000 7fbe75c61640 1 -- 172.21.15.82:0/2063395847 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe700b4080 con 0x7fbe700b4b70 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.259+0000 7fbe75c61640 1 --2- 172.21.15.82:0/2063395847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe700b4b70 0x7fbe70070630 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a9bd961d1642f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.259+0000 7fbe75460640 1 -- 172.21.15.82:0/2063395847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60035020 con 0x7fbe700b4b70 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.259+0000 7fbe75460640 1 -- 172.21.15.82:0/2063395847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe60031640 con 0x7fbe700b4b70 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.259+0000 7fbe75460640 1 -- 172.21.15.82:0/2063395847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6002d040 con 0x7fbe700b4b70 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 -- 172.21.15.82:0/2063395847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe700b4b70 msgr2=0x7fbe70070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 --2- 172.21.15.82:0/2063395847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe700b4b70 0x7fbe70070630 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 -- 172.21.15.82:0/2063395847 shutdown_connections 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 --2- 172.21.15.82:0/2063395847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe7015dde0 0x7fbe7015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 --2- 172.21.15.82:0/2063395847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe700b4b70 0x7fbe70070630 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 -- 172.21.15.82:0/2063395847 >> 172.21.15.82:0/2063395847 conn(0x7fbe700b1c70 msgr2=0x7fbe700cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 -- 172.21.15.82:0/2063395847 shutdown_connections 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.260+0000 7fbe786c6640 1 -- 172.21.15.82:0/2063395847 wait complete. 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.261+0000 7fbe786c6640 1 Processor -- start 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.261+0000 7fbe786c6640 1 -- start start 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.261+0000 7fbe786c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe700b4b70 0x7fbe70143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.261+0000 7fbe786c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe70151230 0x7fbe701452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.261+0000 7fbe786c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe7015dde0 0x7fbe7013e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe786c6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70151610 con 0x7fbe7015dde0 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe786c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7006c690 con 0x7fbe700b4b70 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe786c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70070b70 con 0x7fbe70151230 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe7015dde0 0x7fbe7013e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe7015dde0 0x7fbe7013e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46460/0 (socket says 172.21.15.82:46460) 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 -- 172.21.15.82:0/546956682 learned_addr learned my addr 172.21.15.82:0/546956682 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 -- 172.21.15.82:0/546956682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe70151230 msgr2=0x7fbe701452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe70151230 0x7fbe701452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 -- 172.21.15.82:0/546956682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe700b4b70 msgr2=0x7fbe70143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe700b4b70 0x7fbe70143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 -- 172.21.15.82:0/546956682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe701459f0 con 0x7fbe7015dde0 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe76c63640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe7015dde0 0x7fbe7013e560 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b970 tx=0x7fbe6800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe66ffd640 1 -- 172.21.15.82:0/546956682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68019070 con 0x7fbe7015dde0 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe66ffd640 1 -- 172.21.15.82:0/546956682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6800cb60 con 0x7fbe7015dde0 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe70145cd0 con 0x7fbe7015dde0 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe66ffd640 1 -- 172.21.15.82:0/546956682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68014440 con 0x7fbe7015dde0 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.262+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe70146290 con 0x7fbe7015dde0 2024-03-14T12:17:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.263+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7004f050 con 0x7fbe7015dde0 2024-03-14T12:17:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.264+0000 7fbe66ffd640 1 -- 172.21.15.82:0/546956682 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbe6801c070 con 0x7fbe7015dde0 2024-03-14T12:17:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.264+0000 7fbe66ffd640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe5406bbd0 0x7fbe5406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.264+0000 7fbe76462640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe5406bbd0 0x7fbe5406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.264+0000 7fbe66ffd640 1 -- 172.21.15.82:0/546956682 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe6808a7d0 con 0x7fbe7015dde0 2024-03-14T12:17:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.264+0000 7fbe76462640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe5406bbd0 0x7fbe5406e090 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c001560 tx=0x7fbe6c007990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:10.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.268+0000 7fbe66ffd640 1 -- 172.21.15.82:0/546956682 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fbe680592c0 con 0x7fbe7015dde0 2024-03-14T12:17:10.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.478+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbe7006d180 con 0x7fbe7015dde0 2024-03-14T12:17:10.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.478+0000 7fbe66ffd640 1 -- 172.21.15.82:0/546956682 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbe68058bc0 con 0x7fbe7015dde0 2024-03-14T12:17:10.478 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe5406bbd0 msgr2=0x7fbe5406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe5406bbd0 0x7fbe5406e090 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c001560 tx=0x7fbe6c007990 comp rx=0 tx=0).stop 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe7015dde0 msgr2=0x7fbe7013e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe7015dde0 0x7fbe7013e560 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b970 tx=0x7fbe6800be30 comp rx=0 tx=0).stop 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 shutdown_connections 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe7015dde0 0x7fbe7013e560 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe5406bbd0 0x7fbe5406e090 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe70151230 0x7fbe701452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 --2- 172.21.15.82:0/546956682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe700b4b70 0x7fbe70143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 >> 172.21.15.82:0/546956682 conn(0x7fbe700b1c70 msgr2=0x7fbe70159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 shutdown_connections 2024-03-14T12:17:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:10.480+0000 7fbe786c6640 1 -- 172.21.15.82:0/546956682 wait complete. 2024-03-14T12:17:10.490 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:17:10.490 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:17:10.490 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-14T12:17:10.490 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T12:17:10.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T12:17:10.491 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:17:10.491 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:17:10.491 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:17:10.491 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:17:10.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:10.491 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-14T12:17:10.510 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:17:10.510 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:17:10.510 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:17:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42526sec preferred_lft 42526sec 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::882:aaff:fed8:c2be/64 scope link 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether de:5a:02:9f:88:ac brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::dc5a:2ff:fe9f:88ac/64 scope link 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:17:10.512 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether f6:de:82:8b:5d:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-14T12:17:10.513 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::f4de:82ff:fe8b:5dfb/64 scope link 2024-03-14T12:17:10.513 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:17:10.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:10.513 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:17:10.580 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-14T12:17:10.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:10.582 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-14T12:17:10.640 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-14T12:17:10.640 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:17:10.640 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:17:10.641 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:17:10.641 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:17:10.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:10.641 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec bash -c ' 2024-03-14T12:17:10.642 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-14T12:17:10.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:17:10.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:17:10.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:17:10.642 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-14T12:17:10.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:10.741 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:17:10.756 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-14T12:17:10.756 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T12:17:10.756 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:17:10.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:10.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-03-14T12:17:10.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:17:10.824 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T12:17:10.879 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:17:10.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:17:10.879 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:17:10.934 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T12:17:10.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:10.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:17:11.019 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:17:11.019 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:17:11.027 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:17:11.027 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:17:11.027 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:17:11.027 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with new device syntax: 1@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:17:11.027 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:17:11.027 INFO:teuthology.orchestra.run.smithi164.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-14T12:17:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:17:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:17:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:17:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:17:11.029 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:17:11.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:11.029 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T12:17:11.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:11.063 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:17:11.119 INFO:teuthology.orchestra.run.smithi164.stdout:client6660 2024-03-14T12:17:11.120 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:17:11.120 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:17:11.174 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi082.front.sepia.ceph.com... 2024-03-14T12:17:11.174 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:17:11.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:11.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 down 2024-03-14T12:17:11.210 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:17:11.210 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:17:11.210 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:17:11.268 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:17:11.268 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:17:11.326 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:17:11.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.475+0000 7ff3a8061640 1 Processor -- start 2024-03-14T12:17:11.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.475+0000 7ff3a8061640 1 -- start start 2024-03-14T12:17:11.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.476+0000 7ff3a8061640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a0161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.476+0000 7ff3a8061640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff3a0162010 0x7ff3a006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.476+0000 7ff3a8061640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a006fa90 con 0x7ff3a015e150 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.476+0000 7ff3a8061640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a006c750 con 0x7ff3a0162010 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.476+0000 7ff3a8061640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a006be20 con 0x7ff3a00b4220 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.476+0000 7ff3a5dfd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff3a015e150 0x7ff3a006f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49308/0 (socket says 172.21.15.82:49308) 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.476+0000 7ff3a5dfd640 1 -- 172.21.15.82:0/536396062 learned_addr learned my addr 172.21.15.82:0/536396062 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a55fc640 1 --2- 172.21.15.82:0/536396062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a0161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a65fe640 1 --2- 172.21.15.82:0/536396062 >> v2:172.21.15.164:3300/0 conn(0x7ff3a0162010 0x7ff3a006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a4dfb640 1 -- 172.21.15.82:0/536396062 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652730870 0 0) 0x7ff3a006fa90 con 0x7ff3a015e150 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a4dfb640 1 -- 172.21.15.82:0/536396062 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff38c003520 con 0x7ff3a015e150 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a55fc640 1 -- 172.21.15.82:0/536396062 >> v2:172.21.15.164:3300/0 conn(0x7ff3a0162010 msgr2=0x7ff3a006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a55fc640 1 --2- 172.21.15.82:0/536396062 >> v2:172.21.15.164:3300/0 conn(0x7ff3a0162010 0x7ff3a006cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a55fc640 1 -- 172.21.15.82:0/536396062 >> v1:172.21.15.82:6789/0 conn(0x7ff3a015e150 legacy=0x7ff3a006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a55fc640 1 -- 172.21.15.82:0/536396062 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a006c100 con 0x7ff3a00b4220 2024-03-14T12:17:11.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.477+0000 7ff3a65fe640 1 --2- 172.21.15.82:0/536396062 >> v2:172.21.15.164:3300/0 conn(0x7ff3a0162010 0x7ff3a006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a55fc640 1 --2- 172.21.15.82:0/536396062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a0161ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff3900097a0 tx=0x7ff39002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8286d9c0bc6ac493 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a4dfb640 1 -- 172.21.15.82:0/536396062 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390035020 con 0x7ff3a00b4220 2024-03-14T12:17:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a4dfb640 1 -- 172.21.15.82:0/536396062 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff390031830 con 0x7ff3a00b4220 2024-03-14T12:17:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a4dfb640 1 -- 172.21.15.82:0/536396062 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390031be0 con 0x7ff3a00b4220 2024-03-14T12:17:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a8061640 1 -- 172.21.15.82:0/536396062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 msgr2=0x7ff3a0161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a8061640 1 --2- 172.21.15.82:0/536396062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a0161ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff3900097a0 tx=0x7ff39002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a8061640 1 -- 172.21.15.82:0/536396062 shutdown_connections 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a8061640 1 --2- 172.21.15.82:0/536396062 >> v2:172.21.15.164:3300/0 conn(0x7ff3a0162010 0x7ff3a006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.478+0000 7ff3a8061640 1 --2- 172.21.15.82:0/536396062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a0161ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.479+0000 7ff3a8061640 1 -- 172.21.15.82:0/536396062 >> 172.21.15.82:0/536396062 conn(0x7ff3a00b3670 msgr2=0x7ff3a00b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.479+0000 7ff3a8061640 1 -- 172.21.15.82:0/536396062 shutdown_connections 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.479+0000 7ff3a8061640 1 -- 172.21.15.82:0/536396062 wait complete. 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 Processor -- start 2024-03-14T12:17:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 -- start start 2024-03-14T12:17:11.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a01f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3a015e150 0x7ff3a00b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3a0162010 0x7ff3a00b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a006d2a0 con 0x7ff3a015e150 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a006d120 con 0x7ff3a0162010 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.480+0000 7ff3a8061640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a006d420 con 0x7ff3a00b4220 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3a015e150 0x7ff3a00b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3a015e150 0x7ff3a00b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35384/0 (socket says 172.21.15.82:35384) 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 -- 172.21.15.82:0/1541947145 learned_addr learned my addr 172.21.15.82:0/1541947145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a5dfd640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a01f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a65fe640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3a0162010 0x7ff3a00b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 -- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 msgr2=0x7ff3a01f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a01f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 -- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3a0162010 msgr2=0x7ff3a00b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3a0162010 0x7ff3a00b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 -- 172.21.15.82:0/1541947145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00b5290 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a55fc640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3a015e150 0x7ff3a00b49b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff39002bee0 tx=0x7ff3900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3967fc640 1 -- 172.21.15.82:0/1541947145 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390035040 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3967fc640 1 -- 172.21.15.82:0/1541947145 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39003f070 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a00b5570 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3967fc640 1 -- 172.21.15.82:0/1541947145 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39003ba80 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.481+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a00b5b30 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.482+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a004f050 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.483+0000 7ff3967fc640 1 -- 172.21.15.82:0/1541947145 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff390038020 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.483+0000 7ff3967fc640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff38406bbd0 0x7ff38406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.483+0000 7ff3a5dfd640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff38406bbd0 0x7ff38406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.483+0000 7ff3967fc640 1 -- 172.21.15.82:0/1541947145 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3900ac3d0 con 0x7ff3a015e150 2024-03-14T12:17:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.484+0000 7ff3a5dfd640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff38406bbd0 0x7ff38406e090 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005c00 tx=0x7ff39c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.488+0000 7ff3967fc640 1 -- 172.21.15.82:0/1541947145 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff39007a900 con 0x7ff3a015e150 2024-03-14T12:17:11.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.662+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3a00af560 con 0x7ff3a015e150 2024-03-14T12:17:11.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.663+0000 7ff3967fc640 1 -- 172.21.15.82:0/1541947145 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16656 (secure 0 0 0) 0x7ff39007e9a0 con 0x7ff3a015e150 2024-03-14T12:17:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:11.663 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:17:01.141090+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:17:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.665+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff38406bbd0 msgr2=0x7ff38406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.665+0000 7ff3a8061640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff38406bbd0 0x7ff38406e090 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005c00 tx=0x7ff39c009040 comp rx=0 tx=0).stop 2024-03-14T12:17:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.665+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3a015e150 msgr2=0x7ff3a00b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.665+0000 7ff3a8061640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3a015e150 0x7ff3a00b49b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff39002bee0 tx=0x7ff3900046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 shutdown_connections 2024-03-14T12:17:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3a0162010 0x7ff3a00b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3a015e150 0x7ff3a00b49b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff38406bbd0 0x7ff38406e090 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 --2- 172.21.15.82:0/1541947145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3a00b4220 0x7ff3a01f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 >> 172.21.15.82:0/1541947145 conn(0x7ff3a00b3670 msgr2=0x7ff3a0160540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 shutdown_connections 2024-03-14T12:17:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.666+0000 7ff3a8061640 1 -- 172.21.15.82:0/1541947145 wait complete. 2024-03-14T12:17:11.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:57.716705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:58.458303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:17:11.677 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:17:11.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.782+0000 7faed6df1640 1 Processor -- start 2024-03-14T12:17:11.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.782+0000 7faed6df1640 1 -- start start 2024-03-14T12:17:11.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.782+0000 7faed6df1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed006daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.782+0000 7faed6df1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faed00cf180 0x7faed015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.782+0000 7faed6df1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00b2f70 con 0x7faed015e3a0 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.782+0000 7faed6df1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed0154db0 con 0x7faed00cf180 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.782+0000 7faed6df1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed006d610 con 0x7faed01549d0 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faed4b8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed006daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faecffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faed00cf180 0x7faed015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faed4b8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed006daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:37970/0 (socket says 172.21.15.82:37970) 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faed4b8d640 1 -- 172.21.15.82:0/3760633309 learned_addr learned my addr 172.21.15.82:0/3760633309 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faed4b8d640 1 -- 172.21.15.82:0/3760633309 >> v2:172.21.15.164:3300/0 conn(0x7faed00cf180 msgr2=0x7faed015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faed4b8d640 1 --2- 172.21.15.82:0/3760633309 >> v2:172.21.15.164:3300/0 conn(0x7faed00cf180 0x7faed015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faed4b8d640 1 -- 172.21.15.82:0/3760633309 >> v1:172.21.15.164:6789/0 conn(0x7faed015e3a0 legacy=0x7faed00707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.783+0000 7faed4b8d640 1 -- 172.21.15.82:0/3760633309 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00b29b0 con 0x7faed01549d0 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.784+0000 7faed4b8d640 1 --2- 172.21.15.82:0/3760633309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed006daa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faec4009870 tx=0x7faec402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60607e7d3ca0323c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.784+0000 7faecf7fe640 1 -- 172.21.15.82:0/3760633309 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec402f840 con 0x7faed01549d0 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.784+0000 7faecf7fe640 1 -- 172.21.15.82:0/3760633309 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec402f9e0 con 0x7faed01549d0 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.784+0000 7faecf7fe640 1 -- 172.21.15.82:0/3760633309 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec402fdb0 con 0x7faed01549d0 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.784+0000 7faed6df1640 1 -- 172.21.15.82:0/3760633309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 msgr2=0x7faed006daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.784+0000 7faed6df1640 1 --2- 172.21.15.82:0/3760633309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed006daa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faec4009870 tx=0x7faec402ba20 comp rx=0 tx=0).stop 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.785+0000 7faed6df1640 1 -- 172.21.15.82:0/3760633309 shutdown_connections 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.785+0000 7faed6df1640 1 --2- 172.21.15.82:0/3760633309 >> v2:172.21.15.164:3300/0 conn(0x7faed00cf180 0x7faed015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.785+0000 7faed6df1640 1 --2- 172.21.15.82:0/3760633309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed006daa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.785+0000 7faed6df1640 1 -- 172.21.15.82:0/3760633309 >> 172.21.15.82:0/3760633309 conn(0x7faed006aef0 msgr2=0x7faed006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.785+0000 7faed6df1640 1 -- 172.21.15.82:0/3760633309 shutdown_connections 2024-03-14T12:17:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.785+0000 7faed6df1640 1 -- 172.21.15.82:0/3760633309 wait complete. 2024-03-14T12:17:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 Processor -- start 2024-03-14T12:17:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 -- start start 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed00cf180 0x7faed0145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed015e3a0 0x7faed014f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed006d2a0 con 0x7faed015e3a0 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed006cfc0 con 0x7faed00cf180 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.786+0000 7faed6df1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed015ec30 con 0x7faed01549d0 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed015e3a0 0x7faed014f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed015e3a0 0x7faed014f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35400/0 (socket says 172.21.15.82:35400) 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 -- 172.21.15.82:0/3919255561 learned_addr learned my addr 172.21.15.82:0/3919255561 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faecffff640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed4b8d640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed00cf180 0x7faed0145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 -- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 msgr2=0x7faed014f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed014f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 -- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed00cf180 msgr2=0x7faed0145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed00cf180 0x7faed0145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 -- 172.21.15.82:0/3919255561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0152cf0 con 0x7faed015e3a0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed4b8d640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed00cf180 0x7faed0145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed538e640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed015e3a0 0x7faed014f770 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7faec800c7a0 tx=0x7faec800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faecd7fa640 1 -- 172.21.15.82:0/3919255561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8003b70 con 0x7faed015e3a0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faecd7fa640 1 -- 172.21.15.82:0/3919255561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec8003d30 con 0x7faed015e3a0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed0152fd0 con 0x7faed015e3a0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faecd7fa640 1 -- 172.21.15.82:0/3919255561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800aa90 con 0x7faed015e3a0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.787+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed0153590 con 0x7faed015e3a0 2024-03-14T12:17:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.788+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed004f050 con 0x7faed015e3a0 2024-03-14T12:17:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.789+0000 7faecd7fa640 1 -- 172.21.15.82:0/3919255561 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faec800ac30 con 0x7faed015e3a0 2024-03-14T12:17:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.789+0000 7faecd7fa640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faeb406bbd0 0x7faeb406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.789+0000 7faecd7fa640 1 -- 172.21.15.82:0/3919255561 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faec808a840 con 0x7faed015e3a0 2024-03-14T12:17:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.789+0000 7faed4b8d640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faeb406bbd0 0x7faeb406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.790+0000 7faed4b8d640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faeb406bbd0 0x7faeb406e090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faec4036000 tx=0x7faec4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.794+0000 7faecd7fa640 1 -- 172.21.15.82:0/3919255561 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7faec80593c0 con 0x7faed015e3a0 2024-03-14T12:17:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.962+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faed006e5e0 con 0x7faed015e3a0 2024-03-14T12:17:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.963+0000 7faecd7fa640 1 -- 172.21.15.82:0/3919255561 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7faec8058cc0 con 0x7faed015e3a0 2024-03-14T12:17:11.963 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:17:11.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faeb406bbd0 msgr2=0x7faeb406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faeb406bbd0 0x7faeb406e090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faec4036000 tx=0x7faec4008030 comp rx=0 tx=0).stop 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed015e3a0 msgr2=0x7faed014f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed015e3a0 0x7faed014f770 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7faec800c7a0 tx=0x7faec800cc60 comp rx=0 tx=0).stop 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 shutdown_connections 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed015e3a0 0x7faed014f770 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faeb406bbd0 0x7faeb406e090 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faed01549d0 0x7faed014f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 --2- 172.21.15.82:0/3919255561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed00cf180 0x7faed0145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 >> 172.21.15.82:0/3919255561 conn(0x7faed006aef0 msgr2=0x7faed00af600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 shutdown_connections 2024-03-14T12:17:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:11.965+0000 7faed6df1640 1 -- 172.21.15.82:0/3919255561 wait complete. 2024-03-14T12:17:11.976 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:17:12.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.079+0000 7f91712a1640 1 Processor -- start 2024-03-14T12:17:12.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.079+0000 7f91712a1640 1 -- start start 2024-03-14T12:17:12.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.080+0000 7f91712a1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.080+0000 7f91712a1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0b5a30 con 0x7f916c0cd960 2024-03-14T12:17:12.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.080+0000 7f91712a1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0ce380 con 0x7f916c15bde0 2024-03-14T12:17:12.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.080+0000 7f91712a1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cef90 con 0x7f916c0d08c0 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.080+0000 7f916a7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f916c0cd960 0x7f916c0d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49310/0 (socket says 172.21.15.82:49310) 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.080+0000 7f916a7fc640 1 -- 172.21.15.82:0/1842389027 learned_addr learned my addr 172.21.15.82:0/1842389027 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.080+0000 7f916b7fe640 1 --2- 172.21.15.82:0/1842389027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.081+0000 7f9169ffb640 1 -- 172.21.15.82:0/1842389027 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958191481 0 0) 0x7f9160001110 con 0x7f916c0cd960 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.081+0000 7f9169ffb640 1 -- 172.21.15.82:0/1842389027 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9154003520 con 0x7f916c0cd960 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.081+0000 7f916b7fe640 1 -- 172.21.15.82:0/1842389027 >> v1:172.21.15.164:6789/0 conn(0x7f916c15bde0 legacy=0x7f916c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.081+0000 7f916b7fe640 1 -- 172.21.15.82:0/1842389027 >> v1:172.21.15.82:6789/0 conn(0x7f916c0cd960 legacy=0x7f916c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.081+0000 7f916b7fe640 1 -- 172.21.15.82:0/1842389027 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0ce660 con 0x7f916c0d08c0 2024-03-14T12:17:12.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.081+0000 7f916b7fe640 1 --2- 172.21.15.82:0/1842389027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c15c1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91640097d0 tx=0x7f916402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f694d818f19f114b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f9169ffb640 1 -- 172.21.15.82:0/1842389027 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164034070 con 0x7f916c0d08c0 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f9169ffb640 1 -- 172.21.15.82:0/1842389027 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164030880 con 0x7f916c0d08c0 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f9169ffb640 1 -- 172.21.15.82:0/1842389027 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164030c50 con 0x7f916c0d08c0 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f91712a1640 1 -- 172.21.15.82:0/1842389027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 msgr2=0x7f916c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f91712a1640 1 --2- 172.21.15.82:0/1842389027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c15c1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91640097d0 tx=0x7f916402b9c0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f91712a1640 1 -- 172.21.15.82:0/1842389027 shutdown_connections 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f91712a1640 1 --2- 172.21.15.82:0/1842389027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c15c1c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f91712a1640 1 -- 172.21.15.82:0/1842389027 >> 172.21.15.82:0/1842389027 conn(0x7f916c0d1bd0 msgr2=0x7f916c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f91712a1640 1 -- 172.21.15.82:0/1842389027 shutdown_connections 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.082+0000 7f91712a1640 1 -- 172.21.15.82:0/1842389027 wait complete. 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.083+0000 7f91712a1640 1 Processor -- start 2024-03-14T12:17:12.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.083+0000 7f91712a1640 1 -- start start 2024-03-14T12:17:12.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f91712a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c0cd960 0x7f916c1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f91712a1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f91712a1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f916c15bde0 0x7f916c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f91712a1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cf420 con 0x7f916c0cd960 2024-03-14T12:17:12.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f91712a1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cec20 con 0x7f916c15bde0 2024-03-14T12:17:12.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f91712a1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b6c20 con 0x7f916c0d08c0 2024-03-14T12:17:12.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c0cd960 0x7f916c1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c0cd960 0x7f916c1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35414/0 (socket says 172.21.15.82:35414) 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 -- 172.21.15.82:0/2240405755 learned_addr learned my addr 172.21.15.82:0/2240405755 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916a7fc640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916b7fe640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f916c15bde0 0x7f916c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 -- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 msgr2=0x7f916c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 -- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f916c15bde0 msgr2=0x7f916c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f916c15bde0 0x7f916c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916affd640 1 -- 172.21.15.82:0/2240405755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0ca300 con 0x7f916c0cd960 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.084+0000 7f916b7fe640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f916c15bde0 0x7f916c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.085+0000 7f916affd640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c0cd960 0x7f916c1591c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c760 tx=0x7f915c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.085+0000 7f91537fe640 1 -- 172.21.15.82:0/2240405755 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c016020 con 0x7f916c0cd960 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.085+0000 7f91537fe640 1 -- 172.21.15.82:0/2240405755 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c006d50 con 0x7f916c0cd960 2024-03-14T12:17:12.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.085+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0ca5e0 con 0x7f916c0cd960 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.085+0000 7f91537fe640 1 -- 172.21.15.82:0/2240405755 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c003bb0 con 0x7f916c0cd960 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.085+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0caba0 con 0x7f916c0cd960 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.086+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c04f050 con 0x7f916c0cd960 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.086+0000 7f91537fe640 1 -- 172.21.15.82:0/2240405755 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f915c00a850 con 0x7f916c0cd960 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.086+0000 7f91537fe640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f915806bbd0 0x7f915806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.087+0000 7f916a7fc640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f915806bbd0 0x7f915806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.087+0000 7f91537fe640 1 -- 172.21.15.82:0/2240405755 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f915c08a5b0 con 0x7f916c0cd960 2024-03-14T12:17:12.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.087+0000 7f916a7fc640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f915806bbd0 0x7f915806e090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f916c1f5100 tx=0x7f9160007b70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.091+0000 7f91537fe640 1 -- 172.21.15.82:0/2240405755 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f915c020020 con 0x7f916c0cd960 2024-03-14T12:17:12.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.265+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f916c0b7b50 con 0x7f916c0cd960 2024-03-14T12:17:12.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.265+0000 7f91537fe640 1 -- 172.21.15.82:0/2240405755 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f915c059130 con 0x7f916c0cd960 2024-03-14T12:17:12.265 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:17:12.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.267+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f915806bbd0 msgr2=0x7f915806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.267+0000 7f91712a1640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f915806bbd0 0x7f915806e090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f916c1f5100 tx=0x7f9160007b70 comp rx=0 tx=0).stop 2024-03-14T12:17:12.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.267+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c0cd960 msgr2=0x7f916c1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.267+0000 7f91712a1640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c0cd960 0x7f916c1591c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c760 tx=0x7f915c00cc20 comp rx=0 tx=0).stop 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 shutdown_connections 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f916c15bde0 0x7f916c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f915806bbd0 0x7f915806e090 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f916c0d08c0 0x7f916c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 --2- 172.21.15.82:0/2240405755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c0cd960 0x7f916c1591c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 >> 172.21.15.82:0/2240405755 conn(0x7f916c0d1bd0 msgr2=0x7f916c0b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 shutdown_connections 2024-03-14T12:17:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.268+0000 7f91712a1640 1 -- 172.21.15.82:0/2240405755 wait complete. 2024-03-14T12:17:12.278 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:17:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.382+0000 7f00dffd4640 1 Processor -- start 2024-03-14T12:17:12.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.382+0000 7f00dffd4640 1 -- start start 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00dffd4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00d80d0e10 0x7f00d8168c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00dffd4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80d25a0 con 0x7f00d815dff0 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00dffd4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80d2880 con 0x7f00d80d12a0 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00dffd4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d80d2350 con 0x7f00d80d0e10 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00de571640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00d80d0e10 0x7f00d8168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00ddd70640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f00d815dff0 0x7f00d80d4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49314/0 (socket says 172.21.15.82:49314) 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00de571640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00d80d0e10 0x7f00d8168c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35422/0 (socket says 172.21.15.82:35422) 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.383+0000 7f00ddd70640 1 -- 172.21.15.82:0/3445615366 learned_addr learned my addr 172.21.15.82:0/3445615366 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.384+0000 7f00de571640 1 -- 172.21.15.82:0/3445615366 >> v1:172.21.15.164:6789/0 conn(0x7f00d80d12a0 legacy=0x7f00d80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.384+0000 7f00de571640 1 -- 172.21.15.82:0/3445615366 >> v1:172.21.15.82:6789/0 conn(0x7f00d815dff0 legacy=0x7f00d80d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.384+0000 7f00de571640 1 -- 172.21.15.82:0/3445615366 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d80d2b60 con 0x7f00d80d0e10 2024-03-14T12:17:12.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.384+0000 7f00de571640 1 --2- 172.21.15.82:0/3445615366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d80d0e10 0x7f00d8168c90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f00d80d24d0 tx=0x7f00d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586976c906469263 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.384+0000 7f00dcd6e640 1 -- 172.21.15.82:0/3445615366 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d4035020 con 0x7f00d80d0e10 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.384+0000 7f00dcd6e640 1 -- 172.21.15.82:0/3445615366 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d4031830 con 0x7f00d80d0e10 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.384+0000 7f00dcd6e640 1 -- 172.21.15.82:0/3445615366 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d4031c00 con 0x7f00d80d0e10 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.385+0000 7f00dffd4640 1 -- 172.21.15.82:0/3445615366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d80d0e10 msgr2=0x7f00d8168c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.385+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3445615366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d80d0e10 0x7f00d8168c90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f00d80d24d0 tx=0x7f00d402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.385+0000 7f00dffd4640 1 -- 172.21.15.82:0/3445615366 shutdown_connections 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.385+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3445615366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d80d0e10 0x7f00d8168c90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.385+0000 7f00dffd4640 1 -- 172.21.15.82:0/3445615366 >> 172.21.15.82:0/3445615366 conn(0x7f00d809d0d0 msgr2=0x7f00d80d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.385+0000 7f00dffd4640 1 -- 172.21.15.82:0/3445615366 shutdown_connections 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.385+0000 7f00dffd4640 1 -- 172.21.15.82:0/3445615366 wait complete. 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.386+0000 7f00dffd4640 1 Processor -- start 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.386+0000 7f00dffd4640 1 -- start start 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dffd4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00d80d0e10 0x7f00d809fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dffd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00d80d12a0 0x7f00d81530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dffd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d815dff0 0x7f00d809db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dffd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80cc4b0 con 0x7f00d815dff0 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dffd4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80cc330 con 0x7f00d80d0e10 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dffd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80cc630 con 0x7f00d80d12a0 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d815dff0 0x7f00d809db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d815dff0 0x7f00d809db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35424/0 (socket says 172.21.15.82:35424) 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 -- 172.21.15.82:0/3119236121 learned_addr learned my addr 172.21.15.82:0/3119236121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00ddd70640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00d80d0e10 0x7f00d809fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dd56f640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00d80d12a0 0x7f00d81530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 -- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00d80d12a0 msgr2=0x7f00d81530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00d80d12a0 0x7f00d81530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 -- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00d80d0e10 msgr2=0x7f00d809fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00d80d0e10 0x7f00d809fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 -- 172.21.15.82:0/3119236121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d80b90d0 con 0x7f00d815dff0 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00ddd70640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00d80d0e10 0x7f00d809fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00dd56f640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00d80d12a0 0x7f00d81530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.387+0000 7f00de571640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d815dff0 0x7f00d809db40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009890 tx=0x7f00d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.388+0000 7f00ce7fc640 1 -- 172.21.15.82:0/3119236121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d403b9c0 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.388+0000 7f00ce7fc640 1 -- 172.21.15.82:0/3119236121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d403bb60 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.388+0000 7f00ce7fc640 1 -- 172.21.15.82:0/3119236121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40453f0 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.388+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d80c8310 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.388+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d80c88d0 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.389+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d804f050 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.389+0000 7f00ce7fc640 1 -- 172.21.15.82:0/3119236121 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f00d4038020 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.389+0000 7f00ce7fc640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00bc06bbd0 0x7f00bc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.390+0000 7f00ddd70640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00bc06bbd0 0x7f00bc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.390+0000 7f00ce7fc640 1 -- 172.21.15.82:0/3119236121 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00d40ac7b0 con 0x7f00d815dff0 2024-03-14T12:17:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.390+0000 7f00ddd70640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00bc06bbd0 0x7f00bc06e090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f00d00012e0 tx=0x7f00d0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.394+0000 7f00ce7fc640 1 -- 172.21.15.82:0/3119236121 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f00d4040070 con 0x7f00d815dff0 2024-03-14T12:17:12.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.565+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00d81605f0 con 0x7f00d815dff0 2024-03-14T12:17:12.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.565+0000 7f00ce7fc640 1 -- 172.21.15.82:0/3119236121 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00d407b330 con 0x7f00d815dff0 2024-03-14T12:17:12.565 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00bc06bbd0 msgr2=0x7f00bc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00bc06bbd0 0x7f00bc06e090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f00d00012e0 tx=0x7f00d0009040 comp rx=0 tx=0).stop 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d815dff0 msgr2=0x7f00d809db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d815dff0 0x7f00d809db40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009890 tx=0x7f00d40046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 shutdown_connections 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00d815dff0 0x7f00d809db40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f00bc06bbd0 0x7f00bc06e090 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00d80d12a0 0x7f00d81530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 --2- 172.21.15.82:0/3119236121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00d80d0e10 0x7f00d809fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.567+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 >> 172.21.15.82:0/3119236121 conn(0x7f00d809d0d0 msgr2=0x7f00d806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:12.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.568+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 shutdown_connections 2024-03-14T12:17:12.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.568+0000 7f00dffd4640 1 -- 172.21.15.82:0/3119236121 wait complete. 2024-03-14T12:17:12.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:17:12.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.679+0000 7f68a0759640 1 Processor -- start 2024-03-14T12:17:12.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.679+0000 7f68a0759640 1 -- start start 2024-03-14T12:17:12.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f68a0759640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f68981551d0 0x7f68980b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f68a0759640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f68a0759640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980b60f0 con 0x7f68980b5bb0 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f68a0759640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980b6d00 con 0x7f68981551d0 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f68a0759640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b63d0 con 0x7f6898162190 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689ecf6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689e4f5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f68981551d0 0x7f68980b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689ecf6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:38038/0 (socket says 172.21.15.82:38038) 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689ecf6640 1 -- 172.21.15.82:0/754152187 learned_addr learned my addr 172.21.15.82:0/754152187 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689dcf4640 1 --1- 172.21.15.82:0/754152187 >> v1:172.21.15.164:6789/0 conn(0x7f68980b5bb0 0x7f6898161a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:53868/0 (socket says 172.21.15.82:53868) 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689ecf6640 1 -- 172.21.15.82:0/754152187 >> v2:172.21.15.164:3300/0 conn(0x7f68981551d0 msgr2=0x7f68980b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689ecf6640 1 --2- 172.21.15.82:0/754152187 >> v2:172.21.15.164:3300/0 conn(0x7f68981551d0 0x7f68980b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689ecf6640 1 -- 172.21.15.82:0/754152187 >> v1:172.21.15.164:6789/0 conn(0x7f68980b5bb0 legacy=0x7f6898161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.680+0000 7f689ecf6640 1 -- 172.21.15.82:0/754152187 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980b66b0 con 0x7f6898162190 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.681+0000 7f689ecf6640 1 --2- 172.21.15.82:0/754152187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806f330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6890009e50 tx=0x7f689002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c455a927965575cc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.681+0000 7f689d4f3640 1 -- 172.21.15.82:0/754152187 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689002cb00 con 0x7f6898162190 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.681+0000 7f689d4f3640 1 -- 172.21.15.82:0/754152187 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689002cca0 con 0x7f6898162190 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.681+0000 7f689d4f3640 1 -- 172.21.15.82:0/754152187 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890034610 con 0x7f6898162190 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 -- 172.21.15.82:0/754152187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 msgr2=0x7f689806f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 --2- 172.21.15.82:0/754152187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806f330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6890009e50 tx=0x7f689002bf60 comp rx=0 tx=0).stop 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 -- 172.21.15.82:0/754152187 shutdown_connections 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 --2- 172.21.15.82:0/754152187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806f330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 --2- 172.21.15.82:0/754152187 >> v2:172.21.15.164:3300/0 conn(0x7f68981551d0 0x7f68980b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 -- 172.21.15.82:0/754152187 >> 172.21.15.82:0/754152187 conn(0x7f6898072ee0 msgr2=0x7f689809a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 -- 172.21.15.82:0/754152187 shutdown_connections 2024-03-14T12:17:12.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.682+0000 7f68a0759640 1 -- 172.21.15.82:0/754152187 wait complete. 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.683+0000 7f68a0759640 1 Processor -- start 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.683+0000 7f68a0759640 1 -- start start 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f68a0759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68980b5bb0 0x7f68980b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f68a0759640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68981551d0 0x7f689806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f68a0759640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f68a0759640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689806f9f0 con 0x7f68981551d0 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f68a0759640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689806f870 con 0x7f68980b5bb0 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f68a0759640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689806fb70 con 0x7f6898162190 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68981551d0 0x7f689806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68981551d0 0x7f689806d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35436/0 (socket says 172.21.15.82:35436) 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 -- 172.21.15.82:0/1425391077 learned_addr learned my addr 172.21.15.82:0/1425391077 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689e4f5640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68980b5bb0 0x7f68980b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689ecf6640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 -- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 msgr2=0x7f689806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 -- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68980b5bb0 msgr2=0x7f68980b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68980b5bb0 0x7f68980b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 -- 172.21.15.82:0/1425391077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689813b2a0 con 0x7f68981551d0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.684+0000 7f689dcf4640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68981551d0 0x7f689806d090 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f688800c430 tx=0x7f688800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.685+0000 7f688effd640 1 -- 172.21.15.82:0/1425391077 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880074e0 con 0x7f68981551d0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.685+0000 7f688effd640 1 -- 172.21.15.82:0/1425391077 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888007680 con 0x7f68981551d0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.685+0000 7f688effd640 1 -- 172.21.15.82:0/1425391077 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888004a00 con 0x7f68981551d0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.685+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689813b580 con 0x7f68981551d0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.685+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689813bb40 con 0x7f68981551d0 2024-03-14T12:17:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.686+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689804f050 con 0x7f68981551d0 2024-03-14T12:17:12.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.686+0000 7f688effd640 1 -- 172.21.15.82:0/1425391077 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f688801a020 con 0x7f68981551d0 2024-03-14T12:17:12.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.686+0000 7f688effd640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f687c06bbd0 0x7f687c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:12.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.687+0000 7f689e4f5640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f687c06bbd0 0x7f687c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:12.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.687+0000 7f688effd640 1 -- 172.21.15.82:0/1425391077 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f688808a140 con 0x7f68981551d0 2024-03-14T12:17:12.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.687+0000 7f689e4f5640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f687c06bbd0 0x7f687c06e090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6894009b40 tx=0x7f6894009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:12.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.691+0000 7f688effd640 1 -- 172.21.15.82:0/1425391077 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f68880585e0 con 0x7f68981551d0 2024-03-14T12:17:12.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.874+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f689809b3a0 con 0x7f68981551d0 2024-03-14T12:17:12.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.874+0000 7f688effd640 1 -- 172.21.15.82:0/1425391077 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f688805c680 con 0x7f68981551d0 2024-03-14T12:17:12.874 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:17:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f687c06bbd0 msgr2=0x7f687c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f687c06bbd0 0x7f687c06e090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6894009b40 tx=0x7f6894009190 comp rx=0 tx=0).stop 2024-03-14T12:17:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68981551d0 msgr2=0x7f689806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68981551d0 0x7f689806d090 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f688800c430 tx=0x7f688800c8f0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 shutdown_connections 2024-03-14T12:17:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f687c06bbd0 0x7f687c06e090 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6898162190 0x7f689806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68981551d0 0x7f689806d090 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 --2- 172.21.15.82:0/1425391077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68980b5bb0 0x7f68980b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:12.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 >> 172.21.15.82:0/1425391077 conn(0x7f6898072ee0 msgr2=0x7f689806be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:12.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 shutdown_connections 2024-03-14T12:17:12.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:12.876+0000 7f68a0759640 1 -- 172.21.15.82:0/1425391077 wait complete. 2024-03-14T12:17:12.886 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T12:17:12.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:12.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-14T12:17:12.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:17:12.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:12.914 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T12:17:12.969 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi164.front.sepia.ceph.com... 2024-03-14T12:17:12.969 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T12:17:12.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:12.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 down 2024-03-14T12:17:13.003 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:17:13.003 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:17:13.003 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:17:13.061 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:17:13.061 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:17:13.118 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:17:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.220+0000 7fb863dfa640 1 Processor -- start 2024-03-14T12:17:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.221+0000 7fb863dfa640 1 -- start start 2024-03-14T12:17:13.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.221+0000 7fb863dfa640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c0af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.221+0000 7fb863dfa640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb85c06b420 0x7fb85c157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.221+0000 7fb863dfa640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb85c0b01d0 con 0x7fb85c15f1f0 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.221+0000 7fb863dfa640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c15e710 con 0x7fb85c06b420 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.221+0000 7fb863dfa640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c15e9f0 con 0x7fb85c0af150 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb861395640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb85c15f1f0 0x7fb85c06ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49322/0 (socket says 172.21.15.82:49322) 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb861395640 1 -- 172.21.15.82:0/3397138333 learned_addr learned my addr 172.21.15.82:0/3397138333 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb861b96640 1 --2- 172.21.15.82:0/3397138333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb862397640 1 --2- 172.21.15.82:0/3397138333 >> v2:172.21.15.164:3300/0 conn(0x7fb85c06b420 0x7fb85c157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb860b94640 1 -- 172.21.15.82:0/3397138333 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836987303 0 0) 0x7fb85c0b01d0 con 0x7fb85c15f1f0 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb860b94640 1 -- 172.21.15.82:0/3397138333 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb848003520 con 0x7fb85c15f1f0 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb861b96640 1 -- 172.21.15.82:0/3397138333 >> v2:172.21.15.164:3300/0 conn(0x7fb85c06b420 msgr2=0x7fb85c157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb861b96640 1 --2- 172.21.15.82:0/3397138333 >> v2:172.21.15.164:3300/0 conn(0x7fb85c06b420 0x7fb85c157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb861b96640 1 -- 172.21.15.82:0/3397138333 >> v1:172.21.15.82:6789/0 conn(0x7fb85c15f1f0 legacy=0x7fb85c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.222+0000 7fb861b96640 1 -- 172.21.15.82:0/3397138333 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c15ecd0 con 0x7fb85c0af150 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.223+0000 7fb861b96640 1 --2- 172.21.15.82:0/3397138333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c0af880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0097a0 tx=0x7fb84c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d82173899b83c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.223+0000 7fb860b94640 1 -- 172.21.15.82:0/3397138333 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c031780 con 0x7fb85c0af150 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.223+0000 7fb860b94640 1 -- 172.21.15.82:0/3397138333 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c037070 con 0x7fb85c0af150 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.223+0000 7fb860b94640 1 -- 172.21.15.82:0/3397138333 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c031b40 con 0x7fb85c0af150 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 -- 172.21.15.82:0/3397138333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 msgr2=0x7fb85c0af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 --2- 172.21.15.82:0/3397138333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c0af880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0097a0 tx=0x7fb84c02b820 comp rx=0 tx=0).stop 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 -- 172.21.15.82:0/3397138333 shutdown_connections 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 --2- 172.21.15.82:0/3397138333 >> v2:172.21.15.164:3300/0 conn(0x7fb85c06b420 0x7fb85c157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 --2- 172.21.15.82:0/3397138333 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c0af880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 -- 172.21.15.82:0/3397138333 >> 172.21.15.82:0/3397138333 conn(0x7fb85c0d0a20 msgr2=0x7fb85c0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 -- 172.21.15.82:0/3397138333 shutdown_connections 2024-03-14T12:17:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.224+0000 7fb863dfa640 1 -- 172.21.15.82:0/3397138333 wait complete. 2024-03-14T12:17:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.225+0000 7fb863dfa640 1 Processor -- start 2024-03-14T12:17:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.225+0000 7fb863dfa640 1 -- start start 2024-03-14T12:17:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06b420 0x7fb85c148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c15f1f0 0x7fb85c1e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0aff40 con 0x7fb85c06b420 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0afdc0 con 0x7fb85c15f1f0 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0b58f0 con 0x7fb85c0af150 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06b420 0x7fb85c148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06b420 0x7fb85c148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35438/0 (socket says 172.21.15.82:35438) 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 -- 172.21.15.82:0/2343999569 learned_addr learned my addr 172.21.15.82:0/2343999569 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 -- 172.21.15.82:0/2343999569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 msgr2=0x7fb85c14ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 -- 172.21.15.82:0/2343999569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c15f1f0 msgr2=0x7fb85c1e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c15f1f0 0x7fb85c1e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 -- 172.21.15.82:0/2343999569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c15d130 con 0x7fb85c06b420 2024-03-14T12:17:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb861b96640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06b420 0x7fb85c148e10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb84c034040 tx=0x7fb84c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb8527fc640 1 -- 172.21.15.82:0/2343999569 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c032610 con 0x7fb85c06b420 2024-03-14T12:17:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb8527fc640 1 -- 172.21.15.82:0/2343999569 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c0327d0 con 0x7fb85c06b420 2024-03-14T12:17:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c149da0 con 0x7fb85c06b420 2024-03-14T12:17:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb8527fc640 1 -- 172.21.15.82:0/2343999569 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c032bc0 con 0x7fb85c06b420 2024-03-14T12:17:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.226+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c14a360 con 0x7fb85c06b420 2024-03-14T12:17:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.227+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c06fb70 con 0x7fb85c06b420 2024-03-14T12:17:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.228+0000 7fb8527fc640 1 -- 172.21.15.82:0/2343999569 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb84c04e440 con 0x7fb85c06b420 2024-03-14T12:17:13.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.228+0000 7fb8527fc640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb84006bbd0 0x7fb84006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.228+0000 7fb8527fc640 1 -- 172.21.15.82:0/2343999569 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb84c0ac6a0 con 0x7fb85c06b420 2024-03-14T12:17:13.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.228+0000 7fb861395640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb84006bbd0 0x7fb84006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.228+0000 7fb861395640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb84006bbd0 0x7fb84006e090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb858000f60 tx=0x7fb8580078f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.232+0000 7fb8527fc640 1 -- 172.21.15.82:0/2343999569 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb85c06fb70 con 0x7fb85c06b420 2024-03-14T12:17:13.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.405+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb85c04f050 con 0x7fb85c06b420 2024-03-14T12:17:13.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.406+0000 7fb8527fc640 1 -- 172.21.15.82:0/2343999569 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16656 (secure 0 0 0) 0x7fb84c040070 con 0x7fb85c06b420 2024-03-14T12:17:13.406 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:13.406 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:17:01.141090+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb84006bbd0 msgr2=0x7fb84006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb84006bbd0 0x7fb84006e090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb858000f60 tx=0x7fb8580078f0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06b420 msgr2=0x7fb85c148e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06b420 0x7fb85c148e10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb84c034040 tx=0x7fb84c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 shutdown_connections 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb85c15f1f0 0x7fb85c1e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb84006bbd0 0x7fb84006e090 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb85c0af150 0x7fb85c14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 --2- 172.21.15.82:0/2343999569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb85c06b420 0x7fb85c148e10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 >> 172.21.15.82:0/2343999569 conn(0x7fb85c0d0a20 msgr2=0x7fb85c0ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:13.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.408+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 shutdown_connections 2024-03-14T12:17:13.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.409+0000 7fb863dfa640 1 -- 172.21.15.82:0/2343999569 wait complete. 2024-03-14T12:17:13.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:57.716705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:58.458303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:17:13.419 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:17:13.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.523+0000 7f64ba929640 1 Processor -- start 2024-03-14T12:17:13.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.523+0000 7f64ba929640 1 -- start start 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64ba929640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b40b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64ba929640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b40b41c0 con 0x7f64b4140e70 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64ba929640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b4161c10 con 0x7f64b4143c60 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64ba929640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b4161ef0 con 0x7f64b41626f0 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f64b4140e70 0x7f64b4143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49328/0 (socket says 172.21.15.82:49328) 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b3fff640 1 -- 172.21.15.82:0/284760502 learned_addr learned my addr 172.21.15.82:0/284760502 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b8ec6640 1 --2- 172.21.15.82:0/284760502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b40b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b2ffd640 1 -- 172.21.15.82:0/284760502 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746393753 0 0) 0x7f64b40b41c0 con 0x7f64b4140e70 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b2ffd640 1 -- 172.21.15.82:0/284760502 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f649c003520 con 0x7f64b4140e70 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b8ec6640 1 -- 172.21.15.82:0/284760502 >> v1:172.21.15.164:6789/0 conn(0x7f64b4143c60 legacy=0x7f64b40af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b2ffd640 1 -- 172.21.15.82:0/284760502 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 786560440 0 0) 0x7f649c003520 con 0x7f64b4140e70 2024-03-14T12:17:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b8ec6640 1 -- 172.21.15.82:0/284760502 >> v1:172.21.15.82:6789/0 conn(0x7f64b4140e70 legacy=0x7f64b4143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.524+0000 7f64b8ec6640 1 -- 172.21.15.82:0/284760502 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b41621d0 con 0x7f64b41626f0 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.525+0000 7f64b8ec6640 1 --2- 172.21.15.82:0/284760502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b40b3b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009dc0 tx=0x7f64ac02bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5446e2fb22a56a99 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.525+0000 7f64b2ffd640 1 -- 172.21.15.82:0/284760502 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac039070 con 0x7f64b41626f0 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.525+0000 7f64b2ffd640 1 -- 172.21.15.82:0/284760502 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ac02c770 con 0x7f64b41626f0 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.525+0000 7f64b2ffd640 1 -- 172.21.15.82:0/284760502 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4001670 con 0x7f64b41626f0 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.526+0000 7f64ba929640 1 -- 172.21.15.82:0/284760502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 msgr2=0x7f64b40b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.526+0000 7f64ba929640 1 --2- 172.21.15.82:0/284760502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b40b3b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009dc0 tx=0x7f64ac02bc40 comp rx=0 tx=0).stop 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.526+0000 7f64ba929640 1 -- 172.21.15.82:0/284760502 shutdown_connections 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.526+0000 7f64ba929640 1 --2- 172.21.15.82:0/284760502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b40b3b30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.526+0000 7f64ba929640 1 -- 172.21.15.82:0/284760502 >> 172.21.15.82:0/284760502 conn(0x7f64b40b2830 msgr2=0x7f64b40b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.526+0000 7f64ba929640 1 -- 172.21.15.82:0/284760502 shutdown_connections 2024-03-14T12:17:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.526+0000 7f64ba929640 1 -- 172.21.15.82:0/284760502 wait complete. 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.527+0000 7f64ba929640 1 Processor -- start 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.527+0000 7f64ba929640 1 -- start start 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64ba929640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b4140e70 0x7f64b41f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64ba929640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b4143c60 0x7f64b41f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64ba929640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b4140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64ba929640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b415f2c0 con 0x7f64b4140e70 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64ba929640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b415f140 con 0x7f64b4143c60 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64ba929640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b40aff80 con 0x7f64b41626f0 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64b37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b4143c60 0x7f64b41f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64b8ec6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b4140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64b8ec6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b4140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:38082/0 (socket says 172.21.15.82:38082) 2024-03-14T12:17:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64b8ec6640 1 -- 172.21.15.82:0/2674200887 learned_addr learned my addr 172.21.15.82:0/2674200887 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64b37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b4143c60 0x7f64b41f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:41282/0 (socket says 172.21.15.82:41282) 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.528+0000 7f64b3fff640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b4140e70 0x7f64b41f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b8ec6640 1 -- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b4143c60 msgr2=0x7f64b41f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b8ec6640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b4143c60 0x7f64b41f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b8ec6640 1 -- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b4140e70 msgr2=0x7f64b41f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b8ec6640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b4140e70 0x7f64b41f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b8ec6640 1 -- 172.21.15.82:0/2674200887 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4143a90 con 0x7f64b41626f0 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b37fe640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b4143c60 0x7f64b41f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b8ec6640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b4140390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64ac037040 tx=0x7f64ac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b0ff9640 1 -- 172.21.15.82:0/2674200887 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac039040 con 0x7f64b41626f0 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b0ff9640 1 -- 172.21.15.82:0/2674200887 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ac003b90 con 0x7f64b41626f0 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64b0ff9640 1 -- 172.21.15.82:0/2674200887 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac034770 con 0x7f64b41626f0 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b41431f0 con 0x7f64b41626f0 2024-03-14T12:17:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.529+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b41437b0 con 0x7f64b41626f0 2024-03-14T12:17:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.530+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b404f050 con 0x7f64b41626f0 2024-03-14T12:17:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.531+0000 7f64b0ff9640 1 -- 172.21.15.82:0/2674200887 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f64ac034910 con 0x7f64b41626f0 2024-03-14T12:17:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.531+0000 7f64b0ff9640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f64a006bbd0 0x7f64a006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.531+0000 7f64b3fff640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f64a006bbd0 0x7f64a006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.531+0000 7f64b0ff9640 1 -- 172.21.15.82:0/2674200887 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64ac0ada20 con 0x7f64b41626f0 2024-03-14T12:17:13.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.532+0000 7f64b3fff640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f64a006bbd0 0x7f64a006e090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f64a4001eb0 tx=0x7f64a4001e20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.536+0000 7f64b0ff9640 1 -- 172.21.15.82:0/2674200887 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f64ac07bfa0 con 0x7f64b41626f0 2024-03-14T12:17:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.706+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64b4074140 con 0x7f64b41626f0 2024-03-14T12:17:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.707+0000 7f64b0ff9640 1 -- 172.21.15.82:0/2674200887 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64ac080040 con 0x7f64b41626f0 2024-03-14T12:17:13.707 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f64a006bbd0 msgr2=0x7f64a006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f64a006bbd0 0x7f64a006e090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f64a4001eb0 tx=0x7f64a4001e20 comp rx=0 tx=0).stop 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 msgr2=0x7f64b4140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b4140390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64ac037040 tx=0x7f64ac004a40 comp rx=0 tx=0).stop 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 shutdown_connections 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f64a006bbd0 0x7f64a006e090 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64b41626f0 0x7f64b4140390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b4143c60 0x7f64b41f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 --2- 172.21.15.82:0/2674200887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b4140e70 0x7f64b41f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 >> 172.21.15.82:0/2674200887 conn(0x7f64b40b2830 msgr2=0x7f64b40b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 shutdown_connections 2024-03-14T12:17:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.709+0000 7f64ba929640 1 -- 172.21.15.82:0/2674200887 wait complete. 2024-03-14T12:17:13.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:17:13.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.823+0000 7fc6dcf7e640 1 Processor -- start 2024-03-14T12:17:13.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.823+0000 7fc6dcf7e640 1 -- start start 2024-03-14T12:17:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6dcf7e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc6d80d02d0 0x7fc6d81668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6dcf7e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d80b2a50 con 0x7fc6d815c2a0 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6dcf7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d80ceb30 con 0x7fc6d8145790 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6dcf7e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d80ce8e0 con 0x7fc6d80d02d0 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6d6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc6d80d02d0 0x7fc6d81668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6cffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc6d815c2a0 0x7fc6d80cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49330/0 (socket says 172.21.15.82:49330) 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6d6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc6d80d02d0 0x7fc6d81668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35444/0 (socket says 172.21.15.82:35444) 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6cffff640 1 -- 172.21.15.82:0/1599374160 learned_addr learned my addr 172.21.15.82:0/1599374160 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6d6d9d640 1 -- 172.21.15.82:0/1599374160 >> v1:172.21.15.164:6789/0 conn(0x7fc6d8145790 legacy=0x7fc6d80b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6d6d9d640 1 -- 172.21.15.82:0/1599374160 >> v1:172.21.15.82:6789/0 conn(0x7fc6d815c2a0 legacy=0x7fc6d80cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6d5d9b640 1 -- 172.21.15.82:0/1599374160 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229957505 0 0) 0x7fc6d80b2a50 con 0x7fc6d815c2a0 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.824+0000 7fc6d6d9d640 1 -- 172.21.15.82:0/1599374160 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80cee10 con 0x7fc6d80d02d0 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.825+0000 7fc6d6d9d640 1 --2- 172.21.15.82:0/1599374160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d80d02d0 0x7fc6d81668a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009d90 tx=0x7fc6d002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bced11afec9f30a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.825+0000 7fc6d5d9b640 1 -- 172.21.15.82:0/1599374160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0039070 con 0x7fc6d80d02d0 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.825+0000 7fc6d5d9b640 1 -- 172.21.15.82:0/1599374160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d002cc60 con 0x7fc6d80d02d0 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.825+0000 7fc6d5d9b640 1 -- 172.21.15.82:0/1599374160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0034660 con 0x7fc6d80d02d0 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.825+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/1599374160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d80d02d0 msgr2=0x7fc6d81668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.826+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/1599374160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d80d02d0 0x7fc6d81668a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009d90 tx=0x7fc6d002beb0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.826+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/1599374160 shutdown_connections 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.826+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/1599374160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d80d02d0 0x7fc6d81668a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.826+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/1599374160 >> 172.21.15.82:0/1599374160 conn(0x7fc6d806aef0 msgr2=0x7fc6d806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.826+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/1599374160 shutdown_connections 2024-03-14T12:17:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.826+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/1599374160 wait complete. 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.827+0000 7fc6dcf7e640 1 Processor -- start 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.827+0000 7fc6dcf7e640 1 -- start start 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d8145790 0x7fc6d8158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d815c2a0 0x7fc6d813b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80cf0f0 con 0x7fc6d8145790 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80cc600 con 0x7fc6d815c2a0 2024-03-14T12:17:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d8166de0 con 0x7fc6d80d02d0 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d8145790 0x7fc6d8158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d8145790 0x7fc6d8158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35460/0 (socket says 172.21.15.82:35460) 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 -- 172.21.15.82:0/2454854695 learned_addr learned my addr 172.21.15.82:0/2454854695 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6d659c640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6d6d9d640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d815c2a0 0x7fc6d813b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 -- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6d80d02d0 msgr2=0x7fc6d8157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 -- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d815c2a0 msgr2=0x7fc6d813b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d815c2a0 0x7fc6d813b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 -- 172.21.15.82:0/2454854695 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80b2440 con 0x7fc6d8145790 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6cffff640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d8145790 0x7fc6d8158070 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c420 tx=0x7fc6c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6ceffd640 1 -- 172.21.15.82:0/2454854695 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00074e0 con 0x7fc6d8145790 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6ceffd640 1 -- 172.21.15.82:0/2454854695 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c00047e0 con 0x7fc6d8145790 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d813c0c0 con 0x7fc6d8145790 2024-03-14T12:17:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6ceffd640 1 -- 172.21.15.82:0/2454854695 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0004c10 con 0x7fc6d8145790 2024-03-14T12:17:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.828+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d813c680 con 0x7fc6d8145790 2024-03-14T12:17:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.829+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d804f050 con 0x7fc6d8145790 2024-03-14T12:17:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.830+0000 7fc6ceffd640 1 -- 172.21.15.82:0/2454854695 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc6c0011070 con 0x7fc6d8145790 2024-03-14T12:17:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.830+0000 7fc6ceffd640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6b806bbd0 0x7fc6b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.830+0000 7fc6ceffd640 1 -- 172.21.15.82:0/2454854695 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6c008a5e0 con 0x7fc6d8145790 2024-03-14T12:17:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.830+0000 7fc6d659c640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6b806bbd0 0x7fc6b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.831+0000 7fc6d659c640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6b806bbd0 0x7fc6b806e090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80b1970 tx=0x7fc6c8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:13.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:13.835+0000 7fc6ceffd640 1 -- 172.21.15.82:0/2454854695 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc6c0059160 con 0x7fc6d8145790 2024-03-14T12:17:14.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.006+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6d80d0080 con 0x7fc6d8145790 2024-03-14T12:17:14.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.006+0000 7fc6ceffd640 1 -- 172.21.15.82:0/2454854695 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6c0058a60 con 0x7fc6d8145790 2024-03-14T12:17:14.006 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6b806bbd0 msgr2=0x7fc6b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6b806bbd0 0x7fc6b806e090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80b1970 tx=0x7fc6c8009040 comp rx=0 tx=0).stop 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d8145790 msgr2=0x7fc6d8158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d8145790 0x7fc6d8158070 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c420 tx=0x7fc6c000c8e0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 shutdown_connections 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d815c2a0 0x7fc6d813b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6d8145790 0x7fc6d8158070 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc6b806bbd0 0x7fc6b806e090 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 --2- 172.21.15.82:0/2454854695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.008+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 >> 172.21.15.82:0/2454854695 conn(0x7fc6d806aef0 msgr2=0x7fc6d80b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:14.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.009+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 shutdown_connections 2024-03-14T12:17:14.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.009+0000 7fc6dcf7e640 1 -- 172.21.15.82:0/2454854695 wait complete. 2024-03-14T12:17:14.019 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:17:14.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.123+0000 7fdadb69a640 1 Processor -- start 2024-03-14T12:17:14.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.123+0000 7fdadb69a640 1 -- start start 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdadb69a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 0x7fdad40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdadb69a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdad41478b0 0x7fdad40b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdadb69a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad406e8e0 con 0x7fdad40d0db0 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdadb69a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad406eb80 con 0x7fdad41478b0 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdadb69a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406f100 con 0x7fdad4147360 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdad9436640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdad40d0db0 0x7fdad4162560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49342/0 (socket says 172.21.15.82:49342) 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdad9436640 1 -- 172.21.15.82:0/3448371357 learned_addr learned my addr 172.21.15.82:0/3448371357 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdad9c37640 1 --2- 172.21.15.82:0/3448371357 >> v2:172.21.15.164:3300/0 conn(0x7fdad41478b0 0x7fdad40b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdad8c35640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 0x7fdad40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705784559 0 0) 0x7fdad406e8e0 con 0x7fdad40d0db0 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac0003520 con 0x7fdad40d0db0 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358592260 0 0) 0x7fdac0003520 con 0x7fdad40d0db0 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 msgr2=0x7fdad40b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 0x7fdad40b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 >> v2:172.21.15.164:3300/0 conn(0x7fdad41478b0 msgr2=0x7fdad40b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 --2- 172.21.15.82:0/3448371357 >> v2:172.21.15.164:3300/0 conn(0x7fdad41478b0 0x7fdad40b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad406ee20 con 0x7fdad40d0db0 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fdad00019f0 con 0x7fdad40d0db0 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 >> v1:172.21.15.82:6789/0 conn(0x7fdad40d0db0 legacy=0x7fdad4162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 0x7fdac00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac0003f30 0x7fdac0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 0x7fdac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406eb80 con 0x7fdac0003f30 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406f100 con 0x7fdac0004d40 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac000ba10 con 0x7fdad4147360 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdad8c35640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac0003f30 0x7fdac0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.124+0000 7fdad9436640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 0x7fdac00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad9c37640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 0x7fdac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad8c35640 1 -- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 msgr2=0x7fdac00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad8c35640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 0x7fdac00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad8c35640 1 -- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 msgr2=0x7fdac00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad8c35640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 0x7fdac00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad8c35640 1 -- 172.21.15.82:0/3448371357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac0003520 con 0x7fdac0003f30 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad9c37640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 0x7fdac00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad9436640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad4147360 0x7fdac00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdad8c35640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac0003f30 0x7fdac0004800 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009af0 tx=0x7fdac0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c0f44004c71b62 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4039070 con 0x7fdac0003f30 2024-03-14T12:17:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.125+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac402c6b0 con 0x7fdac0003f30 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdacbfff640 1 -- 172.21.15.82:0/3448371357 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4009690 con 0x7fdac0003f30 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdadb69a640 1 -- 172.21.15.82:0/3448371357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac0003f30 msgr2=0x7fdac0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdadb69a640 1 --2- 172.21.15.82:0/3448371357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac0003f30 0x7fdac0004800 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009af0 tx=0x7fdac0005830 comp rx=0 tx=0).stop 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdad9c37640 1 -- 172.21.15.82:0/3448371357 reap_dead start 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdadb69a640 1 -- 172.21.15.82:0/3448371357 shutdown_connections 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdadb69a640 1 -- 172.21.15.82:0/3448371357 >> 172.21.15.82:0/3448371357 conn(0x7fdad40b4fd0 msgr2=0x7fdad40b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdadb69a640 1 -- 172.21.15.82:0/3448371357 shutdown_connections 2024-03-14T12:17:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.126+0000 7fdadb69a640 1 -- 172.21.15.82:0/3448371357 wait complete. 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.127+0000 7fdadb69a640 1 Processor -- start 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.127+0000 7fdadb69a640 1 -- start start 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdadb69a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0003f30 0x7fdad4150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdadb69a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad409c330 0x7fdad41e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdadb69a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdad409c7b0 0x7fdad41e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdadb69a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406d330 con 0x7fdad409c7b0 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdadb69a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4162040 con 0x7fdac0003f30 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdadb69a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406d4b0 con 0x7fdad409c330 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdad409c7b0 0x7fdad41e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdad409c7b0 0x7fdad41e9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35472/0 (socket says 172.21.15.82:35472) 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 -- 172.21.15.82:0/353097420 learned_addr learned my addr 172.21.15.82:0/353097420 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad8c35640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad409c330 0x7fdad41e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 -- 172.21.15.82:0/353097420 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad409c330 msgr2=0x7fdad41e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad409c330 0x7fdad41e9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 -- 172.21.15.82:0/353097420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0003f30 msgr2=0x7fdad4150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0003f30 0x7fdad4150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.128+0000 7fdad9c37640 1 -- 172.21.15.82:0/353097420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad409cd80 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.129+0000 7fdad9c37640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdad409c7b0 0x7fdad41e9b90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdad4147c90 tx=0x7fdacc011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.129+0000 7fdac9ffb640 1 -- 172.21.15.82:0/353097420 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc003610 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.129+0000 7fdac9ffb640 1 -- 172.21.15.82:0/353097420 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc01b070 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.129+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad409d060 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.129+0000 7fdac9ffb640 1 -- 172.21.15.82:0/353097420 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc003a10 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.129+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad409d620 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.130+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad404f050 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.130+0000 7fdac9ffb640 1 -- 172.21.15.82:0/353097420 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdacc003bb0 con 0x7fdad409c7b0 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.131+0000 7fdac9ffb640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdab806bc00 0x7fdab806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.131+0000 7fdac9ffb640 1 -- 172.21.15.82:0/353097420 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdacc08ab20 con 0x7fdad409c7b0 2024-03-14T12:17:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.131+0000 7fdad9436640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdab806bc00 0x7fdab806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.131+0000 7fdad9436640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdab806bc00 0x7fdab806e0c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdad00020a0 tx=0x7fdad0002010 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.135+0000 7fdac9ffb640 1 -- 172.21.15.82:0/353097420 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdacc025020 con 0x7fdad409c7b0 2024-03-14T12:17:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.306+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdad40b3050 con 0x7fdad409c7b0 2024-03-14T12:17:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.306+0000 7fdac9ffb640 1 -- 172.21.15.82:0/353097420 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdacc01d310 con 0x7fdad409c7b0 2024-03-14T12:17:14.306 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdab806bc00 msgr2=0x7fdab806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdab806bc00 0x7fdab806e0c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdad00020a0 tx=0x7fdad0002010 comp rx=0 tx=0).stop 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdad409c7b0 msgr2=0x7fdad41e9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdad409c7b0 0x7fdad41e9b90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdad4147c90 tx=0x7fdacc011920 comp rx=0 tx=0).stop 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 shutdown_connections 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdad409c7b0 0x7fdad41e9b90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdab806bc00 0x7fdab806e0c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad409c330 0x7fdad41e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 --2- 172.21.15.82:0/353097420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0003f30 0x7fdad4150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 >> 172.21.15.82:0/353097420 conn(0x7fdad40b4fd0 msgr2=0x7fdad40ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 shutdown_connections 2024-03-14T12:17:14.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.308+0000 7fdadb69a640 1 -- 172.21.15.82:0/353097420 wait complete. 2024-03-14T12:17:14.318 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:17:14.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.419+0000 7ffb261c4640 1 Processor -- start 2024-03-14T12:17:14.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.419+0000 7ffb261c4640 1 -- start start 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.420+0000 7ffb261c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 0x7ffb200d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.420+0000 7ffb261c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200d1080 con 0x7ffb200b7720 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.420+0000 7ffb261c4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200cf8c0 con 0x7ffb200cf370 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.420+0000 7ffb261c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200cfba0 con 0x7ffb200d0760 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.420+0000 7ffb1ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffb200b7720 0x7ffb200b7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49354/0 (socket says 172.21.15.82:49354) 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.420+0000 7ffb1ffff640 1 -- 172.21.15.82:0/2789004530 learned_addr learned my addr 172.21.15.82:0/2789004530 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1f7fe640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 0x7ffb200d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859708634 0 0) 0x7ffb200d1080 con 0x7ffb200b7720 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0c003520 con 0x7ffb200b7720 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920964418 0 0) 0x7ffb200cf8c0 con 0x7ffb200cf370 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb200d1080 con 0x7ffb200cf370 2024-03-14T12:17:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1887947644 0 0) 0x7ffb0c003520 con 0x7ffb200b7720 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 msgr2=0x7ffb200d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 0x7ffb200d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 >> v1:172.21.15.164:6789/0 conn(0x7ffb200cf370 legacy=0x7ffb200d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200b6c40 con 0x7ffb200b7720 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7ffb180019f0 con 0x7ffb200b7720 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 >> v1:172.21.15.82:6789/0 conn(0x7ffb200b7720 legacy=0x7ffb200b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c003b70 0x7ffb0c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb0c004800 0x7ffb0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 0x7ffb0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200cfba0 con 0x7ffb0c004800 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c00ba70 con 0x7ffb0c003b70 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c00bc40 con 0x7ffb200d0760 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb16ffd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb0c004800 0x7ffb0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1f7fe640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c003b70 0x7ffb0c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb1ffff640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 0x7ffb0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb16ffd640 1 -- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 msgr2=0x7ffb0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb16ffd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200d0760 0x7ffb0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb16ffd640 1 -- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c003b70 msgr2=0x7ffb0c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb16ffd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c003b70 0x7ffb0c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.421+0000 7ffb16ffd640 1 -- 172.21.15.82:0/2789004530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c003520 con 0x7ffb0c004800 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.422+0000 7ffb16ffd640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb0c004800 0x7ffb0c004c80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffb080018d0 tx=0x7ffb0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218f7daa20c8f6dd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.422+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080014f0 con 0x7ffb0c004800 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.422+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0802f050 con 0x7ffb0c004800 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.422+0000 7ffb1effd640 1 -- 172.21.15.82:0/2789004530 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080035f0 con 0x7ffb0c004800 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.423+0000 7ffb261c4640 1 -- 172.21.15.82:0/2789004530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb0c004800 msgr2=0x7ffb0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.423+0000 7ffb261c4640 1 --2- 172.21.15.82:0/2789004530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb0c004800 0x7ffb0c004c80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffb080018d0 tx=0x7ffb0c005830 comp rx=0 tx=0).stop 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.423+0000 7ffb1ffff640 1 -- 172.21.15.82:0/2789004530 reap_dead start 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.423+0000 7ffb261c4640 1 -- 172.21.15.82:0/2789004530 shutdown_connections 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.423+0000 7ffb261c4640 1 -- 172.21.15.82:0/2789004530 >> 172.21.15.82:0/2789004530 conn(0x7ffb2009d030 msgr2=0x7ffb200d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:14.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.423+0000 7ffb261c4640 1 -- 172.21.15.82:0/2789004530 shutdown_connections 2024-03-14T12:17:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.423+0000 7ffb261c4640 1 -- 172.21.15.82:0/2789004530 wait complete. 2024-03-14T12:17:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.424+0000 7ffb261c4640 1 Processor -- start 2024-03-14T12:17:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.424+0000 7ffb261c4640 1 -- start start 2024-03-14T12:17:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb261c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c004800 0x7ffb20154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb261c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200c9610 0x7ffb200cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb261c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb200c9a90 0x7ffb2006c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb261c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201617a0 con 0x7ffb200c9a90 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb261c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200b7200 con 0x7ffb0c004800 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb261c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20161920 con 0x7ffb200c9610 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb200c9a90 0x7ffb2006c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb200c9a90 0x7ffb2006c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35490/0 (socket says 172.21.15.82:35490) 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 -- 172.21.15.82:0/2491032459 learned_addr learned my addr 172.21.15.82:0/2491032459 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1f7fe640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c004800 0x7ffb20154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb16ffd640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200c9610 0x7ffb200cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 -- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200c9610 msgr2=0x7ffb200cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200c9610 0x7ffb200cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 -- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c004800 msgr2=0x7ffb20154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c004800 0x7ffb20154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.425+0000 7ffb1ffff640 1 -- 172.21.15.82:0/2491032459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200cbfa0 con 0x7ffb200c9a90 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.426+0000 7ffb1ffff640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb200c9a90 0x7ffb2006c7d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffb200b68e0 tx=0x7ffb18007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.426+0000 7ffb1cff9640 1 -- 172.21.15.82:0/2491032459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18017020 con 0x7ffb200c9a90 2024-03-14T12:17:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.426+0000 7ffb1cff9640 1 -- 172.21.15.82:0/2491032459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1800ab30 con 0x7ffb200c9a90 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.426+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb200cc280 con 0x7ffb200c9a90 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.426+0000 7ffb1cff9640 1 -- 172.21.15.82:0/2491032459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180133f0 con 0x7ffb200c9a90 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.426+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb200cc840 con 0x7ffb200c9a90 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.427+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2004f050 con 0x7ffb200c9a90 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.427+0000 7ffb1cff9640 1 -- 172.21.15.82:0/2491032459 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ffb1801a020 con 0x7ffb200c9a90 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.428+0000 7ffb1cff9640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffb0006bbd0 0x7ffb0006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.428+0000 7ffb1f7fe640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffb0006bbd0 0x7ffb0006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.428+0000 7ffb1cff9640 1 -- 172.21.15.82:0/2491032459 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb1808b1b0 con 0x7ffb200c9a90 2024-03-14T12:17:14.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.428+0000 7ffb1f7fe640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffb0006bbd0 0x7ffb0006e090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009a20 tx=0x7ffb10008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.432+0000 7ffb1cff9640 1 -- 172.21.15.82:0/2491032459 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ffb18021020 con 0x7ffb200c9a90 2024-03-14T12:17:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.616+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffb200d0540 con 0x7ffb200c9a90 2024-03-14T12:17:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.616+0000 7ffb1cff9640 1 -- 172.21.15.82:0/2491032459 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffb18059d30 con 0x7ffb200c9a90 2024-03-14T12:17:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-14T12:17:14.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.618+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffb0006bbd0 msgr2=0x7ffb0006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.618+0000 7ffb261c4640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffb0006bbd0 0x7ffb0006e090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009a20 tx=0x7ffb10008040 comp rx=0 tx=0).stop 2024-03-14T12:17:14.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.618+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb200c9a90 msgr2=0x7ffb2006c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb200c9a90 0x7ffb2006c7d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffb200b68e0 tx=0x7ffb18007bd0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 shutdown_connections 2024-03-14T12:17:14.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb200c9a90 0x7ffb2006c7d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ffb0006bbd0 0x7ffb0006e090 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb200c9610 0x7ffb200cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 --2- 172.21.15.82:0/2491032459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0c004800 0x7ffb20154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 >> 172.21.15.82:0/2491032459 conn(0x7ffb2009d030 msgr2=0x7ffb20160550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:14.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 shutdown_connections 2024-03-14T12:17:14.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.619+0000 7ffb261c4640 1 -- 172.21.15.82:0/2491032459 wait complete. 2024-03-14T12:17:14.629 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T12:17:14.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:14.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-14T12:17:14.662 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:17:14.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:17:14.662 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T12:17:14.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T12:17:14.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.821+0000 7f49a7414640 1 Processor -- start 2024-03-14T12:17:14.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.821+0000 7f49a7414640 1 -- start start 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a7414640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f49a015be90 0x7f49a00cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a7414640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a00cfd50 con 0x7f49a00cd3e0 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a7414640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a00cc900 con 0x7f49a00c5ce0 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a7414640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a00ccf50 con 0x7f49a015be90 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a51b0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f49a015be90 0x7f49a00cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a49af640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f49a00cd3e0 0x7f49a00c55d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49360/0 (socket says 172.21.15.82:49360) 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a49af640 1 -- 172.21.15.82:0/755134217 learned_addr learned my addr 172.21.15.82:0/755134217 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a51b0640 1 -- 172.21.15.82:0/755134217 >> v1:172.21.15.164:6789/0 conn(0x7f49a00c5ce0 legacy=0x7f49a015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a51b0640 1 -- 172.21.15.82:0/755134217 >> v1:172.21.15.82:6789/0 conn(0x7f49a00cd3e0 legacy=0x7f49a00c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f4997fff640 1 -- 172.21.15.82:0/755134217 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539780965 0 0) 0x7f49a00cfd50 con 0x7f49a00cd3e0 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.822+0000 7f49a51b0640 1 -- 172.21.15.82:0/755134217 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a00ccbe0 con 0x7f49a015be90 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.823+0000 7f49a51b0640 1 --2- 172.21.15.82:0/755134217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a00cf650 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f499c0097a0 tx=0x7f499c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93e7bae855cd725 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.823+0000 7f4997fff640 1 -- 172.21.15.82:0/755134217 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c035020 con 0x7f49a015be90 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.823+0000 7f4997fff640 1 -- 172.21.15.82:0/755134217 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c031830 con 0x7f49a015be90 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.823+0000 7f4997fff640 1 -- 172.21.15.82:0/755134217 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c031c00 con 0x7f49a015be90 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.823+0000 7f49a7414640 1 -- 172.21.15.82:0/755134217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 msgr2=0x7f49a00cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.823+0000 7f49a7414640 1 --2- 172.21.15.82:0/755134217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a00cf650 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f499c0097a0 tx=0x7f499c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.824+0000 7f49a7414640 1 -- 172.21.15.82:0/755134217 shutdown_connections 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.824+0000 7f49a7414640 1 --2- 172.21.15.82:0/755134217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a00cf650 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.824+0000 7f49a7414640 1 -- 172.21.15.82:0/755134217 >> 172.21.15.82:0/755134217 conn(0x7f49a00b3610 msgr2=0x7f49a00b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.824+0000 7f49a7414640 1 -- 172.21.15.82:0/755134217 shutdown_connections 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.824+0000 7f49a7414640 1 -- 172.21.15.82:0/755134217 wait complete. 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 Processor -- start 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 -- start start 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a00c5ce0 0x7f49a0151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a00cd3e0 0x7f49a009ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a009ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a015c270 con 0x7f49a015be90 2024-03-14T12:17:14.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a00c6170 con 0x7f49a00c5ce0 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.825+0000 7f49a7414640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a00cc520 con 0x7f49a00cd3e0 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a009ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a009ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35520/0 (socket says 172.21.15.82:35520) 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 -- 172.21.15.82:0/3347395572 learned_addr learned my addr 172.21.15.82:0/3347395572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a51b0640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a00c5ce0 0x7f49a0151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a49af640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a00cd3e0 0x7f49a009ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 -- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a00cd3e0 msgr2=0x7f49a009ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a00cd3e0 0x7f49a009ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 -- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a00c5ce0 msgr2=0x7f49a0151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a00c5ce0 0x7f49a0151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 -- 172.21.15.82:0/3347395572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a009e6c0 con 0x7f49a015be90 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a51b0640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a00c5ce0 0x7f49a0151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a59b1640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a009ba70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f499800c7b0 tx=0x7f499800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f4995ffb640 1 -- 172.21.15.82:0/3347395572 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998004b60 con 0x7f49a015be90 2024-03-14T12:17:14.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f4995ffb640 1 -- 172.21.15.82:0/3347395572 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4998004d00 con 0x7f49a015be90 2024-03-14T12:17:14.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a009e9a0 con 0x7f49a015be90 2024-03-14T12:17:14.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f4995ffb640 1 -- 172.21.15.82:0/3347395572 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800aa90 con 0x7f49a015be90 2024-03-14T12:17:14.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.826+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a009ef60 con 0x7f49a015be90 2024-03-14T12:17:14.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.827+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a004f050 con 0x7f49a015be90 2024-03-14T12:17:14.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.828+0000 7f4995ffb640 1 -- 172.21.15.82:0/3347395572 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f499800ac30 con 0x7f49a015be90 2024-03-14T12:17:14.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.828+0000 7f4995ffb640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f498406bbd0 0x7f498406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:14.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.828+0000 7f4995ffb640 1 -- 172.21.15.82:0/3347395572 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f499808ac80 con 0x7f49a015be90 2024-03-14T12:17:14.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.828+0000 7f49a51b0640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f498406bbd0 0x7f498406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:14.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.828+0000 7f49a51b0640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f498406bbd0 0x7f498406e090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f499c036000 tx=0x7f499c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:14.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:14.832+0000 7f4995ffb640 1 -- 172.21.15.82:0/3347395572 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4998020070 con 0x7f49a015be90 2024-03-14T12:17:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.039+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f49a006fbe0 con 0x7f49a015be90 2024-03-14T12:17:15.040 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-14T12:17:15.040+0000 7f8354c85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T12:17:15.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.041+0000 7f4995ffb640 1 -- 172.21.15.82:0/3347395572 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4998059800 con 0x7f49a015be90 2024-03-14T12:17:15.041 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f498406bbd0 msgr2=0x7f498406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f498406bbd0 0x7f498406e090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f499c036000 tx=0x7f499c008030 comp rx=0 tx=0).stop 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 msgr2=0x7f49a009ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a009ba70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f499800c7b0 tx=0x7f499800cc70 comp rx=0 tx=0).stop 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 shutdown_connections 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a015be90 0x7f49a009ba70 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f498406bbd0 0x7f498406e090 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a00cd3e0 0x7f49a009ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 --2- 172.21.15.82:0/3347395572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a00c5ce0 0x7f49a0151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.043+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 >> 172.21.15.82:0/3347395572 conn(0x7f49a00b3610 msgr2=0x7f49a006e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:15.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.044+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 shutdown_connections 2024-03-14T12:17:15.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.044+0000 7f49a7414640 1 -- 172.21.15.82:0/3347395572 wait complete. 2024-03-14T12:17:15.046 INFO:tasks.ceph.mds.b.smithi164.stderr: -1802> 2024-03-14T12:17:02.126+0000 7ff1edec5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:17:15.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-14T12:17:15.074 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-03-14T12:17:15.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.160+0000 7f16bd909640 1 Processor -- start 2024-03-14T12:17:15.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.160+0000 7f16bd909640 1 -- start start 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16bd909640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d07a0 0x7f16b80d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16bd909640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f16b80cfa30 0x7f16b80d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16bd909640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b80ce8a0 con 0x7f16b80d1ea0 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16bd909640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b815eeb0 con 0x7f16b80cfa30 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16bd909640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b815f150 con 0x7f16b80d07a0 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16b77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f16b80cfa30 0x7f16b80d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16b77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f16b80cfa30 0x7f16b80d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35530/0 (socket says 172.21.15.82:35530) 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16b77fe640 1 -- 172.21.15.82:0/3411439172 learned_addr learned my addr 172.21.15.82:0/3411439172 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.161+0000 7f16b6ffd640 1 --2- 172.21.15.82:0/3411439172 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d07a0 0x7f16b80d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:15.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b77fe640 1 -- 172.21.15.82:0/3411439172 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d07a0 msgr2=0x7f16b80d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:15.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b77fe640 1 --2- 172.21.15.82:0/3411439172 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d07a0 0x7f16b80d1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b77fe640 1 -- 172.21.15.82:0/3411439172 >> v1:172.21.15.164:6789/0 conn(0x7f16b80d1ea0 legacy=0x7f16b80cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:15.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b77fe640 1 -- 172.21.15.82:0/3411439172 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80ceb20 con 0x7f16b80cfa30 2024-03-14T12:17:15.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b77fe640 1 --2- 172.21.15.82:0/3411439172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80cfa30 0x7f16b80d0b80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009e50 tx=0x7f16b002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fa94ecb4a6461 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:15.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b67fc640 1 -- 172.21.15.82:0/3411439172 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b002cb00 con 0x7f16b80cfa30 2024-03-14T12:17:15.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b67fc640 1 -- 172.21.15.82:0/3411439172 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b002cca0 con 0x7f16b80cfa30 2024-03-14T12:17:15.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.162+0000 7f16b67fc640 1 -- 172.21.15.82:0/3411439172 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0034610 con 0x7f16b80cfa30 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 -- 172.21.15.82:0/3411439172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80cfa30 msgr2=0x7f16b80d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 --2- 172.21.15.82:0/3411439172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80cfa30 0x7f16b80d0b80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009e50 tx=0x7f16b002bf60 comp rx=0 tx=0).stop 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 -- 172.21.15.82:0/3411439172 shutdown_connections 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 --2- 172.21.15.82:0/3411439172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80cfa30 0x7f16b80d0b80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 --2- 172.21.15.82:0/3411439172 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d07a0 0x7f16b80d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 -- 172.21.15.82:0/3411439172 >> 172.21.15.82:0/3411439172 conn(0x7f16b80d3d90 msgr2=0x7f16b80d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 -- 172.21.15.82:0/3411439172 shutdown_connections 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.163+0000 7f16bd909640 1 -- 172.21.15.82:0/3411439172 wait complete. 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.164+0000 7f16bd909640 1 Processor -- start 2024-03-14T12:17:15.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.164+0000 7f16bd909640 1 -- start start 2024-03-14T12:17:15.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16bd909640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b80cfa30 0x7f16b80c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16bd909640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80d07a0 0x7f16b8156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16bd909640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d1ea0 0x7f16b80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16bd909640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80b3460 con 0x7f16b80d07a0 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16bd909640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80b32e0 con 0x7f16b80cfa30 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16bd909640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b815bbf0 con 0x7f16b80d1ea0 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80d07a0 0x7f16b8156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80d07a0 0x7f16b8156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35540/0 (socket says 172.21.15.82:35540) 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 -- 172.21.15.82:0/802595262 learned_addr learned my addr 172.21.15.82:0/802595262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16b6ffd640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b80cfa30 0x7f16b80c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16b77fe640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d1ea0 0x7f16b80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 -- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d1ea0 msgr2=0x7f16b80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d1ea0 0x7f16b80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 -- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b80cfa30 msgr2=0x7f16b80c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b80cfa30 0x7f16b80c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 -- 172.21.15.82:0/802595262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80b6ae0 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.165+0000 7f16affff640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80d07a0 0x7f16b8156470 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f16a000c470 tx=0x7f16a0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.166+0000 7f16b4ff9640 1 -- 172.21.15.82:0/802595262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00052d0 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.166+0000 7f16b4ff9640 1 -- 172.21.15.82:0/802595262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a0005470 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.166+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b80b6dc0 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.166+0000 7f16b4ff9640 1 -- 172.21.15.82:0/802595262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0007260 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.166+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b80b4bb0 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.167+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b80d18c0 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.167+0000 7f16b4ff9640 1 -- 172.21.15.82:0/802595262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f16a0007400 con 0x7f16b80d07a0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.167+0000 7f16b4ff9640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f169806bc00 0x7f169806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.168+0000 7f16b6ffd640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f169806bc00 0x7f169806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.168+0000 7f16b4ff9640 1 -- 172.21.15.82:0/802595262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f16a0089ed0 con 0x7f16b80d07a0 2024-03-14T12:17:15.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.168+0000 7f16b6ffd640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f169806bc00 0x7f169806e0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f16b80b8a30 tx=0x7f16a8009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:15.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.172+0000 7f16b4ff9640 1 -- 172.21.15.82:0/802595262 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f16b80ce8a0 con 0x7f16b80d07a0 2024-03-14T12:17:15.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:15.380+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f16b804f050 con 0x7f16b80d07a0 2024-03-14T12:17:16.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.043+0000 7f16b4ff9640 1 -- 172.21.15.82:0/802595262 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v36) v1 ==== 162+0+0 (secure 0 0 0) 0x7f16a00583a0 con 0x7f16b80d07a0 2024-03-14T12:17:16.043 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f169806bc00 msgr2=0x7f169806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f169806bc00 0x7f169806e0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f16b80b8a30 tx=0x7f16a8009190 comp rx=0 tx=0).stop 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80d07a0 msgr2=0x7f16b8156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80d07a0 0x7f16b8156470 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f16a000c470 tx=0x7f16a0001e70 comp rx=0 tx=0).stop 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 shutdown_connections 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f169806bc00 0x7f169806e0c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16b80d1ea0 0x7f16b80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b80d07a0 0x7f16b8156470 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 --2- 172.21.15.82:0/802595262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b80cfa30 0x7f16b80c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 >> 172.21.15.82:0/802595262 conn(0x7f16b80d3d90 msgr2=0x7f16b815c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:16.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 shutdown_connections 2024-03-14T12:17:16.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.045+0000 7f16bd909640 1 -- 172.21.15.82:0/802595262 wait complete. 2024-03-14T12:17:16.060 INFO:tasks.cephfs.test_admin:Waiting for reconnect... 2024-03-14T12:17:16.061 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:16.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.165+0000 7f69d86c0640 1 Processor -- start 2024-03-14T12:17:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.165+0000 7f69d86c0640 1 -- start start 2024-03-14T12:17:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d86c0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f69d00af150 0x7f69d00af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d86c0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f69d015f1f0 0x7f69d006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d86c0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d00b01d0 con 0x7f69d00b58f0 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d86c0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d015e710 con 0x7f69d015f1f0 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d86c0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d015e9f0 con 0x7f69d00af150 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d5c5b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f69d015f1f0 0x7f69d006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d6c5d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f69d00b58f0 0x7f69d0157cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49364/0 (socket says 172.21.15.82:49364) 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d5c5b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f69d015f1f0 0x7f69d006ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35544/0 (socket says 172.21.15.82:35544) 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d6c5d640 1 -- 172.21.15.82:0/3450687979 learned_addr learned my addr 172.21.15.82:0/3450687979 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.166+0000 7f69d645c640 1 --2- 172.21.15.82:0/3450687979 >> v2:172.21.15.164:3300/0 conn(0x7f69d00af150 0x7f69d00af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d5c5b640 1 -- 172.21.15.82:0/3450687979 >> v2:172.21.15.164:3300/0 conn(0x7f69d00af150 msgr2=0x7f69d00af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d5c5b640 1 --2- 172.21.15.82:0/3450687979 >> v2:172.21.15.164:3300/0 conn(0x7f69d00af150 0x7f69d00af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d5c5b640 1 -- 172.21.15.82:0/3450687979 >> v1:172.21.15.82:6789/0 conn(0x7f69d00b58f0 legacy=0x7f69d0157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d5c5b640 1 -- 172.21.15.82:0/3450687979 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d015ecd0 con 0x7f69d015f1f0 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d545a640 1 -- 172.21.15.82:0/3450687979 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160882348 0 0) 0x7f69d00b01d0 con 0x7f69d00b58f0 2024-03-14T12:17:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d5c5b640 1 --2- 172.21.15.82:0/3450687979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d006ad10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f69d004f5e0 tx=0x7f69c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609d9a8e93a7440a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d545a640 1 -- 172.21.15.82:0/3450687979 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c00315a0 con 0x7f69d015f1f0 2024-03-14T12:17:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d545a640 1 -- 172.21.15.82:0/3450687979 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0037070 con 0x7f69d015f1f0 2024-03-14T12:17:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.167+0000 7f69d545a640 1 -- 172.21.15.82:0/3450687979 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0031960 con 0x7f69d015f1f0 2024-03-14T12:17:16.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 -- 172.21.15.82:0/3450687979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 msgr2=0x7f69d006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 --2- 172.21.15.82:0/3450687979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d006ad10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f69d004f5e0 tx=0x7f69c002b840 comp rx=0 tx=0).stop 2024-03-14T12:17:16.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 -- 172.21.15.82:0/3450687979 shutdown_connections 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 --2- 172.21.15.82:0/3450687979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d006ad10 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 --2- 172.21.15.82:0/3450687979 >> v2:172.21.15.164:3300/0 conn(0x7f69d00af150 0x7f69d00af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 -- 172.21.15.82:0/3450687979 >> 172.21.15.82:0/3450687979 conn(0x7f69d00d0a20 msgr2=0x7f69d00d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 -- 172.21.15.82:0/3450687979 shutdown_connections 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.168+0000 7f69d86c0640 1 -- 172.21.15.82:0/3450687979 wait complete. 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.169+0000 7f69d86c0640 1 Processor -- start 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.169+0000 7f69d86c0640 1 -- start start 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d86c0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f69d00af150 0x7f69d00cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d86c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d00b58f0 0x7f69d01f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d86c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d86c0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00aff40 con 0x7f69d015f1f0 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d86c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d00afdc0 con 0x7f69d00b58f0 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d86c0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d006b250 con 0x7f69d00af150 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:16.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d00b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35550/0 (socket says 172.21.15.82:35550) 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 -- 172.21.15.82:0/2409434720 learned_addr learned my addr 172.21.15.82:0/2409434720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d5c5b640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d00b58f0 0x7f69d01f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d645c640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f69d00af150 0x7f69d00cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 -- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f69d00af150 msgr2=0x7f69d00cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f69d00af150 0x7f69d00cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 -- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d00b58f0 msgr2=0x7f69d01f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d00b58f0 0x7f69d01f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 -- 172.21.15.82:0/2409434720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00cf9c0 con 0x7f69d015f1f0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.170+0000 7f69d6c5d640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d00b19d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c7a0 tx=0x7f69c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.171+0000 7f69c6ffd640 1 -- 172.21.15.82:0/2409434720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8018070 con 0x7f69d015f1f0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.171+0000 7f69c6ffd640 1 -- 172.21.15.82:0/2409434720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c800a640 con 0x7f69d015f1f0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.171+0000 7f69c6ffd640 1 -- 172.21.15.82:0/2409434720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c800aab0 con 0x7f69d015f1f0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.171+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d00cfc70 con 0x7f69d015f1f0 2024-03-14T12:17:16.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.171+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d00d0230 con 0x7f69d015f1f0 2024-03-14T12:17:16.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.172+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d004f050 con 0x7f69d015f1f0 2024-03-14T12:17:16.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.172+0000 7f69c6ffd640 1 -- 172.21.15.82:0/2409434720 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f69c801b070 con 0x7f69d015f1f0 2024-03-14T12:17:16.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.172+0000 7f69c6ffd640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69b406bbd0 0x7f69b406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:16.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.173+0000 7f69c6ffd640 1 -- 172.21.15.82:0/2409434720 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f69c808a0d0 con 0x7f69d015f1f0 2024-03-14T12:17:16.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.173+0000 7f69d645c640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69b406bbd0 0x7f69b406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:16.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.173+0000 7f69d645c640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69b406bbd0 0x7f69b406e090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009b40 tx=0x7f69cc009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:16.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.177+0000 7f69c6ffd640 1 -- 172.21.15.82:0/2409434720 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f69c8058480 con 0x7f69d015f1f0 2024-03-14T12:17:16.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.386+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69d006b980 con 0x7f69d015f1f0 2024-03-14T12:17:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.386+0000 7f69c6ffd640 1 -- 172.21.15.82:0/2409434720 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f69c805c520 con 0x7f69d015f1f0 2024-03-14T12:17:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:16.387 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.047212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 37 2024-03-14T12:17:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69b406bbd0 msgr2=0x7f69b406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69b406bbd0 0x7f69b406e090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009b40 tx=0x7f69cc009190 comp rx=0 tx=0).stop 2024-03-14T12:17:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 msgr2=0x7f69d00b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d00b19d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c7a0 tx=0x7f69c800cc60 comp rx=0 tx=0).stop 2024-03-14T12:17:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 shutdown_connections 2024-03-14T12:17:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69d015f1f0 0x7f69d00b19d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d00b58f0 0x7f69d01f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69b406bbd0 0x7f69b406e090 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.388+0000 7f69d86c0640 1 --2- 172.21.15.82:0/2409434720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f69d00af150 0x7f69d00cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.389+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 >> 172.21.15.82:0/2409434720 conn(0x7f69d00d0a20 msgr2=0x7f69d0071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.389+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 shutdown_connections 2024-03-14T12:17:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:16.389+0000 7f69d86c0640 1 -- 172.21.15.82:0/2409434720 wait complete. 2024-03-14T12:17:16.400 DEBUG:tasks.cephfs.filesystem:mapped states ['up:replay'] to None 2024-03-14T12:17:17.400 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:17.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.512+0000 7f6845acb640 1 Processor -- start 2024-03-14T12:17:17.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.512+0000 7f6845acb640 1 -- start start 2024-03-14T12:17:17.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f6845acb640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 0x7f68400d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f6845acb640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f68400cf370 0x7f68400d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f6845acb640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68400d1080 con 0x7f68400d1da0 2024-03-14T12:17:17.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f6845acb640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68400cf8c0 con 0x7f68400cf370 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f6845acb640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68400cfba0 con 0x7f68400d0760 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f683ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f68400d1da0 0x7f68400d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49380/0 (socket says 172.21.15.82:49380) 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f683ffff640 1 -- 172.21.15.82:0/976889189 learned_addr learned my addr 172.21.15.82:0/976889189 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f683f7fe640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 0x7f68400d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f683effd640 1 --2- 172.21.15.82:0/976889189 >> v2:172.21.15.164:3300/0 conn(0x7f68400cf370 0x7f68400d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945993769 0 0) 0x7f68400d1080 con 0x7f68400d1da0 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.513+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6828003520 con 0x7f68400d1da0 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3592483819 0 0) 0x7f6828003520 con 0x7f68400d1da0 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 msgr2=0x7f68400d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 0x7f68400d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 >> v2:172.21.15.164:3300/0 conn(0x7f68400cf370 msgr2=0x7f68400d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 --2- 172.21.15.82:0/976889189 >> v2:172.21.15.164:3300/0 conn(0x7f68400cf370 0x7f68400d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400b6c40 con 0x7f68400d1da0 2024-03-14T12:17:17.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f68380019f0 con 0x7f68400d1da0 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 >> v1:172.21.15.82:6789/0 conn(0x7f68400d1da0 legacy=0x7f68400d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 0x7f68280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6828003f30 0x7f6828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6828004d40 0x7f68280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400cf8c0 con 0x7f6828004d40 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400cfba0 con 0x7f6828003f30 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682800b9f0 con 0x7f68400d0760 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683ffff640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6828004d40 0x7f68280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683f7fe640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 0x7f68280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683effd640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6828003f30 0x7f6828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683ffff640 1 -- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 msgr2=0x7f68280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683ffff640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f68400d0760 0x7f68280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683ffff640 1 -- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6828003f30 msgr2=0x7f6828004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683ffff640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6828003f30 0x7f6828004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.514+0000 7f683ffff640 1 -- 172.21.15.82:0/976889189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828003520 con 0x7f6828004d40 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.515+0000 7f683ffff640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6828004d40 0x7f68280051c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f683800a870 tx=0x7f6828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34259f4a538d2d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.515+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800a4c0 con 0x7f6828004d40 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.515+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838034630 con 0x7f6828004d40 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.515+0000 7f683e7fc640 1 -- 172.21.15.82:0/976889189 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838034a20 con 0x7f6828004d40 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.516+0000 7f6845acb640 1 -- 172.21.15.82:0/976889189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6828004d40 msgr2=0x7f68280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.516+0000 7f6845acb640 1 --2- 172.21.15.82:0/976889189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6828004d40 0x7f68280051c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f683800a870 tx=0x7f6828005830 comp rx=0 tx=0).stop 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.516+0000 7f683ffff640 1 -- 172.21.15.82:0/976889189 reap_dead start 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.516+0000 7f6845acb640 1 -- 172.21.15.82:0/976889189 shutdown_connections 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.516+0000 7f6845acb640 1 -- 172.21.15.82:0/976889189 >> 172.21.15.82:0/976889189 conn(0x7f684009d030 msgr2=0x7f68400d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.516+0000 7f6845acb640 1 -- 172.21.15.82:0/976889189 shutdown_connections 2024-03-14T12:17:17.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.516+0000 7f6845acb640 1 -- 172.21.15.82:0/976889189 wait complete. 2024-03-14T12:17:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.517+0000 7f6845acb640 1 Processor -- start 2024-03-14T12:17:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.517+0000 7f6845acb640 1 -- start start 2024-03-14T12:17:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f6845acb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f684015dc10 0x7f68400b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f6845acb640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f684015dff0 0x7f68400b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f6845acb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68400b1d90 0x7f684006c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f6845acb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400d4ba0 con 0x7f68400b1d90 2024-03-14T12:17:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f6845acb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400b7200 con 0x7f684015dc10 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f6845acb640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68401617a0 con 0x7f684015dff0 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68400b1d90 0x7f684006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68400b1d90 0x7f684006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35566/0 (socket says 172.21.15.82:35566) 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 -- 172.21.15.82:0/1867361110 learned_addr learned my addr 172.21.15.82:0/1867361110 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683effd640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f684015dff0 0x7f68400b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683f7fe640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f684015dc10 0x7f68400b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 -- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f684015dff0 msgr2=0x7f68400b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f684015dff0 0x7f68400b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 -- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f684015dc10 msgr2=0x7f68400b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f684015dc10 0x7f68400b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 -- 172.21.15.82:0/1867361110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68401f74d0 con 0x7f68400b1d90 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683f7fe640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f684015dc10 0x7f68400b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f683ffff640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68400b1d90 0x7f684006c610 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f683800a0b0 tx=0x7f68400ce7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.518+0000 7f6827fff640 1 -- 172.21.15.82:0/1867361110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838039040 con 0x7f68400b1d90 2024-03-14T12:17:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.519+0000 7f6827fff640 1 -- 172.21.15.82:0/1867361110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838045070 con 0x7f68400b1d90 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.519+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68401f77b0 con 0x7f68400b1d90 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.519+0000 7f6827fff640 1 -- 172.21.15.82:0/1867361110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800a4c0 con 0x7f68400b1d90 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.519+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68401f7d70 con 0x7f68400b1d90 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.520+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684004f050 con 0x7f68400b1d90 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.520+0000 7f6827fff640 1 -- 172.21.15.82:0/1867361110 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6838034a00 con 0x7f68400b1d90 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.520+0000 7f6827fff640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f682c06bbd0 0x7f682c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.521+0000 7f6827fff640 1 -- 172.21.15.82:0/1867361110 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f68380acde0 con 0x7f68400b1d90 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.521+0000 7f683f7fe640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f682c06bbd0 0x7f682c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.521+0000 7f683f7fe640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f682c06bbd0 0x7f682c06e090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f68400a13a0 tx=0x7f6830008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.525+0000 7f6827fff640 1 -- 172.21.15.82:0/1867361110 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f683807b990 con 0x7f68400b1d90 2024-03-14T12:17:17.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.734+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68400d0540 con 0x7f68400b1d90 2024-03-14T12:17:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.734+0000 7f6827fff640 1 -- 172.21.15.82:0/1867361110 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f683807b290 con 0x7f68400b1d90 2024-03-14T12:17:17.735 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:17.735 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f682c06bbd0 msgr2=0x7f682c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f682c06bbd0 0x7f682c06e090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f68400a13a0 tx=0x7f6830008040 comp rx=0 tx=0).stop 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68400b1d90 msgr2=0x7f684006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68400b1d90 0x7f684006c610 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f683800a0b0 tx=0x7f68400ce7b0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 shutdown_connections 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68400b1d90 0x7f684006c610 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f682c06bbd0 0x7f682c06e090 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f684015dff0 0x7f68400b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 --2- 172.21.15.82:0/1867361110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f684015dc10 0x7f68400b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.736+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 >> 172.21.15.82:0/1867361110 conn(0x7f684009d030 msgr2=0x7f68400b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.737+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 shutdown_connections 2024-03-14T12:17:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.737+0000 7f6845acb640 1 -- 172.21.15.82:0/1867361110 wait complete. 2024-03-14T12:17:17.748 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to up:reconnect 2024-03-14T12:17:17.748 DEBUG:tasks.cephfs.filesystem:reached state 'up:reconnect' in 1.6872944831848145s 2024-03-14T12:17:17.748 INFO:tasks.cephfs.test_admin:Waiting for active... 2024-03-14T12:17:17.748 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:17.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.848+0000 7f4d64a52640 1 Processor -- start 2024-03-14T12:17:17.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.848+0000 7f4d64a52640 1 -- start start 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d64a52640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4d60071740 0x7f4d60071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d64a52640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 0x7f4d6015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d64a52640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d600b3050 con 0x7f4d6015e670 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d64a52640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d60072060 con 0x7f4d60071740 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d64a52640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6015ebe0 con 0x7f4d60143f60 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d5dd9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4d6015e670 0x7f4d600b2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49384/0 (socket says 172.21.15.82:49384) 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d5dd9b640 1 -- 172.21.15.82:0/3552351708 learned_addr learned my addr 172.21.15.82:0/3552351708 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d5ed9d640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 0x7f4d6015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453507613 0 0) 0x7f4d600b3050 con 0x7f4d6015e670 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d5e59c640 1 --2- 172.21.15.82:0/3552351708 >> v2:172.21.15.164:3300/0 conn(0x7f4d60071740 0x7f4d60071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.849+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d48005730 con 0x7f4d6015e670 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3733023169 0 0) 0x7f4d48005730 con 0x7f4d6015e670 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 msgr2=0x7f4d6015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 0x7f4d6015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 >> v2:172.21.15.164:3300/0 conn(0x7f4d60071740 msgr2=0x7f4d60071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 --2- 172.21.15.82:0/3552351708 >> v2:172.21.15.164:3300/0 conn(0x7f4d60071740 0x7f4d60071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6015eec0 con 0x7f4d6015e670 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f4d54001670 con 0x7f4d6015e670 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 >> v1:172.21.15.82:6789/0 conn(0x7f4d6015e670 legacy=0x7f4d600b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 0x7f4d48005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d48006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d48006f50 0x7f4d480073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60072060 con 0x7f4d48006140 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6015ebe0 con 0x7f4d48006f50 2024-03-14T12:17:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4800dbd0 con 0x7f4d60143f60 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d48006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5e59c640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 0x7f4d48005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5ed9d640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d48006f50 0x7f4d480073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5dd9b640 1 -- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 msgr2=0x7f4d48005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60143f60 0x7f4d48005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5dd9b640 1 -- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d48006f50 msgr2=0x7f4d480073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d48006f50 0x7f4d480073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.850+0000 7f4d5dd9b640 1 -- 172.21.15.82:0/3552351708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48005730 con 0x7f4d48006140 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d48006a10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c240 tx=0x7f4d48007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3dda00c6735b23 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54007d30 con 0x7f4d48006140 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54030070 con 0x7f4d48006140 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d5d59a640 1 -- 172.21.15.82:0/3552351708 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5402cd60 con 0x7f4d48006140 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d64a52640 1 -- 172.21.15.82:0/3552351708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 msgr2=0x7f4d48006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d64a52640 1 --2- 172.21.15.82:0/3552351708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d48006a10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c240 tx=0x7f4d48007a40 comp rx=0 tx=0).stop 2024-03-14T12:17:17.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d5ed9d640 1 -- 172.21.15.82:0/3552351708 reap_dead start 2024-03-14T12:17:17.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d64a52640 1 -- 172.21.15.82:0/3552351708 shutdown_connections 2024-03-14T12:17:17.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.851+0000 7f4d64a52640 1 -- 172.21.15.82:0/3552351708 >> 172.21.15.82:0/3552351708 conn(0x7f4d600b1c70 msgr2=0x7f4d600cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:17.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.852+0000 7f4d64a52640 1 -- 172.21.15.82:0/3552351708 shutdown_connections 2024-03-14T12:17:17.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.852+0000 7f4d64a52640 1 -- 172.21.15.82:0/3552351708 wait complete. 2024-03-14T12:17:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.853+0000 7f4d64a52640 1 Processor -- start 2024-03-14T12:17:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.853+0000 7f4d64a52640 1 -- start start 2024-03-14T12:17:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d600aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d600cd990 0x7f4d6013cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60142450 0x7f4d60142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60143ce0 con 0x7f4d48006140 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6015f480 con 0x7f4d600cd990 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6015b080 con 0x7f4d60142450 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d600aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d600aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35578/0 (socket says 172.21.15.82:35578) 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 -- 172.21.15.82:0/2272446098 learned_addr learned my addr 172.21.15.82:0/2272446098 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d600cd990 0x7f4d6013cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5ed9d640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60142450 0x7f4d60142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 -- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60142450 msgr2=0x7f4d60142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60142450 0x7f4d60142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 -- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d600cd990 msgr2=0x7f4d6013cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d600cd990 0x7f4d6013cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 -- 172.21.15.82:0/2272446098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6013d870 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d600cd990 0x7f4d6013cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d5e59c640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d600aeea0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4d50011e30 tx=0x7f4d5000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d4effd640 1 -- 172.21.15.82:0/2272446098 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50015630 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d4effd640 1 -- 172.21.15.82:0/2272446098 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d500157d0 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d600cdee0 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d4effd640 1 -- 172.21.15.82:0/2272446098 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50015ba0 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.854+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6015cb20 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.855+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6004f050 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.856+0000 7f4d4effd640 1 -- 172.21.15.82:0/2272446098 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4d5001d400 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.856+0000 7f4d4effd640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d4006bc00 0x7f4d4006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.856+0000 7f4d4effd640 1 -- 172.21.15.82:0/2272446098 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4d5008bdb0 con 0x7f4d48006140 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.856+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d4006bc00 0x7f4d4006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.857+0000 7f4d5dd9b640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d4006bc00 0x7f4d4006e0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4d54034000 tx=0x7f4d54007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:17.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:17.861+0000 7f4d4effd640 1 -- 172.21.15.82:0/2272446098 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4d6013d870 con 0x7f4d48006140 2024-03-14T12:17:18.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.068+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d6006d650 con 0x7f4d48006140 2024-03-14T12:17:18.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.068+0000 7f4d4effd640 1 -- 172.21.15.82:0/2272446098 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f4d50019030 con 0x7f4d48006140 2024-03-14T12:17:18.069 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:18.069 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:18.069 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d4006bc00 msgr2=0x7f4d4006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d4006bc00 0x7f4d4006e0c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4d54034000 tx=0x7f4d54007000 comp rx=0 tx=0).stop 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 msgr2=0x7f4d600aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d600aeea0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4d50011e30 tx=0x7f4d5000f580 comp rx=0 tx=0).stop 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 shutdown_connections 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d4006bc00 0x7f4d4006e0c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4d60142450 0x7f4d60142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d600cd990 0x7f4d6013cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 --2- 172.21.15.82:0/2272446098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d48006140 0x7f4d600aeea0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:18.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 >> 172.21.15.82:0/2272446098 conn(0x7f4d600b1c70 msgr2=0x7f4d601409d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:18.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 shutdown_connections 2024-03-14T12:17:18.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:18.070+0000 7f4d64a52640 1 -- 172.21.15.82:0/2272446098 wait complete. 2024-03-14T12:17:18.082 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:19.082 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.193+0000 7f7df786c640 1 Processor -- start 2024-03-14T12:17:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.194+0000 7f7df786c640 1 -- start start 2024-03-14T12:17:19.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.194+0000 7f7df786c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7df00cf370 0x7f7df00d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:19.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.194+0000 7f7df786c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00b7720 0x7f7df00b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:19.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df786c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df00cf8c0 con 0x7f7df00d0760 2024-03-14T12:17:19.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df786c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df00cfba0 con 0x7f7df00cf370 2024-03-14T12:17:19.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df786c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00b6c40 con 0x7f7df00b7720 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df4e07640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7df00cf370 0x7f7df00d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df5e09640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00b7720 0x7f7df00b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df4e07640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7df00cf370 0x7f7df00d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:41444/0 (socket says 172.21.15.82:41444) 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df4e07640 1 -- 172.21.15.82:0/4037111163 learned_addr learned my addr 172.21.15.82:0/4037111163 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df5e09640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00b7720 0x7f7df00b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:38230/0 (socket says 172.21.15.82:38230) 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df4e07640 1 -- 172.21.15.82:0/4037111163 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00b7720 msgr2=0x7f7df00b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df4e07640 1 --2- 172.21.15.82:0/4037111163 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00b7720 0x7f7df00b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df4e07640 1 -- 172.21.15.82:0/4037111163 >> v1:172.21.15.164:6789/0 conn(0x7f7df00d0760 legacy=0x7f7df00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.195+0000 7f7df4e07640 1 -- 172.21.15.82:0/4037111163 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00b6f20 con 0x7f7df00cf370 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.196+0000 7f7df4e07640 1 --2- 172.21.15.82:0/4037111163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00cf370 0x7f7df00d4660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7de00098a0 tx=0x7f7de002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2169004a63b4660 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.196+0000 7f7de7fff640 1 -- 172.21.15.82:0/4037111163 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de002f830 con 0x7f7df00cf370 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.196+0000 7f7de7fff640 1 -- 172.21.15.82:0/4037111163 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de002f9d0 con 0x7f7df00cf370 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.196+0000 7f7de7fff640 1 -- 172.21.15.82:0/4037111163 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de002fda0 con 0x7f7df00cf370 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 -- 172.21.15.82:0/4037111163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00cf370 msgr2=0x7f7df00d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 --2- 172.21.15.82:0/4037111163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00cf370 0x7f7df00d4660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7de00098a0 tx=0x7f7de002ba10 comp rx=0 tx=0).stop 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 -- 172.21.15.82:0/4037111163 shutdown_connections 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 --2- 172.21.15.82:0/4037111163 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00b7720 0x7f7df00b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 --2- 172.21.15.82:0/4037111163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00cf370 0x7f7df00d4660 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 -- 172.21.15.82:0/4037111163 >> 172.21.15.82:0/4037111163 conn(0x7f7df009d030 msgr2=0x7f7df00d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 -- 172.21.15.82:0/4037111163 shutdown_connections 2024-03-14T12:17:19.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.197+0000 7f7df786c640 1 -- 172.21.15.82:0/4037111163 wait complete. 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.198+0000 7f7df786c640 1 Processor -- start 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.198+0000 7f7df786c640 1 -- start start 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df786c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00b7720 0x7f7df01f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df786c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00cf370 0x7f7df014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df786c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00d0760 0x7f7df014e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df786c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df01617a0 con 0x7f7df00d0760 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df786c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00d4ba0 con 0x7f7df00b7720 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df786c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0161920 con 0x7f7df00cf370 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00d0760 0x7f7df014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00d0760 0x7f7df014e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35586/0 (socket says 172.21.15.82:35586) 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 -- 172.21.15.82:0/2074673541 learned_addr learned my addr 172.21.15.82:0/2074673541 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:19.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df4e07640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00cf370 0x7f7df014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5608640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00b7720 0x7f7df01f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 -- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00cf370 msgr2=0x7f7df014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00cf370 0x7f7df014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 -- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00b7720 msgr2=0x7f7df01f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00b7720 0x7f7df01f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.199+0000 7f7df5e09640 1 -- 172.21.15.82:0/2074673541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df014f960 con 0x7f7df00d0760 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.200+0000 7f7df5e09640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00d0760 0x7f7df014e110 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00e930 tx=0x7f7dec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.200+0000 7f7de5ffb640 1 -- 172.21.15.82:0/2074673541 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec017070 con 0x7f7df00d0760 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.200+0000 7f7de5ffb640 1 -- 172.21.15.82:0/2074673541 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec00c980 con 0x7f7df00d0760 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.200+0000 7f7de5ffb640 1 -- 172.21.15.82:0/2074673541 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec00cd70 con 0x7f7df00d0760 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.200+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df014fc40 con 0x7f7df00d0760 2024-03-14T12:17:19.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.200+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df0150200 con 0x7f7df00d0760 2024-03-14T12:17:19.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.201+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df004f050 con 0x7f7df00d0760 2024-03-14T12:17:19.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.202+0000 7f7de5ffb640 1 -- 172.21.15.82:0/2074673541 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7dec01a070 con 0x7f7df00d0760 2024-03-14T12:17:19.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.202+0000 7f7de5ffb640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd406bbd0 0x7f7dd406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:19.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.202+0000 7f7de5ffb640 1 -- 172.21.15.82:0/2074673541 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7dec08ad30 con 0x7f7df00d0760 2024-03-14T12:17:19.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.202+0000 7f7df5608640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd406bbd0 0x7f7dd406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:19.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.202+0000 7f7df5608640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd406bbd0 0x7f7dd406e090 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7de8001e70 tx=0x7f7de8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:19.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.206+0000 7f7de5ffb640 1 -- 172.21.15.82:0/2074673541 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7dec0592c0 con 0x7f7df00d0760 2024-03-14T12:17:19.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.413+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7df00d0540 con 0x7f7df00d0760 2024-03-14T12:17:19.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.413+0000 7f7de5ffb640 1 -- 172.21.15.82:0/2074673541 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f7dec05d360 con 0x7f7df00d0760 2024-03-14T12:17:19.414 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:19.414 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:19.414 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd406bbd0 msgr2=0x7f7dd406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd406bbd0 0x7f7dd406e090 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7de8001e70 tx=0x7f7de8009040 comp rx=0 tx=0).stop 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00d0760 msgr2=0x7f7df014e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00d0760 0x7f7df014e110 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00e930 tx=0x7f7dec00edf0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 shutdown_connections 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df00d0760 0x7f7df014e110 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7dd406bbd0 0x7f7dd406e090 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df00cf370 0x7f7df014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 --2- 172.21.15.82:0/2074673541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df00b7720 0x7f7df01f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:19.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 >> 172.21.15.82:0/2074673541 conn(0x7f7df009d030 msgr2=0x7f7df00d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:19.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 shutdown_connections 2024-03-14T12:17:19.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:19.415+0000 7f7df786c640 1 -- 172.21.15.82:0/2074673541 wait complete. 2024-03-14T12:17:19.427 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:20.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:20.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.539+0000 7fb9ef953640 1 Processor -- start 2024-03-14T12:17:20.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ef953640 1 -- start start 2024-03-14T12:17:20.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ef953640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb9e80b4190 0x7fb9e8160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:20.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ef953640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb9e80cc770 0x7fb9e80ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:20.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ef953640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e806fc70 con 0x7fb9e806ed90 2024-03-14T12:17:20.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ef953640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e815d340 con 0x7fb9e80cc770 2024-03-14T12:17:20.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ef953640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80ce610 con 0x7fb9e80b4190 2024-03-14T12:17:20.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9edef0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb9e80cc770 0x7fb9e80ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:20.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ed6ef640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb9e806ed90 0x7fb9e806f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49398/0 (socket says 172.21.15.82:49398) 2024-03-14T12:17:20.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.540+0000 7fb9ed6ef640 1 -- 172.21.15.82:0/1245190581 learned_addr learned my addr 172.21.15.82:0/1245190581 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:20.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.541+0000 7fb9edef0640 1 -- 172.21.15.82:0/1245190581 >> v2:172.21.15.164:3300/0 conn(0x7fb9e80b4190 msgr2=0x7fb9e8160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:20.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.541+0000 7fb9edef0640 1 --2- 172.21.15.82:0/1245190581 >> v2:172.21.15.164:3300/0 conn(0x7fb9e80b4190 0x7fb9e8160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.541+0000 7fb9edef0640 1 -- 172.21.15.82:0/1245190581 >> v1:172.21.15.82:6789/0 conn(0x7fb9e806ed90 legacy=0x7fb9e806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:20.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.541+0000 7fb9edef0640 1 -- 172.21.15.82:0/1245190581 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80ce8f0 con 0x7fb9e80cc770 2024-03-14T12:17:20.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.541+0000 7fb9dffff640 1 -- 172.21.15.82:0/1245190581 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008066647 0 0) 0x7fb9e4001110 con 0x7fb9e806ed90 2024-03-14T12:17:20.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.541+0000 7fb9edef0640 1 --2- 172.21.15.82:0/1245190581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e80cc770 0x7fb9e80ccbe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8070b30 tx=0x7fb9e002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f505f08edf8c4f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9dffff640 1 -- 172.21.15.82:0/1245190581 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0039070 con 0x7fb9e80cc770 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9dffff640 1 -- 172.21.15.82:0/1245190581 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e002cc80 con 0x7fb9e80cc770 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9dffff640 1 -- 172.21.15.82:0/1245190581 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0034660 con 0x7fb9e80cc770 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 -- 172.21.15.82:0/1245190581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e80cc770 msgr2=0x7fb9e80ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 --2- 172.21.15.82:0/1245190581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e80cc770 0x7fb9e80ccbe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8070b30 tx=0x7fb9e002bed0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 -- 172.21.15.82:0/1245190581 shutdown_connections 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 --2- 172.21.15.82:0/1245190581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e80cc770 0x7fb9e80ccbe0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 --2- 172.21.15.82:0/1245190581 >> v2:172.21.15.164:3300/0 conn(0x7fb9e80b4190 0x7fb9e8160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 -- 172.21.15.82:0/1245190581 >> 172.21.15.82:0/1245190581 conn(0x7fb9e80b3670 msgr2=0x7fb9e80b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 -- 172.21.15.82:0/1245190581 shutdown_connections 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.542+0000 7fb9ef953640 1 -- 172.21.15.82:0/1245190581 wait complete. 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.543+0000 7fb9ef953640 1 Processor -- start 2024-03-14T12:17:20.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ef953640 1 -- start start 2024-03-14T12:17:20.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ef953640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e806ed90 0x7fb9e80c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:20.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ef953640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e80b4190 0x7fb9e80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ef953640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9e80cc770 0x7fb9e80afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ef953640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80cf190 con 0x7fb9e806ed90 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ef953640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80ceeb0 con 0x7fb9e80b4190 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ef953640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80cd120 con 0x7fb9e80cc770 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ed6ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e806ed90 0x7fb9e80c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ed6ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e806ed90 0x7fb9e80c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35602/0 (socket says 172.21.15.82:35602) 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9ed6ef640 1 -- 172.21.15.82:0/4045665176 learned_addr learned my addr 172.21.15.82:0/4045665176 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9edef0640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9e80cc770 0x7fb9e80afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.544+0000 7fb9eceee640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e80b4190 0x7fb9e80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ed6ef640 1 -- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9e80cc770 msgr2=0x7fb9e80afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ed6ef640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9e80cc770 0x7fb9e80afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ed6ef640 1 -- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e80b4190 msgr2=0x7fb9e80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:20.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ed6ef640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e80b4190 0x7fb9e80af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ed6ef640 1 -- 172.21.15.82:0/4045665176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80c5e70 con 0x7fb9e806ed90 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ed6ef640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e806ed90 0x7fb9e80c4af0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400c780 tx=0x7fb9e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ddffb640 1 -- 172.21.15.82:0/4045665176 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4018070 con 0x7fb9e806ed90 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ddffb640 1 -- 172.21.15.82:0/4045665176 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e40072d0 con 0x7fb9e806ed90 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ddffb640 1 -- 172.21.15.82:0/4045665176 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40047e0 con 0x7fb9e806ed90 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e806ce30 con 0x7fb9e806ed90 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.545+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e806d3f0 con 0x7fb9e806ed90 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.546+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e806d5c0 con 0x7fb9e806ed90 2024-03-14T12:17:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.546+0000 7fb9ddffb640 1 -- 172.21.15.82:0/4045665176 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb9e401b070 con 0x7fb9e806ed90 2024-03-14T12:17:20.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.547+0000 7fb9ddffb640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb9cc06bbd0 0x7fb9cc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:20.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.547+0000 7fb9ddffb640 1 -- 172.21.15.82:0/4045665176 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb9e4089ec0 con 0x7fb9e806ed90 2024-03-14T12:17:20.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.547+0000 7fb9eceee640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb9cc06bbd0 0x7fb9cc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:20.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.547+0000 7fb9eceee640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb9cc06bbd0 0x7fb9cc06e090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80b1a70 tx=0x7fb9d8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:20.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.551+0000 7fb9ddffb640 1 -- 172.21.15.82:0/4045665176 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb9e806fc70 con 0x7fb9e806ed90 2024-03-14T12:17:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.759+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9e804f050 con 0x7fb9e806ed90 2024-03-14T12:17:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.759+0000 7fb9ddffb640 1 -- 172.21.15.82:0/4045665176 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fb9e40588b0 con 0x7fb9e806ed90 2024-03-14T12:17:20.760 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:20.760 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb9cc06bbd0 msgr2=0x7fb9cc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb9cc06bbd0 0x7fb9cc06e090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80b1a70 tx=0x7fb9d8008040 comp rx=0 tx=0).stop 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e806ed90 msgr2=0x7fb9e80c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e806ed90 0x7fb9e80c4af0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400c780 tx=0x7fb9e400cc40 comp rx=0 tx=0).stop 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 shutdown_connections 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb9cc06bbd0 0x7fb9cc06e090 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9e80cc770 0x7fb9e80afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e80b4190 0x7fb9e80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 --2- 172.21.15.82:0/4045665176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9e806ed90 0x7fb9e80c4af0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 >> 172.21.15.82:0/4045665176 conn(0x7fb9e80b3670 msgr2=0x7fb9e8151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 shutdown_connections 2024-03-14T12:17:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:20.761+0000 7fb9ef953640 1 -- 172.21.15.82:0/4045665176 wait complete. 2024-03-14T12:17:20.772 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:21.774 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:21.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.884+0000 7effad9f1640 1 Processor -- start 2024-03-14T12:17:21.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.884+0000 7effad9f1640 1 -- start start 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effad9f1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7effa80af150 0x7effa80af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effad9f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effad9f1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa80b01d0 con 0x7effa80b58f0 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effad9f1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa815e710 con 0x7effa80af150 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effad9f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa815e9f0 con 0x7effa815f1f0 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effa6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7effa80af150 0x7effa80af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effa77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7effa80b58f0 0x7effa8157cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51184/0 (socket says 172.21.15.82:51184) 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effa77fe640 1 -- 172.21.15.82:0/1102186432 learned_addr learned my addr 172.21.15.82:0/1102186432 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.885+0000 7effa67fc640 1 --2- 172.21.15.82:0/1102186432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.886+0000 7effa5ffb640 1 -- 172.21.15.82:0/1102186432 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522461365 0 0) 0x7effa80b01d0 con 0x7effa80b58f0 2024-03-14T12:17:21.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.886+0000 7effa5ffb640 1 -- 172.21.15.82:0/1102186432 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff90003520 con 0x7effa80b58f0 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.886+0000 7effa6ffd640 1 -- 172.21.15.82:0/1102186432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 msgr2=0x7effa806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.886+0000 7effa6ffd640 1 --2- 172.21.15.82:0/1102186432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa806ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.886+0000 7effa6ffd640 1 -- 172.21.15.82:0/1102186432 >> v1:172.21.15.82:6789/0 conn(0x7effa80b58f0 legacy=0x7effa8157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.886+0000 7effa6ffd640 1 -- 172.21.15.82:0/1102186432 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa815ecd0 con 0x7effa80af150 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.886+0000 7effa6ffd640 1 --2- 172.21.15.82:0/1102186432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80af150 0x7effa80af880 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d90 tx=0x7eff9802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e1884999f270b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effa5ffb640 1 -- 172.21.15.82:0/1102186432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98039070 con 0x7effa80af150 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effa5ffb640 1 -- 172.21.15.82:0/1102186432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9802cbc0 con 0x7effa80af150 2024-03-14T12:17:21.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effa5ffb640 1 -- 172.21.15.82:0/1102186432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98034440 con 0x7effa80af150 2024-03-14T12:17:21.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 -- 172.21.15.82:0/1102186432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80af150 msgr2=0x7effa80af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:21.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 --2- 172.21.15.82:0/1102186432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80af150 0x7effa80af880 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d90 tx=0x7eff9802be10 comp rx=0 tx=0).stop 2024-03-14T12:17:21.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 -- 172.21.15.82:0/1102186432 shutdown_connections 2024-03-14T12:17:21.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 --2- 172.21.15.82:0/1102186432 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:21.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 --2- 172.21.15.82:0/1102186432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80af150 0x7effa80af880 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 -- 172.21.15.82:0/1102186432 >> 172.21.15.82:0/1102186432 conn(0x7effa80d0a20 msgr2=0x7effa80d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 -- 172.21.15.82:0/1102186432 shutdown_connections 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.887+0000 7effad9f1640 1 -- 172.21.15.82:0/1102186432 wait complete. 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.888+0000 7effad9f1640 1 Processor -- start 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effad9f1640 1 -- start start 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effad9f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa80af150 0x7effa81e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effad9f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80b58f0 0x7effa814b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effad9f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa814e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effad9f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa80aff40 con 0x7effa80b58f0 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effad9f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa80afdc0 con 0x7effa80af150 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effad9f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa806b250 con 0x7effa815f1f0 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effa67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80b58f0 0x7effa814b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effa67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80b58f0 0x7effa814b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58118/0 (socket says 172.21.15.82:58118) 2024-03-14T12:17:21.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.889+0000 7effa67fc640 1 -- 172.21.15.82:0/1070106459 learned_addr learned my addr 172.21.15.82:0/1070106459 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa6ffd640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa80af150 0x7effa81e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa77fe640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa814e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa67fc640 1 -- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 msgr2=0x7effa814e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa67fc640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa814e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa67fc640 1 -- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa80af150 msgr2=0x7effa81e6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa67fc640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa80af150 0x7effa81e6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa67fc640 1 -- 172.21.15.82:0/1070106459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa814ecf0 con 0x7effa80b58f0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa6ffd640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa80af150 0x7effa81e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effa67fc640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80b58f0 0x7effa814b910 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00d5f0 tx=0x7eff9c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7eff8ffff640 1 -- 172.21.15.82:0/1070106459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0125f0 con 0x7effa80b58f0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7eff8ffff640 1 -- 172.21.15.82:0/1070106459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c019070 con 0x7effa80b58f0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa814efd0 con 0x7effa80b58f0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7eff8ffff640 1 -- 172.21.15.82:0/1070106459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0129d0 con 0x7effa80b58f0 2024-03-14T12:17:21.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.890+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa81e69d0 con 0x7effa80b58f0 2024-03-14T12:17:21.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.891+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa80b2550 con 0x7effa80b58f0 2024-03-14T12:17:21.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.891+0000 7eff8ffff640 1 -- 172.21.15.82:0/1070106459 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7eff9c012b70 con 0x7effa80b58f0 2024-03-14T12:17:21.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.892+0000 7eff8ffff640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7eff9406bbd0 0x7eff9406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:21.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.892+0000 7eff8ffff640 1 -- 172.21.15.82:0/1070106459 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff9c08af00 con 0x7effa80b58f0 2024-03-14T12:17:21.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.892+0000 7effa6ffd640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7eff9406bbd0 0x7eff9406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:21.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.892+0000 7effa6ffd640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7eff9406bbd0 0x7eff9406e090 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff98036000 tx=0x7eff98030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:21.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:21.896+0000 7eff8ffff640 1 -- 172.21.15.82:0/1070106459 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7effa80b2550 con 0x7effa80b58f0 2024-03-14T12:17:22.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.102+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effa80b47c0 con 0x7effa80b58f0 2024-03-14T12:17:22.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.103+0000 7eff8ffff640 1 -- 172.21.15.82:0/1070106459 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7effa80b47c0 con 0x7effa80b58f0 2024-03-14T12:17:22.103 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:22.104 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:22.104 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:22.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7eff9406bbd0 msgr2=0x7eff9406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:22.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7eff9406bbd0 0x7eff9406e090 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff98036000 tx=0x7eff98030000 comp rx=0 tx=0).stop 2024-03-14T12:17:22.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80b58f0 msgr2=0x7effa814b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:22.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80b58f0 0x7effa814b910 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00d5f0 tx=0x7eff9c00dab0 comp rx=0 tx=0).stop 2024-03-14T12:17:22.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 shutdown_connections 2024-03-14T12:17:22.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7eff9406bbd0 0x7eff9406e090 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:22.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effa815f1f0 0x7effa814e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:22.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effa80b58f0 0x7effa814b910 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:22.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 --2- 172.21.15.82:0/1070106459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa80af150 0x7effa81e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:22.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 >> 172.21.15.82:0/1070106459 conn(0x7effa80d0a20 msgr2=0x7effa806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:22.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 shutdown_connections 2024-03-14T12:17:22.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:22.105+0000 7effad9f1640 1 -- 172.21.15.82:0/1070106459 wait complete. 2024-03-14T12:17:22.116 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:22.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:17:22.189 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:17:23.117 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:23.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.227+0000 7fb55d2ea640 1 Processor -- start 2024-03-14T12:17:23.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.227+0000 7fb55d2ea640 1 -- start start 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb55d2ea640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 0x7fb55815b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb55d2ea640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5580b3050 con 0x7fb558071740 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb55d2ea640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55815ebe0 con 0x7fb55815e670 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb55d2ea640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55815eec0 con 0x7fb558143f60 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb556ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb558071740 0x7fb558071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51186/0 (socket says 172.21.15.82:51186) 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb556ffd640 1 -- 172.21.15.82:0/2749233133 learned_addr learned my addr 172.21.15.82:0/2749233133 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb5577fe640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 0x7fb55815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399023870 0 0) 0x7fb5580b3050 con 0x7fb558071740 2024-03-14T12:17:23.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.228+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb544003520 con 0x7fb558071740 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3388588498 0 0) 0x7fb544003520 con 0x7fb558071740 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 msgr2=0x7fb55815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 0x7fb55815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 >> v1:172.21.15.164:6789/0 conn(0x7fb55815e670 legacy=0x7fb5580b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55815f1a0 con 0x7fb558071740 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fb548001670 con 0x7fb558071740 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 >> v1:172.21.15.82:6789/0 conn(0x7fb558071740 legacy=0x7fb558071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5440039e0 0x7fb544003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 0x7fb544004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55815eec0 con 0x7fb544004d40 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54400ba70 con 0x7fb5440039e0 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54400bc70 con 0x7fb558143f60 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5577fe640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb556ffd640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5440039e0 0x7fb544003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb54ffff640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 0x7fb544004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5577fe640 1 -- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 msgr2=0x7fb544004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5577fe640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558143f60 0x7fb544004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5577fe640 1 -- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5440039e0 msgr2=0x7fb544003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5577fe640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5440039e0 0x7fb544003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.229+0000 7fb5577fe640 1 -- 172.21.15.82:0/2749233133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544003520 con 0x7fb544004d40 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.230+0000 7fb5577fe640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb550009f90 tx=0x7fb544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbac92ba0f0c174 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.230+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550039070 con 0x7fb544004d40 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.230+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55002cbd0 con 0x7fb544004d40 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.230+0000 7fb5567fc640 1 -- 172.21.15.82:0/2749233133 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550009c10 con 0x7fb544004d40 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.231+0000 7fb55d2ea640 1 -- 172.21.15.82:0/2749233133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb544004d40 msgr2=0x7fb5440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.231+0000 7fb55d2ea640 1 --2- 172.21.15.82:0/2749233133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb550009f90 tx=0x7fb544005830 comp rx=0 tx=0).stop 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.231+0000 7fb5577fe640 1 -- 172.21.15.82:0/2749233133 reap_dead start 2024-03-14T12:17:23.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.231+0000 7fb55d2ea640 1 -- 172.21.15.82:0/2749233133 shutdown_connections 2024-03-14T12:17:23.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.231+0000 7fb55d2ea640 1 -- 172.21.15.82:0/2749233133 >> 172.21.15.82:0/2749233133 conn(0x7fb5580b1c70 msgr2=0x7fb5580cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:23.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.231+0000 7fb55d2ea640 1 -- 172.21.15.82:0/2749233133 shutdown_connections 2024-03-14T12:17:23.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.231+0000 7fb55d2ea640 1 -- 172.21.15.82:0/2749233133 wait complete. 2024-03-14T12:17:23.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 Processor -- start 2024-03-14T12:17:23.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 -- start start 2024-03-14T12:17:23.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb558157860 0x7fb55815dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb55815b510 0x7fb558135830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558135d70 0x7fb5580ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55815b080 con 0x7fb55815b510 2024-03-14T12:17:23.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558143ce0 con 0x7fb558157860 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.232+0000 7fb55d2ea640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb558144670 con 0x7fb558135d70 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb55815b510 0x7fb558135830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb55815b510 0x7fb558135830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58126/0 (socket says 172.21.15.82:58126) 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 -- 172.21.15.82:0/3588984473 learned_addr learned my addr 172.21.15.82:0/3588984473 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb556ffd640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb558157860 0x7fb55815dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb5577fe640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558135d70 0x7fb5580ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 -- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558135d70 msgr2=0x7fb5580ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558135d70 0x7fb5580ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 -- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb558157860 msgr2=0x7fb55815dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb558157860 0x7fb55815dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 -- 172.21.15.82:0/3588984473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb558133ac0 con 0x7fb55815b510 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb5577fe640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558135d70 0x7fb5580ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54ffff640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb55815b510 0x7fb558135830 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb54000f850 tx=0x7fb54000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54f7fe640 1 -- 172.21.15.82:0/3588984473 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000fe40 con 0x7fb55815b510 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54f7fe640 1 -- 172.21.15.82:0/3588984473 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb540018070 con 0x7fb55815b510 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb558133da0 con 0x7fb55815b510 2024-03-14T12:17:23.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb54f7fe640 1 -- 172.21.15.82:0/3588984473 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000d730 con 0x7fb55815b510 2024-03-14T12:17:23.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.233+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb558137520 con 0x7fb55815b510 2024-03-14T12:17:23.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.234+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb558136180 con 0x7fb55815b510 2024-03-14T12:17:23.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.235+0000 7fb54f7fe640 1 -- 172.21.15.82:0/3588984473 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb54000d8d0 con 0x7fb55815b510 2024-03-14T12:17:23.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.235+0000 7fb54f7fe640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb53806bbd0 0x7fb53806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:23.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.235+0000 7fb556ffd640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb53806bbd0 0x7fb53806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:23.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.235+0000 7fb54f7fe640 1 -- 172.21.15.82:0/3588984473 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb54008a880 con 0x7fb55815b510 2024-03-14T12:17:23.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.236+0000 7fb556ffd640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb53806bbd0 0x7fb53806e090 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb548004840 tx=0x7fb548009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:23.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.240+0000 7fb54f7fe640 1 -- 172.21.15.82:0/3588984473 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb558136180 con 0x7fb55815b510 2024-03-14T12:17:23.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.445+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb55804f050 con 0x7fb55815b510 2024-03-14T12:17:23.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.445+0000 7fb54f7fe640 1 -- 172.21.15.82:0/3588984473 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fb5400593a0 con 0x7fb55815b510 2024-03-14T12:17:23.446 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:23.446 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:23.446 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb53806bbd0 msgr2=0x7fb53806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb53806bbd0 0x7fb53806e090 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb548004840 tx=0x7fb548009040 comp rx=0 tx=0).stop 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb55815b510 msgr2=0x7fb558135830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb55815b510 0x7fb558135830 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb54000f850 tx=0x7fb54000fd10 comp rx=0 tx=0).stop 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 shutdown_connections 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb53806bbd0 0x7fb53806e090 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb558135d70 0x7fb5580ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb55815b510 0x7fb558135830 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 --2- 172.21.15.82:0/3588984473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb558157860 0x7fb55815dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.447+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 >> 172.21.15.82:0/3588984473 conn(0x7fb5580b1c70 msgr2=0x7fb5580ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.448+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 shutdown_connections 2024-03-14T12:17:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:23.448+0000 7fb55d2ea640 1 -- 172.21.15.82:0/3588984473 wait complete. 2024-03-14T12:17:23.459 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:24.460 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.564+0000 7f9a37f59640 1 Processor -- start 2024-03-14T12:17:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.564+0000 7f9a37f59640 1 -- start start 2024-03-14T12:17:24.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a37f59640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a3015be40 0x7f9a300d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:24.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a37f59640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9a30066b00 0x7f9a3006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:24.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a37f59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015c220 0x7f9a3006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a37f59640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a300cd750 con 0x7f9a3015be40 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a37f59640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a300ccb40 con 0x7f9a30066b00 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a37f59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300cce20 con 0x7f9a3015c220 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a3015be40 0x7f9a300d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a3015be40 0x7f9a300d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58142/0 (socket says 172.21.15.82:58142) 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 -- 172.21.15.82:0/1084685544 learned_addr learned my addr 172.21.15.82:0/1084685544 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a354f4640 1 --2- 172.21.15.82:0/1084685544 >> v2:172.21.15.164:3300/0 conn(0x7f9a30066b00 0x7f9a3006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a364f6640 1 --2- 172.21.15.82:0/1084685544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015c220 0x7f9a3006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 -- 172.21.15.82:0/1084685544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015c220 msgr2=0x7f9a3006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:24.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 --2- 172.21.15.82:0/1084685544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015c220 0x7f9a3006b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 -- 172.21.15.82:0/1084685544 >> v2:172.21.15.164:3300/0 conn(0x7f9a30066b00 msgr2=0x7f9a3006ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 --2- 172.21.15.82:0/1084685544 >> v2:172.21.15.164:3300/0 conn(0x7f9a30066b00 0x7f9a3006ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.565+0000 7f9a35cf5640 1 -- 172.21.15.82:0/1084685544 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a300cd100 con 0x7f9a3015be40 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.566+0000 7f9a35cf5640 1 --2- 172.21.15.82:0/1084685544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3015be40 0x7f9a300d0a30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9a300afde0 tx=0x7f9a2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca22c4e0b8cf2a39 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.566+0000 7f9a34cf3640 1 -- 172.21.15.82:0/1084685544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c035020 con 0x7f9a3015be40 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.566+0000 7f9a34cf3640 1 -- 172.21.15.82:0/1084685544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c031870 con 0x7f9a3015be40 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.566+0000 7f9a34cf3640 1 -- 172.21.15.82:0/1084685544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c031c40 con 0x7f9a3015be40 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 -- 172.21.15.82:0/1084685544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3015be40 msgr2=0x7f9a300d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 --2- 172.21.15.82:0/1084685544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3015be40 0x7f9a300d0a30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9a300afde0 tx=0x7f9a2c02b910 comp rx=0 tx=0).stop 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 -- 172.21.15.82:0/1084685544 shutdown_connections 2024-03-14T12:17:24.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 --2- 172.21.15.82:0/1084685544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015c220 0x7f9a3006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 --2- 172.21.15.82:0/1084685544 >> v2:172.21.15.164:3300/0 conn(0x7f9a30066b00 0x7f9a3006ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 --2- 172.21.15.82:0/1084685544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3015be40 0x7f9a300d0a30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 -- 172.21.15.82:0/1084685544 >> 172.21.15.82:0/1084685544 conn(0x7f9a30072310 msgr2=0x7f9a30072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:24.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 -- 172.21.15.82:0/1084685544 shutdown_connections 2024-03-14T12:17:24.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.567+0000 7f9a37f59640 1 -- 172.21.15.82:0/1084685544 wait complete. 2024-03-14T12:17:24.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.568+0000 7f9a37f59640 1 Processor -- start 2024-03-14T12:17:24.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.568+0000 7f9a37f59640 1 -- start start 2024-03-14T12:17:24.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.568+0000 7f9a37f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a30066b00 0x7f9a3009dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.568+0000 7f9a37f59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015be40 0x7f9a3015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a37f59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3015c220 0x7f9a30159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a37f59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300cb860 con 0x7f9a30066b00 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a37f59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300d0f70 con 0x7f9a3015c220 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a37f59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300cb9e0 con 0x7f9a3015be40 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a30066b00 0x7f9a3009dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a30066b00 0x7f9a3009dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58158/0 (socket says 172.21.15.82:58158) 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 -- 172.21.15.82:0/807325827 learned_addr learned my addr 172.21.15.82:0/807325827 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 -- 172.21.15.82:0/807325827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015be40 msgr2=0x7f9a3015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:24.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015be40 0x7f9a3015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 -- 172.21.15.82:0/807325827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3015c220 msgr2=0x7f9a30159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a364f6640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3015c220 0x7f9a30159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3015c220 0x7f9a30159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 -- 172.21.15.82:0/807325827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30156950 con 0x7f9a30066b00 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a35cf5640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a30066b00 0x7f9a3009dc90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c02b950 tx=0x7f9a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a267fc640 1 -- 172.21.15.82:0/807325827 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c040070 con 0x7f9a30066b00 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a267fc640 1 -- 172.21.15.82:0/807325827 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c030d00 con 0x7f9a30066b00 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a267fc640 1 -- 172.21.15.82:0/807325827 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c035040 con 0x7f9a30066b00 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a300c7560 con 0x7f9a30066b00 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.569+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a300c7b20 con 0x7f9a30066b00 2024-03-14T12:17:24.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.570+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3004f050 con 0x7f9a30066b00 2024-03-14T12:17:24.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.571+0000 7f9a267fc640 1 -- 172.21.15.82:0/807325827 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9a2c04b060 con 0x7f9a30066b00 2024-03-14T12:17:24.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.571+0000 7f9a267fc640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9a1406bbd0 0x7f9a1406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:24.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.571+0000 7f9a267fc640 1 -- 172.21.15.82:0/807325827 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9a2c0ac8c0 con 0x7f9a30066b00 2024-03-14T12:17:24.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.571+0000 7f9a354f4640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9a1406bbd0 0x7f9a1406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:24.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.571+0000 7f9a354f4640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9a1406bbd0 0x7f9a1406e090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9a300b2e90 tx=0x7f9a20008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:24.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.575+0000 7f9a267fc640 1 -- 172.21.15.82:0/807325827 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9a30156950 con 0x7f9a30066b00 2024-03-14T12:17:24.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.781+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a300c8520 con 0x7f9a30066b00 2024-03-14T12:17:24.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.782+0000 7f9a267fc640 1 -- 172.21.15.82:0/807325827 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f9a2c0ac070 con 0x7f9a30066b00 2024-03-14T12:17:24.782 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:24.783 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:24.783 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9a1406bbd0 msgr2=0x7f9a1406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9a1406bbd0 0x7f9a1406e090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9a300b2e90 tx=0x7f9a20008040 comp rx=0 tx=0).stop 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a30066b00 msgr2=0x7f9a3009dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a30066b00 0x7f9a3009dc90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c02b950 tx=0x7f9a2c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 shutdown_connections 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a3015c220 0x7f9a30159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9a1406bbd0 0x7f9a1406e090 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a3015be40 0x7f9a3015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 --2- 172.21.15.82:0/807325827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a30066b00 0x7f9a3009dc90 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 >> 172.21.15.82:0/807325827 conn(0x7f9a30072310 msgr2=0x7f9a300cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 shutdown_connections 2024-03-14T12:17:24.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:24.784+0000 7f9a37f59640 1 -- 172.21.15.82:0/807325827 wait complete. 2024-03-14T12:17:24.795 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:25.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:25.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.903+0000 7f71bc909640 1 Processor -- start 2024-03-14T12:17:25.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.903+0000 7f71bc909640 1 -- start start 2024-03-14T12:17:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71bc909640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 0x7f71b80afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71bc909640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b80b1340 con 0x7f71b80b0f60 2024-03-14T12:17:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71bc909640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b80cd030 con 0x7f71b806b420 2024-03-14T12:17:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71bc909640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b4ee0 con 0x7f71b80cc9a0 2024-03-14T12:17:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71b659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f71b80b0f60 0x7f71b806ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51198/0 (socket says 172.21.15.82:51198) 2024-03-14T12:17:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71b659c640 1 -- 172.21.15.82:0/2369207562 learned_addr learned my addr 172.21.15.82:0/2369207562 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71b6d9d640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 0x7f71b80afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.904+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508535582 0 0) 0x7f71b80b1340 con 0x7f71b80b0f60 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a0003520 con 0x7f71b80b0f60 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732230176 0 0) 0x7f71b80cd030 con 0x7f71b806b420 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b80b1340 con 0x7f71b806b420 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 890826226 0 0) 0x7f71a0003520 con 0x7f71b80b0f60 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 msgr2=0x7f71b80afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 0x7f71b80afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 >> v1:172.21.15.164:6789/0 conn(0x7f71b806b420 legacy=0x7f71b80b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80cd310 con 0x7f71b80b0f60 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f71a8001670 con 0x7f71b80b0f60 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 >> v1:172.21.15.82:6789/0 conn(0x7f71b80b0f60 legacy=0x7f71b806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71a0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71a0004800 0x7f71a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 0x7f71a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b4ee0 con 0x7f71a0003b70 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a000ba70 con 0x7f71a0004800 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a000bc70 con 0x7f71b80cc9a0 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b659c640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71a0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b5d9b640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71a0004800 0x7f71a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b6d9d640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 0x7f71a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b659c640 1 -- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 msgr2=0x7f71a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b659c640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b80cc9a0 0x7f71a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b659c640 1 -- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71a0004800 msgr2=0x7f71a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b659c640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71a0004800 0x7f71a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.905+0000 7f71b659c640 1 -- 172.21.15.82:0/2369207562 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0003520 con 0x7f71a0003b70 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.906+0000 7f71b659c640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71a0003f70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c190 tx=0x7f71a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30eb4ac10b451444 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.906+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8007cc0 con 0x7f71a0003b70 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.906+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8030070 con 0x7f71a0003b70 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.906+0000 7f71b559a640 1 -- 172.21.15.82:0/2369207562 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8005d30 con 0x7f71a0003b70 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.907+0000 7f71bc909640 1 -- 172.21.15.82:0/2369207562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 msgr2=0x7f71a0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.907+0000 7f71bc909640 1 --2- 172.21.15.82:0/2369207562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71a0003f70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c190 tx=0x7f71a0005830 comp rx=0 tx=0).stop 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.907+0000 7f71b6d9d640 1 -- 172.21.15.82:0/2369207562 reap_dead start 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.907+0000 7f71bc909640 1 -- 172.21.15.82:0/2369207562 shutdown_connections 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.907+0000 7f71bc909640 1 -- 172.21.15.82:0/2369207562 >> 172.21.15.82:0/2369207562 conn(0x7f71b8072310 msgr2=0x7f71b8072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.907+0000 7f71bc909640 1 -- 172.21.15.82:0/2369207562 shutdown_connections 2024-03-14T12:17:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.907+0000 7f71bc909640 1 -- 172.21.15.82:0/2369207562 wait complete. 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.908+0000 7f71bc909640 1 Processor -- start 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.908+0000 7f71bc909640 1 -- start start 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71bc909640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71b809cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71bc909640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8099930 0x7f71b81530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71bc909640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b806e140 0x7f71b806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71bc909640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80afff0 con 0x7f71a0003b70 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71bc909640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80cd8d0 con 0x7f71b8099930 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71bc909640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b0170 con 0x7f71b806e140 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71b809cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71b809cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58170/0 (socket says 172.21.15.82:58170) 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 -- 172.21.15.82:0/4123728287 learned_addr learned my addr 172.21.15.82:0/4123728287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b6d9d640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b806e140 0x7f71b806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b5d9b640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8099930 0x7f71b81530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 -- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b806e140 msgr2=0x7f71b806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b806e140 0x7f71b806e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 -- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8099930 msgr2=0x7f71b81530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8099930 0x7f71b81530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.909+0000 7f71b659c640 1 -- 172.21.15.82:0/4123728287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b81535f0 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.910+0000 7f71b659c640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71b809cc00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f71a80018f0 tx=0x7f71a80019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.910+0000 7f71a6ffd640 1 -- 172.21.15.82:0/4123728287 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8007cc0 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.910+0000 7f71a6ffd640 1 -- 172.21.15.82:0/4123728287 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8004bf0 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.910+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b809d230 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.910+0000 7f71a6ffd640 1 -- 172.21.15.82:0/4123728287 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8001c10 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.910+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b809d7f0 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.911+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b804f050 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.911+0000 7f71a6ffd640 1 -- 172.21.15.82:0/4123728287 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f71a8047020 con 0x7f71a0003b70 2024-03-14T12:17:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.911+0000 7f71a6ffd640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f719806bbd0 0x7f719806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.912+0000 7f71a6ffd640 1 -- 172.21.15.82:0/4123728287 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f71a80abca0 con 0x7f71a0003b70 2024-03-14T12:17:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.912+0000 7f71b5d9b640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f719806bbd0 0x7f719806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.912+0000 7f71b5d9b640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f719806bbd0 0x7f719806e090 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71b80cf760 tx=0x7f71ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:25.916+0000 7f71a6ffd640 1 -- 172.21.15.82:0/4123728287 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f71a807a730 con 0x7f71a0003b70 2024-03-14T12:17:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.124+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71b8071200 con 0x7f71a0003b70 2024-03-14T12:17:26.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.124+0000 7f71a6ffd640 1 -- 172.21.15.82:0/4123728287 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f71a807a030 con 0x7f71a0003b70 2024-03-14T12:17:26.125 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:26.125 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:26.125 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f719806bbd0 msgr2=0x7f719806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f719806bbd0 0x7f719806e090 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71b80cf760 tx=0x7f71ac009040 comp rx=0 tx=0).stop 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 msgr2=0x7f71b809cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71b809cc00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f71a80018f0 tx=0x7f71a80019c0 comp rx=0 tx=0).stop 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 shutdown_connections 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f719806bbd0 0x7f719806e090 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f71b806e140 0x7f71b806e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b8099930 0x7f71b81530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 --2- 172.21.15.82:0/4123728287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0003b70 0x7f71b809cc00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 >> 172.21.15.82:0/4123728287 conn(0x7f71b8072310 msgr2=0x7f71b80cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 shutdown_connections 2024-03-14T12:17:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:26.126+0000 7f71bc909640 1 -- 172.21.15.82:0/4123728287 wait complete. 2024-03-14T12:17:26.137 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:27.138 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:27.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.250+0000 7fc027e4b640 1 Processor -- start 2024-03-14T12:17:27.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.251+0000 7fc027e4b640 1 -- start start 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc027e4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 0x7fc0200b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc027e4b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0200b53f0 con 0x7fc020158750 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc025be7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc020158750 0x7fc02015bba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51204/0 (socket says 172.21.15.82:51204) 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc027e4b640 1 -- 172.21.15.82:0/1091961999 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc020158b30 con 0x7fc0200716b0 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc027e4b640 1 -- 172.21.15.82:0/1091961999 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200b4e30 con 0x7fc02006efe0 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc025be7640 1 -- 172.21.15.82:0/1091961999 learned_addr learned my addr 172.21.15.82:0/1091961999 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc0253e6640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 0x7fc0200b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613710003 0 0) 0x7fc0200b53f0 con 0x7fc020158750 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc00c003520 con 0x7fc020158750 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113202972 0 0) 0x7fc00c003520 con 0x7fc020158750 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 msgr2=0x7fc0200b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 0x7fc0200b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 >> v1:172.21.15.164:6789/0 conn(0x7fc0200716b0 legacy=0x7fc020164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0200b5110 con 0x7fc020158750 2024-03-14T12:17:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fc01c001a10 con 0x7fc020158750 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 >> v1:172.21.15.82:6789/0 conn(0x7fc020158750 legacy=0x7fc02015bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0039e0 0x7fc00c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c004800 0x7fc00c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 0x7fc00c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200b4e30 con 0x7fc00c0039e0 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c00ba70 con 0x7fc00c004800 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c00bc70 con 0x7fc02006efe0 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc025be7640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0039e0 0x7fc00c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc0253e6640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c004800 0x7fc00c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.252+0000 7fc0263e8640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 0x7fc00c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.253+0000 7fc025be7640 1 -- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 msgr2=0x7fc00c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.253+0000 7fc025be7640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc02006efe0 0x7fc00c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.253+0000 7fc025be7640 1 -- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c004800 msgr2=0x7fc00c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.253+0000 7fc025be7640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c004800 0x7fc00c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.253+0000 7fc025be7640 1 -- 172.21.15.82:0/1091961999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c003520 con 0x7fc00c0039e0 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.253+0000 7fc025be7640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0039e0 0x7fc00c003e00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc0200b55c0 tx=0x7fc00c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f22678e7af13276 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00a480 con 0x7fc00c0039e0 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c034650 con 0x7fc00c0039e0 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc024be5640 1 -- 172.21.15.82:0/1091961999 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c02ce60 con 0x7fc00c0039e0 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc027e4b640 1 -- 172.21.15.82:0/1091961999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0039e0 msgr2=0x7fc00c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc027e4b640 1 --2- 172.21.15.82:0/1091961999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0039e0 0x7fc00c003e00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc0200b55c0 tx=0x7fc00c005830 comp rx=0 tx=0).stop 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc0263e8640 1 -- 172.21.15.82:0/1091961999 reap_dead start 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc027e4b640 1 -- 172.21.15.82:0/1091961999 shutdown_connections 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc027e4b640 1 -- 172.21.15.82:0/1091961999 >> 172.21.15.82:0/1091961999 conn(0x7fc0200d0a20 msgr2=0x7fc0200d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc027e4b640 1 -- 172.21.15.82:0/1091961999 shutdown_connections 2024-03-14T12:17:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.254+0000 7fc027e4b640 1 -- 172.21.15.82:0/1091961999 wait complete. 2024-03-14T12:17:27.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 Processor -- start 2024-03-14T12:17:27.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 -- start start 2024-03-14T12:17:27.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c0039e0 0x7fc020140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc020143ee0 0x7fc020159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc020144360 0x7fc02015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200b47e0 con 0x7fc020143ee0 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02006dab0 con 0x7fc00c0039e0 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc027e4b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200b4960 con 0x7fc020144360 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc0253e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc020143ee0 0x7fc020159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc0253e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc020143ee0 0x7fc020159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58194/0 (socket says 172.21.15.82:58194) 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.256+0000 7fc0253e6640 1 -- 172.21.15.82:0/3367075930 learned_addr learned my addr 172.21.15.82:0/3367075930 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0263e8640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc020144360 0x7fc02015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc025be7640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c0039e0 0x7fc020140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0253e6640 1 -- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc020144360 msgr2=0x7fc02015a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0253e6640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc020144360 0x7fc02015a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0253e6640 1 -- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c0039e0 msgr2=0x7fc020140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0253e6640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c0039e0 0x7fc020140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0253e6640 1 -- 172.21.15.82:0/3367075930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0201405c0 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc025be7640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c0039e0 0x7fc020140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0253e6640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc020143ee0 0x7fc020159f80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc0100119a0 tx=0x7fc010011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0167fc640 1 -- 172.21.15.82:0/3367075930 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000faf0 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0167fc640 1 -- 172.21.15.82:0/3367075930 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01000fc90 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc0167fc640 1 -- 172.21.15.82:0/3367075930 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010014440 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc020167f20 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.257+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0201684e0 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.258+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02004f050 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.259+0000 7fc0167fc640 1 -- 172.21.15.82:0/3367075930 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc010012040 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.259+0000 7fc0167fc640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc00406bbd0 0x7fc00406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.259+0000 7fc0167fc640 1 -- 172.21.15.82:0/3367075930 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc01008af00 con 0x7fc020143ee0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.259+0000 7fc025be7640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc00406bbd0 0x7fc00406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.260+0000 7fc025be7640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc00406bbd0 0x7fc00406e090 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc01c034000 tx=0x7fc01c009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:27.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.263+0000 7fc0167fc640 1 -- 172.21.15.82:0/3367075930 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc010019030 con 0x7fc020143ee0 2024-03-14T12:17:27.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.472+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc020157f70 con 0x7fc020143ee0 2024-03-14T12:17:27.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.472+0000 7fc0167fc640 1 -- 172.21.15.82:0/3367075930 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fc010059a20 con 0x7fc020143ee0 2024-03-14T12:17:27.473 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:27.473 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:27.473 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc00406bbd0 msgr2=0x7fc00406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc00406bbd0 0x7fc00406e090 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc01c034000 tx=0x7fc01c009030 comp rx=0 tx=0).stop 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc020143ee0 msgr2=0x7fc020159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc020143ee0 0x7fc020159f80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc0100119a0 tx=0x7fc010011e60 comp rx=0 tx=0).stop 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 shutdown_connections 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc00406bbd0 0x7fc00406e090 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc020144360 0x7fc02015a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc020143ee0 0x7fc020159f80 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 --2- 172.21.15.82:0/3367075930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc00c0039e0 0x7fc020140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:27.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 >> 172.21.15.82:0/3367075930 conn(0x7fc0200d0a20 msgr2=0x7fc02006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:27.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 shutdown_connections 2024-03-14T12:17:27.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:27.474+0000 7fc027e4b640 1 -- 172.21.15.82:0/3367075930 wait complete. 2024-03-14T12:17:27.486 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:28.487 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:28.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.596+0000 7f6a80ad1640 1 Processor -- start 2024-03-14T12:17:28.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.596+0000 7f6a80ad1640 1 -- start start 2024-03-14T12:17:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.597+0000 7f6a80ad1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a7c0b3fe0 0x7f6a7c0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.597+0000 7f6a80ad1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c070d70 con 0x7f6a7c0cdbb0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.597+0000 7f6a80ad1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c071050 con 0x7f6a7c15b9a0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.597+0000 7f6a80ad1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0cfb10 con 0x7f6a7c0b3fe0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.597+0000 7f6a79d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a7c0cdbb0 0x7f6a7c0cf400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51206/0 (socket says 172.21.15.82:51206) 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.597+0000 7f6a79d9b640 1 -- 172.21.15.82:0/3768889828 learned_addr learned my addr 172.21.15.82:0/3768889828 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.597+0000 7f6a7ad9d640 1 --2- 172.21.15.82:0/3768889828 >> v2:172.21.15.164:3300/0 conn(0x7f6a7c0b3fe0 0x7f6a7c0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795923126 0 0) 0x7f6a7c070d70 con 0x7f6a7c0cdbb0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a64003520 con 0x7f6a7c0cdbb0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1807642977 0 0) 0x7f6a64003520 con 0x7f6a7c0cdbb0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 >> v2:172.21.15.164:3300/0 conn(0x7f6a7c0b3fe0 msgr2=0x7f6a7c0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 --2- 172.21.15.82:0/3768889828 >> v2:172.21.15.164:3300/0 conn(0x7f6a7c0b3fe0 0x7f6a7c0b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 >> v1:172.21.15.164:6789/0 conn(0x7f6a7c15b9a0 legacy=0x7f6a7c15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c071330 con 0x7f6a7c0cdbb0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6a70001670 con 0x7f6a7c0cdbb0 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 >> v1:172.21.15.82:6789/0 conn(0x7f6a7c0cdbb0 legacy=0x7f6a7c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a640039b0 0x7f6a64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a64004800 0x7f6a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a640051c0 0x7f6a6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0cfb10 con 0x7f6a64004800 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6400c060 con 0x7f6a640039b0 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6400c260 con 0x7f6a640051c0 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a79d9b640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a64004800 0x7f6a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7a59c640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a640039b0 0x7f6a64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7ad9d640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a640051c0 0x7f6a6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a79d9b640 1 -- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a640051c0 msgr2=0x7f6a6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a79d9b640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a640051c0 0x7f6a6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a79d9b640 1 -- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a640039b0 msgr2=0x7f6a64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a79d9b640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a640039b0 0x7f6a64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a79d9b640 1 -- 172.21.15.82:0/3768889828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64003520 con 0x7f6a64004800 2024-03-14T12:17:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.598+0000 7f6a7ad9d640 1 -- 172.21.15.82:0/3768889828 reap_dead start 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.599+0000 7f6a79d9b640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a64004800 0x7f6a64004c80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c240 tx=0x7f6a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfc10955d5bb849 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.599+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70007cf0 con 0x7f6a64004800 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.599+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a70030070 con 0x7f6a64004800 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.599+0000 7f6a7959a640 1 -- 172.21.15.82:0/3768889828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70005d70 con 0x7f6a64004800 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.600+0000 7f6a80ad1640 1 -- 172.21.15.82:0/3768889828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a64004800 msgr2=0x7f6a64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.600+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a64004800 0x7f6a64004c80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c240 tx=0x7f6a64005830 comp rx=0 tx=0).stop 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.600+0000 7f6a80ad1640 1 -- 172.21.15.82:0/3768889828 shutdown_connections 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.600+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/3768889828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a64004800 0x7f6a64004c80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.600+0000 7f6a80ad1640 1 -- 172.21.15.82:0/3768889828 >> 172.21.15.82:0/3768889828 conn(0x7f6a7c0b1bd0 msgr2=0x7f6a7c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.600+0000 7f6a80ad1640 1 -- 172.21.15.82:0/3768889828 shutdown_connections 2024-03-14T12:17:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.600+0000 7f6a80ad1640 1 -- 172.21.15.82:0/3768889828 wait complete. 2024-03-14T12:17:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.601+0000 7f6a80ad1640 1 Processor -- start 2024-03-14T12:17:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.601+0000 7f6a80ad1640 1 -- start start 2024-03-14T12:17:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a80ad1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a64004800 0x7f6a7c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a80ad1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c13c960 0x7f6a7c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a80ad1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a7c06d760 0x7f6a7c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a80ad1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b4420 con 0x7f6a7c13c960 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a80ad1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c070b20 con 0x7f6a64004800 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a80ad1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b45a0 con 0x7f6a7c06d760 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c13c960 0x7f6a7c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c13c960 0x7f6a7c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58220/0 (socket says 172.21.15.82:58220) 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 -- 172.21.15.82:0/1677403936 learned_addr learned my addr 172.21.15.82:0/1677403936 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a7ad9d640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a7c06d760 0x7f6a7c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a7a59c640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a64004800 0x7f6a7c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 -- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a7c06d760 msgr2=0x7f6a7c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a7c06d760 0x7f6a7c06dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 -- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a64004800 msgr2=0x7f6a7c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a64004800 0x7f6a7c1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.602+0000 7f6a79d9b640 1 -- 172.21.15.82:0/1677403936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c158dc0 con 0x7f6a7c13c960 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.603+0000 7f6a79d9b640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c13c960 0x7f6a7c156de0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c360 tx=0x7f6a7c071220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.603+0000 7f6a6affd640 1 -- 172.21.15.82:0/1677403936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70046070 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.603+0000 7f6a6affd640 1 -- 172.21.15.82:0/1677403936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a70000ed0 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.603+0000 7f6a6affd640 1 -- 172.21.15.82:0/1677403936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004e10 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.603+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c159060 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.603+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c159620 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.604+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c04f050 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.604+0000 7f6a6affd640 1 -- 172.21.15.82:0/1677403936 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6a70007070 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.605+0000 7f6a6affd640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a5c06bc00 0x7f6a5c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.605+0000 7f6a7a59c640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a5c06bc00 0x7f6a5c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.605+0000 7f6a6affd640 1 -- 172.21.15.82:0/1677403936 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6a700abd60 con 0x7f6a7c13c960 2024-03-14T12:17:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.605+0000 7f6a7a59c640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a5c06bc00 0x7f6a5c06e0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c160710 tx=0x7f6a6c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.609+0000 7f6a6affd640 1 -- 172.21.15.82:0/1677403936 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6a7007a1a0 con 0x7f6a7c13c960 2024-03-14T12:17:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.817+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a7c152070 con 0x7f6a7c13c960 2024-03-14T12:17:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.817+0000 7f6a6affd640 1 -- 172.21.15.82:0/1677403936 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f6a7007e240 con 0x7f6a7c13c960 2024-03-14T12:17:28.818 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:28.818 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a5c06bc00 msgr2=0x7f6a5c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a5c06bc00 0x7f6a5c06e0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c160710 tx=0x7f6a6c009040 comp rx=0 tx=0).stop 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c13c960 msgr2=0x7f6a7c156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c13c960 0x7f6a7c156de0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c360 tx=0x7f6a7c071220 comp rx=0 tx=0).stop 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 shutdown_connections 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a5c06bc00 0x7f6a5c06e0c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a7c06d760 0x7f6a7c06dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c13c960 0x7f6a7c156de0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 --2- 172.21.15.82:0/1677403936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a64004800 0x7f6a7c1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 >> 172.21.15.82:0/1677403936 conn(0x7f6a7c0b1bd0 msgr2=0x7f6a7c150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 shutdown_connections 2024-03-14T12:17:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:28.819+0000 7f6a80ad1640 1 -- 172.21.15.82:0/1677403936 wait complete. 2024-03-14T12:17:28.831 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:29.831 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.942+0000 7ff98d737640 1 Processor -- start 2024-03-14T12:17:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.942+0000 7ff98d737640 1 -- start start 2024-03-14T12:17:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.943+0000 7ff98d737640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9880d0e80 0x7ff9880c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.943+0000 7ff98d737640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880cfcf0 con 0x7ff9880d2390 2024-03-14T12:17:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.943+0000 7ff98d737640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880d4a50 con 0x7ff9880d1b90 2024-03-14T12:17:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.943+0000 7ff98d737640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9880d4d30 con 0x7ff9880d0e80 2024-03-14T12:17:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.943+0000 7ff9877fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9880d0e80 0x7ff9880c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.943+0000 7ff986ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff9880d2390 0x7ff9880d2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51220/0 (socket says 172.21.15.82:51220) 2024-03-14T12:17:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.943+0000 7ff986ffd640 1 -- 172.21.15.82:0/342755764 learned_addr learned my addr 172.21.15.82:0/342755764 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.944+0000 7ff9877fe640 1 -- 172.21.15.82:0/342755764 >> v1:172.21.15.164:6789/0 conn(0x7ff9880d1b90 legacy=0x7ff9880c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.944+0000 7ff9877fe640 1 -- 172.21.15.82:0/342755764 >> v1:172.21.15.82:6789/0 conn(0x7ff9880d2390 legacy=0x7ff9880d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.944+0000 7ff9877fe640 1 -- 172.21.15.82:0/342755764 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9880c7a30 con 0x7ff9880d0e80 2024-03-14T12:17:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.944+0000 7ff985ffb640 1 -- 172.21.15.82:0/342755764 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419055402 0 0) 0x7ff9880cfcf0 con 0x7ff9880d2390 2024-03-14T12:17:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.944+0000 7ff9877fe640 1 --2- 172.21.15.82:0/342755764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9880c8c20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097a0 tx=0x7ff98002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b73d3a1b4c770a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff985ffb640 1 -- 172.21.15.82:0/342755764 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9800315a0 con 0x7ff9880d0e80 2024-03-14T12:17:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff985ffb640 1 -- 172.21.15.82:0/342755764 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff980037070 con 0x7ff9880d0e80 2024-03-14T12:17:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff985ffb640 1 -- 172.21.15.82:0/342755764 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980031960 con 0x7ff9880d0e80 2024-03-14T12:17:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff98d737640 1 -- 172.21.15.82:0/342755764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 msgr2=0x7ff9880c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff98d737640 1 --2- 172.21.15.82:0/342755764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9880c8c20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097a0 tx=0x7ff98002b840 comp rx=0 tx=0).stop 2024-03-14T12:17:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff98d737640 1 -- 172.21.15.82:0/342755764 shutdown_connections 2024-03-14T12:17:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff98d737640 1 --2- 172.21.15.82:0/342755764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9880c8c20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff98d737640 1 -- 172.21.15.82:0/342755764 >> 172.21.15.82:0/342755764 conn(0x7ff9880d3db0 msgr2=0x7ff9880d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff98d737640 1 -- 172.21.15.82:0/342755764 shutdown_connections 2024-03-14T12:17:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.945+0000 7ff98d737640 1 -- 172.21.15.82:0/342755764 wait complete. 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 Processor -- start 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 -- start start 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9881f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9880d1b90 0x7ff9881f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9880d2390 0x7ff9880cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880ca590 con 0x7ff9880d0e80 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c7ff0 con 0x7ff9880d2390 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff98d737640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880ca710 con 0x7ff9880d1b90 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9881f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9881f6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58242/0 (socket says 172.21.15.82:58242) 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.947+0000 7ff986ffd640 1 -- 172.21.15.82:0/4252477461 learned_addr learned my addr 172.21.15.82:0/4252477461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff9867fc640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9880d1b90 0x7ff9881f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff9877fe640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9880d2390 0x7ff9880cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff986ffd640 1 -- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9880d1b90 msgr2=0x7ff9881f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff986ffd640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9880d1b90 0x7ff9881f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff986ffd640 1 -- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9880d2390 msgr2=0x7ff9880cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff986ffd640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9880d2390 0x7ff9880cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff986ffd640 1 -- 172.21.15.82:0/4252477461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9881f5560 con 0x7ff9880d0e80 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff9877fe640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9880d2390 0x7ff9880cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff986ffd640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9881f6600 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c7f0 tx=0x7ff97800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff96f7fe640 1 -- 172.21.15.82:0/4252477461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978018070 con 0x7ff9880d0e80 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff96f7fe640 1 -- 172.21.15.82:0/4252477461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800a5e0 con 0x7ff9880d0e80 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9881f5840 con 0x7ff9880d0e80 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff96f7fe640 1 -- 172.21.15.82:0/4252477461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800a9d0 con 0x7ff9880d0e80 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.948+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9881f5e00 con 0x7ff9880d0e80 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.949+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98804f050 con 0x7ff9880d0e80 2024-03-14T12:17:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.950+0000 7ff96f7fe640 1 -- 172.21.15.82:0/4252477461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff97801b070 con 0x7ff9880d0e80 2024-03-14T12:17:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.950+0000 7ff96f7fe640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff97406bbd0 0x7ff97406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.950+0000 7ff96f7fe640 1 -- 172.21.15.82:0/4252477461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff97808a050 con 0x7ff9880d0e80 2024-03-14T12:17:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.950+0000 7ff9867fc640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff97406bbd0 0x7ff97406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.950+0000 7ff9867fc640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff97406bbd0 0x7ff97406e090 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff97c007fd0 tx=0x7ff97c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:29.954+0000 7ff96f7fe640 1 -- 172.21.15.82:0/4252477461 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff978058b70 con 0x7ff9880d0e80 2024-03-14T12:17:30.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.159+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9880d2170 con 0x7ff9880d0e80 2024-03-14T12:17:30.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.160+0000 7ff96f7fe640 1 -- 172.21.15.82:0/4252477461 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff978058470 con 0x7ff9880d0e80 2024-03-14T12:17:30.162 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:30.162 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:30.165 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:30.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff97406bbd0 msgr2=0x7ff97406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:30.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff97406bbd0 0x7ff97406e090 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff97c007fd0 tx=0x7ff97c009040 comp rx=0 tx=0).stop 2024-03-14T12:17:30.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 msgr2=0x7ff9881f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:30.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9881f6600 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c7f0 tx=0x7ff97800ccb0 comp rx=0 tx=0).stop 2024-03-14T12:17:30.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 shutdown_connections 2024-03-14T12:17:30.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9880d2390 0x7ff9880cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:30.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff97406bbd0 0x7ff97406e090 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:30.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9880d1b90 0x7ff9881f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:30.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 --2- 172.21.15.82:0/4252477461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9880d0e80 0x7ff9881f6600 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:30.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 >> 172.21.15.82:0/4252477461 conn(0x7ff9880d3db0 msgr2=0x7ff98806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:30.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 shutdown_connections 2024-03-14T12:17:30.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:30.162+0000 7ff98d737640 1 -- 172.21.15.82:0/4252477461 wait complete. 2024-03-14T12:17:30.174 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:31.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:31.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.286+0000 7efe5244b640 1 Processor -- start 2024-03-14T12:17:31.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.286+0000 7efe5244b640 1 -- start start 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe5244b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe4c0d0760 0x7efe4c0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe5244b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0cf370 0x7efe4c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe5244b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c0d1080 con 0x7efe4c0d1da0 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe5244b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4c0cf8c0 con 0x7efe4c0d0760 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe5244b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0cfba0 con 0x7efe4c0cf370 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0cf370 0x7efe4c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe4c0d0760 0x7efe4c0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0cf370 0x7efe4c0d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:59866/0 (socket says 172.21.15.82:59866) 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe509e8640 1 --1- 172.21.15.82:0/1424445329 >> v1:172.21.15.164:6789/0 conn(0x7efe4c0d1da0 0x7efe4c0d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:47046/0 (socket says 172.21.15.82:47046) 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4b7fe640 1 -- 172.21.15.82:0/1424445329 learned_addr learned my addr 172.21.15.82:0/1424445329 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4b7fe640 1 -- 172.21.15.82:0/1424445329 >> v2:172.21.15.164:3300/0 conn(0x7efe4c0d0760 msgr2=0x7efe4c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4b7fe640 1 --2- 172.21.15.82:0/1424445329 >> v2:172.21.15.164:3300/0 conn(0x7efe4c0d0760 0x7efe4c0d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4b7fe640 1 -- 172.21.15.82:0/1424445329 >> v1:172.21.15.164:6789/0 conn(0x7efe4c0d1da0 legacy=0x7efe4c0d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4affd640 1 -- 172.21.15.82:0/1424445329 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049202736 0 0) 0x7efe4c0d1080 con 0x7efe4c0d1da0 2024-03-14T12:17:31.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4b7fe640 1 -- 172.21.15.82:0/1424445329 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0b6c40 con 0x7efe4c0cf370 2024-03-14T12:17:31.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.287+0000 7efe4bfff640 1 --2- 172.21.15.82:0/1424445329 >> v2:172.21.15.164:3300/0 conn(0x7efe4c0d0760 0x7efe4c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:31.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.288+0000 7efe4b7fe640 1 --2- 172.21.15.82:0/1424445329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0cf370 0x7efe4c0d4660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe40009870 tx=0x7efe4002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba92a5547900d47e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:31.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.288+0000 7efe4affd640 1 -- 172.21.15.82:0/1424445329 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4002f840 con 0x7efe4c0cf370 2024-03-14T12:17:31.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.288+0000 7efe4affd640 1 -- 172.21.15.82:0/1424445329 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4002f9e0 con 0x7efe4c0cf370 2024-03-14T12:17:31.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.288+0000 7efe4affd640 1 -- 172.21.15.82:0/1424445329 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4002fdb0 con 0x7efe4c0cf370 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 -- 172.21.15.82:0/1424445329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0cf370 msgr2=0x7efe4c0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 --2- 172.21.15.82:0/1424445329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0cf370 0x7efe4c0d4660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe40009870 tx=0x7efe4002ba20 comp rx=0 tx=0).stop 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 -- 172.21.15.82:0/1424445329 shutdown_connections 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 --2- 172.21.15.82:0/1424445329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0cf370 0x7efe4c0d4660 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 --2- 172.21.15.82:0/1424445329 >> v2:172.21.15.164:3300/0 conn(0x7efe4c0d0760 0x7efe4c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 -- 172.21.15.82:0/1424445329 >> 172.21.15.82:0/1424445329 conn(0x7efe4c09d030 msgr2=0x7efe4c0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 -- 172.21.15.82:0/1424445329 shutdown_connections 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.289+0000 7efe5244b640 1 -- 172.21.15.82:0/1424445329 wait complete. 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.290+0000 7efe5244b640 1 Processor -- start 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.290+0000 7efe5244b640 1 -- start start 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe5244b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4c0cf370 0x7efe4c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:31.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe5244b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe4c0d0760 0x7efe4c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe5244b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0d1da0 0x7efe4c154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe5244b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0d4ba0 con 0x7efe4c0cf370 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe5244b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0b7200 con 0x7efe4c0d0760 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe5244b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c1617a0 con 0x7efe4c0d1da0 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4c0cf370 0x7efe4c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4c0cf370 0x7efe4c155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58244/0 (socket says 172.21.15.82:58244) 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe4bfff640 1 -- 172.21.15.82:0/3176166658 learned_addr learned my addr 172.21.15.82:0/3176166658 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe4b7fe640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe4c0d0760 0x7efe4c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.291+0000 7efe509e8640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0d1da0 0x7efe4c154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe4bfff640 1 -- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0d1da0 msgr2=0x7efe4c154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe4bfff640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0d1da0 0x7efe4c154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe4bfff640 1 -- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe4c0d0760 msgr2=0x7efe4c0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe4bfff640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe4c0d0760 0x7efe4c0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe4bfff640 1 -- 172.21.15.82:0/3176166658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0ca660 con 0x7efe4c0cf370 2024-03-14T12:17:31.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe4bfff640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4c0cf370 0x7efe4c155520 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00ebe0 tx=0x7efe3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:31.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe48ff9640 1 -- 172.21.15.82:0/3176166658 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c017070 con 0x7efe4c0cf370 2024-03-14T12:17:31.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe48ff9640 1 -- 172.21.15.82:0/3176166658 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c00cc30 con 0x7efe4c0cf370 2024-03-14T12:17:31.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c0cbd80 con 0x7efe4c0cf370 2024-03-14T12:17:31.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe48ff9640 1 -- 172.21.15.82:0/3176166658 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c013470 con 0x7efe4c0cf370 2024-03-14T12:17:31.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.292+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0c7920 con 0x7efe4c0cf370 2024-03-14T12:17:31.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.293+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c04f050 con 0x7efe4c0cf370 2024-03-14T12:17:31.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.293+0000 7efe48ff9640 1 -- 172.21.15.82:0/3176166658 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7efe3c01a070 con 0x7efe4c0cf370 2024-03-14T12:17:31.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.294+0000 7efe48ff9640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe3806bbd0 0x7efe3806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:31.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.294+0000 7efe4b7fe640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe3806bbd0 0x7efe3806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:31.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.294+0000 7efe48ff9640 1 -- 172.21.15.82:0/3176166658 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efe3c08ad90 con 0x7efe4c0cf370 2024-03-14T12:17:31.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.294+0000 7efe4b7fe640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe3806bbd0 0x7efe3806e090 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe40036000 tx=0x7efe40008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:31.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.298+0000 7efe48ff9640 1 -- 172.21.15.82:0/3176166658 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7efe4c0ca660 con 0x7efe4c0cf370 2024-03-14T12:17:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.504+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe4c0d1860 con 0x7efe4c0cf370 2024-03-14T12:17:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.505+0000 7efe48ff9640 1 -- 172.21.15.82:0/3176166658 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7efe3c059150 con 0x7efe4c0cf370 2024-03-14T12:17:31.505 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:31.506 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:31.506 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe3806bbd0 msgr2=0x7efe3806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe3806bbd0 0x7efe3806e090 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe40036000 tx=0x7efe40008030 comp rx=0 tx=0).stop 2024-03-14T12:17:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4c0cf370 msgr2=0x7efe4c155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4c0cf370 0x7efe4c155520 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00ebe0 tx=0x7efe3c00c520 comp rx=0 tx=0).stop 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 shutdown_connections 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efe3806bbd0 0x7efe3806e090 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe4c0d1da0 0x7efe4c154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe4c0d0760 0x7efe4c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 --2- 172.21.15.82:0/3176166658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe4c0cf370 0x7efe4c155520 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 >> 172.21.15.82:0/3176166658 conn(0x7efe4c09d030 msgr2=0x7efe4c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 shutdown_connections 2024-03-14T12:17:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:31.507+0000 7efe5244b640 1 -- 172.21.15.82:0/3176166658 wait complete. 2024-03-14T12:17:31.518 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:32.519 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:32.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.629+0000 7ff26dca1640 1 Processor -- start 2024-03-14T12:17:32.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff26dca1640 1 -- start start 2024-03-14T12:17:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff26dca1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2680d0760 0x7ff2680d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff26dca1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680b7720 0x7ff2680b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff26dca1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2680d1080 con 0x7ff2680cf370 2024-03-14T12:17:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff26dca1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2680cf8c0 con 0x7ff2680d0760 2024-03-14T12:17:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff26dca1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680cfba0 con 0x7ff2680b7720 2024-03-14T12:17:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff2677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2680d0760 0x7ff2680d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.630+0000 7ff25effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff2680cf370 0x7ff2680d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36954/0 (socket says 172.21.15.82:36954) 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff2677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2680d0760 0x7ff2680d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33034/0 (socket says 172.21.15.82:33034) 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff2677fe640 1 -- 172.21.15.82:0/2099551312 learned_addr learned my addr 172.21.15.82:0/2099551312 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff267fff640 1 --2- 172.21.15.82:0/2099551312 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680b7720 0x7ff2680b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff2677fe640 1 -- 172.21.15.82:0/2099551312 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680b7720 msgr2=0x7ff2680b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff2677fe640 1 --2- 172.21.15.82:0/2099551312 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680b7720 0x7ff2680b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff2677fe640 1 -- 172.21.15.82:0/2099551312 >> v1:172.21.15.82:6789/0 conn(0x7ff2680cf370 legacy=0x7ff2680d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff266ffd640 1 -- 172.21.15.82:0/2099551312 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443836618 0 0) 0x7ff2680d1080 con 0x7ff2680cf370 2024-03-14T12:17:32.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.631+0000 7ff2677fe640 1 -- 172.21.15.82:0/2099551312 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2680b6c40 con 0x7ff2680d0760 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff2677fe640 1 --2- 172.21.15.82:0/2099551312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680d0760 0x7ff2680d0b40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff258009870 tx=0x7ff25802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47eb45402e7314de server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff266ffd640 1 -- 172.21.15.82:0/2099551312 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25802f840 con 0x7ff2680d0760 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff266ffd640 1 -- 172.21.15.82:0/2099551312 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25802f9e0 con 0x7ff2680d0760 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff266ffd640 1 -- 172.21.15.82:0/2099551312 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25802fdb0 con 0x7ff2680d0760 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 -- 172.21.15.82:0/2099551312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680d0760 msgr2=0x7ff2680d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 --2- 172.21.15.82:0/2099551312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680d0760 0x7ff2680d0b40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff258009870 tx=0x7ff25802ba20 comp rx=0 tx=0).stop 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 -- 172.21.15.82:0/2099551312 shutdown_connections 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 --2- 172.21.15.82:0/2099551312 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680b7720 0x7ff2680b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 --2- 172.21.15.82:0/2099551312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680d0760 0x7ff2680d0b40 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 -- 172.21.15.82:0/2099551312 >> 172.21.15.82:0/2099551312 conn(0x7ff26809d030 msgr2=0x7ff2680d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 -- 172.21.15.82:0/2099551312 shutdown_connections 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.632+0000 7ff26dca1640 1 -- 172.21.15.82:0/2099551312 wait complete. 2024-03-14T12:17:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.633+0000 7ff26dca1640 1 Processor -- start 2024-03-14T12:17:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff26dca1640 1 -- start start 2024-03-14T12:17:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff26dca1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2680b7720 0x7ff2680cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff26dca1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680cf370 0x7ff268152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff26dca1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680d0760 0x7ff268160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff26dca1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2681617a0 con 0x7ff2680cf370 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff26dca1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680b7200 con 0x7ff2680b7720 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff26dca1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff268161920 con 0x7ff2680d0760 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.634+0000 7ff25effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680cf370 0x7ff268152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680cf370 0x7ff268152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33044/0 (socket says 172.21.15.82:33044) 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 -- 172.21.15.82:0/27297762 learned_addr learned my addr 172.21.15.82:0/27297762 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff2677fe640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2680b7720 0x7ff2680cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff267fff640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680d0760 0x7ff268160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 -- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680d0760 msgr2=0x7ff268160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680d0760 0x7ff268160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 -- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2680b7720 msgr2=0x7ff2680cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2680b7720 0x7ff2680cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 -- 172.21.15.82:0/27297762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2680ca5b0 con 0x7ff2680cf370 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff267fff640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680d0760 0x7ff268160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff25effd640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680cf370 0x7ff268152010 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c4d0 tx=0x7ff25000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff264ff9640 1 -- 172.21.15.82:0/27297762 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250018070 con 0x7ff2680cf370 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff264ff9640 1 -- 172.21.15.82:0/27297762 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff250007500 con 0x7ff2680cf370 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2680cafc0 con 0x7ff2680cf370 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff264ff9640 1 -- 172.21.15.82:0/27297762 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250004a20 con 0x7ff2680cf370 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.635+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff268152550 con 0x7ff2680cf370 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.636+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26804f050 con 0x7ff2680cf370 2024-03-14T12:17:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.637+0000 7ff264ff9640 1 -- 172.21.15.82:0/27297762 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff25001b070 con 0x7ff2680cf370 2024-03-14T12:17:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.637+0000 7ff264ff9640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff24806bbd0 0x7ff24806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.637+0000 7ff264ff9640 1 -- 172.21.15.82:0/27297762 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff250089e30 con 0x7ff2680cf370 2024-03-14T12:17:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.637+0000 7ff2677fe640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff24806bbd0 0x7ff24806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.637+0000 7ff2677fe640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff24806bbd0 0x7ff24806e090 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff258036000 tx=0x7ff258008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:32.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.641+0000 7ff264ff9640 1 -- 172.21.15.82:0/27297762 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff250058890 con 0x7ff2680cf370 2024-03-14T12:17:32.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.847+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2680d0540 con 0x7ff2680cf370 2024-03-14T12:17:32.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.847+0000 7ff264ff9640 1 -- 172.21.15.82:0/27297762 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff250058190 con 0x7ff2680cf370 2024-03-14T12:17:32.848 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:32.848 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:32.848 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff24806bbd0 msgr2=0x7ff24806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff24806bbd0 0x7ff24806e090 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff258036000 tx=0x7ff258008030 comp rx=0 tx=0).stop 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680cf370 msgr2=0x7ff268152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680cf370 0x7ff268152010 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c4d0 tx=0x7ff25000c990 comp rx=0 tx=0).stop 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 shutdown_connections 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff24806bbd0 0x7ff24806e090 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff2680d0760 0x7ff268160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2680cf370 0x7ff268152010 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 --2- 172.21.15.82:0/27297762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2680b7720 0x7ff2680cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:32.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 >> 172.21.15.82:0/27297762 conn(0x7ff26809d030 msgr2=0x7ff2680b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:32.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 shutdown_connections 2024-03-14T12:17:32.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:32.849+0000 7ff26dca1640 1 -- 172.21.15.82:0/27297762 wait complete. 2024-03-14T12:17:32.860 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:33.861 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:33.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.972+0000 7efc68ad7640 1 Processor -- start 2024-03-14T12:17:33.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.972+0000 7efc68ad7640 1 -- start start 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc68ad7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efc640b3fe0 0x7efc640b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc68ad7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efc64071230 0x7efc64168920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc68ad7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6415f2a0 con 0x7efc64072650 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc68ad7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc640b4900 con 0x7efc640b3fe0 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc68ad7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6406f120 con 0x7efc64071230 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc6259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efc640b3fe0 0x7efc640b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc61d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7efc64072650 0x7efc64070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36956/0 (socket says 172.21.15.82:36956) 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc61d9b640 1 -- 172.21.15.82:0/1640713139 learned_addr learned my addr 172.21.15.82:0/1640713139 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc62d9d640 1 --2- 172.21.15.82:0/1640713139 >> v2:172.21.15.164:3300/0 conn(0x7efc64071230 0x7efc64168920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:33.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc6259c640 1 -- 172.21.15.82:0/1640713139 >> v2:172.21.15.164:3300/0 conn(0x7efc64071230 msgr2=0x7efc64168920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:33.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc6259c640 1 --2- 172.21.15.82:0/1640713139 >> v2:172.21.15.164:3300/0 conn(0x7efc64071230 0x7efc64168920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:33.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc6259c640 1 -- 172.21.15.82:0/1640713139 >> v1:172.21.15.82:6789/0 conn(0x7efc64072650 legacy=0x7efc64070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:33.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.973+0000 7efc6159a640 1 -- 172.21.15.82:0/1640713139 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760752154 0 0) 0x7efc6415f2a0 con 0x7efc64072650 2024-03-14T12:17:33.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.974+0000 7efc6259c640 1 -- 172.21.15.82:0/1640713139 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6415e690 con 0x7efc640b3fe0 2024-03-14T12:17:33.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.974+0000 7efc6259c640 1 --2- 172.21.15.82:0/1640713139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc640b3fe0 0x7efc640b43c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7efc64161820 tx=0x7efc5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee80a17ac3e9ae3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:33.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.974+0000 7efc6159a640 1 -- 172.21.15.82:0/1640713139 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c035020 con 0x7efc640b3fe0 2024-03-14T12:17:33.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.974+0000 7efc6159a640 1 -- 172.21.15.82:0/1640713139 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c031830 con 0x7efc640b3fe0 2024-03-14T12:17:33.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.974+0000 7efc6159a640 1 -- 172.21.15.82:0/1640713139 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c031c00 con 0x7efc640b3fe0 2024-03-14T12:17:33.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 -- 172.21.15.82:0/1640713139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc640b3fe0 msgr2=0x7efc640b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:33.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 --2- 172.21.15.82:0/1640713139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc640b3fe0 0x7efc640b43c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7efc64161820 tx=0x7efc5c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:17:33.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 -- 172.21.15.82:0/1640713139 shutdown_connections 2024-03-14T12:17:33.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 --2- 172.21.15.82:0/1640713139 >> v2:172.21.15.164:3300/0 conn(0x7efc64071230 0x7efc64168920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:33.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 --2- 172.21.15.82:0/1640713139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc640b3fe0 0x7efc640b43c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:33.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 -- 172.21.15.82:0/1640713139 >> 172.21.15.82:0/1640713139 conn(0x7efc640b1bd0 msgr2=0x7efc640b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:33.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 -- 172.21.15.82:0/1640713139 shutdown_connections 2024-03-14T12:17:33.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.975+0000 7efc68ad7640 1 -- 172.21.15.82:0/1640713139 wait complete. 2024-03-14T12:17:33.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.976+0000 7efc68ad7640 1 Processor -- start 2024-03-14T12:17:33.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.976+0000 7efc68ad7640 1 -- start start 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc68ad7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc64071230 0x7efc641f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc68ad7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc64072650 0x7efc6415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc68ad7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc640b3fe0 0x7efc64142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc68ad7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6415ef30 con 0x7efc64071230 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc68ad7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6415ec50 con 0x7efc640b3fe0 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc68ad7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc64141400 con 0x7efc64072650 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc6259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc64071230 0x7efc641f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc6259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc64071230 0x7efc641f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33066/0 (socket says 172.21.15.82:33066) 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc6259c640 1 -- 172.21.15.82:0/3772921260 learned_addr learned my addr 172.21.15.82:0/3772921260 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc62d9d640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc640b3fe0 0x7efc64142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.977+0000 7efc61d9b640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc64072650 0x7efc6415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc6259c640 1 -- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc64072650 msgr2=0x7efc6415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:33.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc6259c640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc64072650 0x7efc6415a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc6259c640 1 -- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc640b3fe0 msgr2=0x7efc64142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc6259c640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc640b3fe0 0x7efc64142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc6259c640 1 -- 172.21.15.82:0/3772921260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6415add0 con 0x7efc64071230 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc62d9d640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc640b3fe0 0x7efc64142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc6259c640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc64071230 0x7efc641f2da0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009890 tx=0x7efc5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc52ffd640 1 -- 172.21.15.82:0/3772921260 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c041070 con 0x7efc64071230 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc52ffd640 1 -- 172.21.15.82:0/3772921260 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c03aa30 con 0x7efc64071230 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6415b0b0 con 0x7efc64071230 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc52ffd640 1 -- 172.21.15.82:0/3772921260 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c035040 con 0x7efc64071230 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.978+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6415bda0 con 0x7efc64071230 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.979+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6404f050 con 0x7efc64071230 2024-03-14T12:17:33.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.979+0000 7efc52ffd640 1 -- 172.21.15.82:0/3772921260 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7efc5c038020 con 0x7efc64071230 2024-03-14T12:17:33.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.980+0000 7efc52ffd640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc4406bbd0 0x7efc4406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:33.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.980+0000 7efc52ffd640 1 -- 172.21.15.82:0/3772921260 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7efc5c0ac4d0 con 0x7efc64071230 2024-03-14T12:17:33.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.980+0000 7efc61d9b640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc4406bbd0 0x7efc4406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:33.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.980+0000 7efc61d9b640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc4406bbd0 0x7efc4406e090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efc640ae300 tx=0x7efc58007b50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:33.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:33.984+0000 7efc52ffd640 1 -- 172.21.15.82:0/3772921260 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7efc5c07af30 con 0x7efc64071230 2024-03-14T12:17:34.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.190+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc6406bd20 con 0x7efc64071230 2024-03-14T12:17:34.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.190+0000 7efc52ffd640 1 -- 172.21.15.82:0/3772921260 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7efc5c07a830 con 0x7efc64071230 2024-03-14T12:17:34.191 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:34.191 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:34.191 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc4406bbd0 msgr2=0x7efc4406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc4406bbd0 0x7efc4406e090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efc640ae300 tx=0x7efc58007b50 comp rx=0 tx=0).stop 2024-03-14T12:17:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc64071230 msgr2=0x7efc641f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc64071230 0x7efc641f2da0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009890 tx=0x7efc5c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 shutdown_connections 2024-03-14T12:17:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc640b3fe0 0x7efc64142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc4406bbd0 0x7efc4406e090 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc64072650 0x7efc6415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 --2- 172.21.15.82:0/3772921260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc64071230 0x7efc641f2da0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 >> 172.21.15.82:0/3772921260 conn(0x7efc640b1bd0 msgr2=0x7efc640b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 shutdown_connections 2024-03-14T12:17:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:34.192+0000 7efc68ad7640 1 -- 172.21.15.82:0/3772921260 wait complete. 2024-03-14T12:17:34.203 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:35.205 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:35.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.313+0000 7f5157861640 1 Processor -- start 2024-03-14T12:17:35.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.313+0000 7f5157861640 1 -- start start 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.313+0000 7f5157861640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51500b4190 0x7f515015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.313+0000 7f5157861640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51500cd0d0 0x7f515015f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.313+0000 7f5157861640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51500cd540 con 0x7f515015b9e0 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.313+0000 7f5157861640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51500cf730 con 0x7f51500b4190 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.313+0000 7f5157861640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51500cd810 con 0x7f51500cd0d0 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5155dfe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51500cd0d0 0x7f515015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5154dfc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51500b4190 0x7f515015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5154dfc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51500b4190 0x7f515015bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:57600/0 (socket says 172.21.15.82:57600) 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5155dfe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51500cd0d0 0x7f515015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:33698/0 (socket says 172.21.15.82:33698) 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f51555fd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f515015b9e0 0x7f51500d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:49782/0 (socket says 172.21.15.82:49782) 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5154dfc640 1 -- 172.21.15.82:0/1987846382 learned_addr learned my addr 172.21.15.82:0/1987846382 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5154dfc640 1 -- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51500cd0d0 msgr2=0x7f515015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5154dfc640 1 --2- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51500cd0d0 0x7f515015f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5154dfc640 1 -- 172.21.15.82:0/1987846382 >> v1:172.21.15.164:6789/0 conn(0x7f515015b9e0 legacy=0x7f51500d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:35.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5154dfc640 1 -- 172.21.15.82:0/1987846382 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51500cf980 con 0x7f51500b4190 2024-03-14T12:17:35.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5147fff640 1 -- 172.21.15.82:0/1987846382 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388417225 0 0) 0x7f51500cd540 con 0x7f515015b9e0 2024-03-14T12:17:35.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.314+0000 7f5155dfe640 1 --2- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51500cd0d0 0x7f515015f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:35.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.315+0000 7f5154dfc640 1 --2- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500b4190 0x7f515015bdc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5140009870 tx=0x7f514002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a882aadd933bd2a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:35.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.315+0000 7f5147fff640 1 -- 172.21.15.82:0/1987846382 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140035020 con 0x7f51500b4190 2024-03-14T12:17:35.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.315+0000 7f5147fff640 1 -- 172.21.15.82:0/1987846382 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5140031640 con 0x7f51500b4190 2024-03-14T12:17:35.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.315+0000 7f5147fff640 1 -- 172.21.15.82:0/1987846382 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514002d040 con 0x7f51500b4190 2024-03-14T12:17:35.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 -- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500b4190 msgr2=0x7f515015bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:35.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 --2- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500b4190 0x7f515015bdc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5140009870 tx=0x7f514002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 -- 172.21.15.82:0/1987846382 shutdown_connections 2024-03-14T12:17:35.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 --2- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51500cd0d0 0x7f515015f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 --2- 172.21.15.82:0/1987846382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500b4190 0x7f515015bdc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 -- 172.21.15.82:0/1987846382 >> 172.21.15.82:0/1987846382 conn(0x7f51500b3670 msgr2=0x7f51500b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:35.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 -- 172.21.15.82:0/1987846382 shutdown_connections 2024-03-14T12:17:35.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.316+0000 7f5157861640 1 -- 172.21.15.82:0/1987846382 wait complete. 2024-03-14T12:17:35.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.317+0000 7f5157861640 1 Processor -- start 2024-03-14T12:17:35.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.317+0000 7f5157861640 1 -- start start 2024-03-14T12:17:35.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5157861640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51500b4190 0x7f515006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:35.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5157861640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500cd0d0 0x7f515006be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:35.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5157861640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f515015b9e0 0x7f51501f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:35.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5157861640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515015fe80 con 0x7f51500b4190 2024-03-14T12:17:35.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5157861640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51500cff40 con 0x7f51500cd0d0 2024-03-14T12:17:35.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5157861640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5150160000 con 0x7f515015b9e0 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51500b4190 0x7f515006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51500b4190 0x7f515006d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33072/0 (socket says 172.21.15.82:33072) 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 -- 172.21.15.82:0/3604605287 learned_addr learned my addr 172.21.15.82:0/3604605287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5154dfc640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500cd0d0 0x7f515006be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5155dfe640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f515015b9e0 0x7f51501f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 -- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f515015b9e0 msgr2=0x7f51501f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f515015b9e0 0x7f51501f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 -- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500cd0d0 msgr2=0x7f515006be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500cd0d0 0x7f515006be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 -- 172.21.15.82:0/3604605287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515006c520 con 0x7f51500b4190 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5155dfe640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f515015b9e0 0x7f51501f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f51555fd640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51500b4190 0x7f515006d5c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c890 tx=0x7f514c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5145ffb640 1 -- 172.21.15.82:0/3604605287 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c018070 con 0x7f51500b4190 2024-03-14T12:17:35.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.318+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515006c800 con 0x7f51500b4190 2024-03-14T12:17:35.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.319+0000 7f5145ffb640 1 -- 172.21.15.82:0/3604605287 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c00a640 con 0x7f51500b4190 2024-03-14T12:17:35.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.319+0000 7f5145ffb640 1 -- 172.21.15.82:0/3604605287 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00aa30 con 0x7f51500b4190 2024-03-14T12:17:35.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.319+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515006cdc0 con 0x7f51500b4190 2024-03-14T12:17:35.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.320+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5150072a90 con 0x7f51500b4190 2024-03-14T12:17:35.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.320+0000 7f5145ffb640 1 -- 172.21.15.82:0/3604605287 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f514c00abd0 con 0x7f51500b4190 2024-03-14T12:17:35.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.320+0000 7f5145ffb640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f513406bc00 0x7f513406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:35.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.321+0000 7f5145ffb640 1 -- 172.21.15.82:0/3604605287 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f514c08a0a0 con 0x7f51500b4190 2024-03-14T12:17:35.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.321+0000 7f5154dfc640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f513406bc00 0x7f513406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:35.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.321+0000 7f5154dfc640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f513406bc00 0x7f513406e0c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5140036000 tx=0x7f514003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:35.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.325+0000 7f5145ffb640 1 -- 172.21.15.82:0/3604605287 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5150072a90 con 0x7f51500b4190 2024-03-14T12:17:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.535+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51500684d0 con 0x7f51500b4190 2024-03-14T12:17:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.536+0000 7f5145ffb640 1 -- 172.21.15.82:0/3604605287 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f51500684d0 con 0x7f51500b4190 2024-03-14T12:17:35.536 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:35.536 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:35.537 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f513406bc00 msgr2=0x7f513406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f513406bc00 0x7f513406e0c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5140036000 tx=0x7f514003c040 comp rx=0 tx=0).stop 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51500b4190 msgr2=0x7f515006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51500b4190 0x7f515006d5c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c890 tx=0x7f514c00cd50 comp rx=0 tx=0).stop 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 shutdown_connections 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f513406bc00 0x7f513406e0c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f515015b9e0 0x7f51501f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51500cd0d0 0x7f515006be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 --2- 172.21.15.82:0/3604605287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51500b4190 0x7f515006d5c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 >> 172.21.15.82:0/3604605287 conn(0x7f51500b3670 msgr2=0x7f515006ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 shutdown_connections 2024-03-14T12:17:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:35.538+0000 7f5157861640 1 -- 172.21.15.82:0/3604605287 wait complete. 2024-03-14T12:17:35.549 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:36.550 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:36.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.661+0000 7f1e21e2b640 1 Processor -- start 2024-03-14T12:17:36.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.661+0000 7f1e21e2b640 1 -- start start 2024-03-14T12:17:36.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.662+0000 7f1e21e2b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e1c06b420 0x7f1e1c15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:36.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.662+0000 7f1e21e2b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0d1590 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.662+0000 7f1e21e2b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c161b70 con 0x7f1e1c15df30 2024-03-14T12:17:36.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.662+0000 7f1e21e2b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c161e50 con 0x7f1e1c06b420 2024-03-14T12:17:36.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.662+0000 7f1e1bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e1c06b420 0x7f1e1c15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:36.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.662+0000 7f1e1affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1e1c06f7f0 0x7f1e1c15e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36962/0 (socket says 172.21.15.82:36962) 2024-03-14T12:17:36.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.662+0000 7f1e1affd640 1 -- 172.21.15.82:0/2290641682 learned_addr learned my addr 172.21.15.82:0/2290641682 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:36.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.663+0000 7f1e1bfff640 1 -- 172.21.15.82:0/2290641682 >> v1:172.21.15.164:6789/0 conn(0x7f1e1c15df30 legacy=0x7f1e1c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:36.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.663+0000 7f1e1bfff640 1 -- 172.21.15.82:0/2290641682 >> v1:172.21.15.82:6789/0 conn(0x7f1e1c06f7f0 legacy=0x7f1e1c15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:36.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.663+0000 7f1e1bfff640 1 -- 172.21.15.82:0/2290641682 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c162130 con 0x7f1e1c06b420 2024-03-14T12:17:36.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.663+0000 7f1e1a7fc640 1 -- 172.21.15.82:0/2290641682 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069799769 0 0) 0x7f1e1c0d1590 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.663+0000 7f1e1bfff640 1 --2- 172.21.15.82:0/2290641682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06b420 0x7f1e1c15ea20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009d90 tx=0x7f1e1402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150e0b1745bc928b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:36.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e1a7fc640 1 -- 172.21.15.82:0/2290641682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14039070 con 0x7f1e1c06b420 2024-03-14T12:17:36.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e1a7fc640 1 -- 172.21.15.82:0/2290641682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1402cc60 con 0x7f1e1c06b420 2024-03-14T12:17:36.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e1a7fc640 1 -- 172.21.15.82:0/2290641682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14034660 con 0x7f1e1c06b420 2024-03-14T12:17:36.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e21e2b640 1 -- 172.21.15.82:0/2290641682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06b420 msgr2=0x7f1e1c15ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:36.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/2290641682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06b420 0x7f1e1c15ea20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009d90 tx=0x7f1e1402beb0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e21e2b640 1 -- 172.21.15.82:0/2290641682 shutdown_connections 2024-03-14T12:17:36.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/2290641682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06b420 0x7f1e1c15ea20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e21e2b640 1 -- 172.21.15.82:0/2290641682 >> 172.21.15.82:0/2290641682 conn(0x7f1e1c0d09d0 msgr2=0x7f1e1c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:36.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e21e2b640 1 -- 172.21.15.82:0/2290641682 shutdown_connections 2024-03-14T12:17:36.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.664+0000 7f1e21e2b640 1 -- 172.21.15.82:0/2290641682 wait complete. 2024-03-14T12:17:36.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.665+0000 7f1e21e2b640 1 Processor -- start 2024-03-14T12:17:36.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e21e2b640 1 -- start start 2024-03-14T12:17:36.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e21e2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1c06b420 0x7f1e1c14c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:36.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e21e2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06f7f0 0x7f1e1c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:36.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e21e2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1c15df30 0x7f1e1c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:36.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e21e2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c162650 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e21e2b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0b3c50 con 0x7f1e1c15df30 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e21e2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1627d0 con 0x7f1e1c06b420 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e1affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06f7f0 0x7f1e1c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e1affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06f7f0 0x7f1e1c06e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33096/0 (socket says 172.21.15.82:33096) 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.666+0000 7f1e1affd640 1 -- 172.21.15.82:0/195279216 learned_addr learned my addr 172.21.15.82:0/195279216 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1affd640 1 -- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1c06b420 msgr2=0x7f1e1c14c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1b7fe640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1c06b420 0x7f1e1c14c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1bfff640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1c15df30 0x7f1e1c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1affd640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1c06b420 0x7f1e1c14c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1affd640 1 -- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1c15df30 msgr2=0x7f1e1c15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1affd640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1c15df30 0x7f1e1c15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1affd640 1 -- 172.21.15.82:0/195279216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c06d8e0 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e1affd640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06f7f0 0x7f1e1c06e980 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c500 tx=0x7f1e1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:36.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e03fff640 1 -- 172.21.15.82:0/195279216 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10018070 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e03fff640 1 -- 172.21.15.82:0/195279216 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10007580 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e03fff640 1 -- 172.21.15.82:0/195279216 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10004a20 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c06dbc0 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.667+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c06e180 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.668+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c04f050 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.669+0000 7f1e03fff640 1 -- 172.21.15.82:0/195279216 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1e1001b070 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.669+0000 7f1e03fff640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0806bbd0 0x7f1e0806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:36.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.669+0000 7f1e03fff640 1 -- 172.21.15.82:0/195279216 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1e10089db0 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.669+0000 7f1e1b7fe640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0806bbd0 0x7f1e0806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:36.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.670+0000 7f1e1b7fe640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0806bbd0 0x7f1e0806e090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006fd0 tx=0x7f1e0c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:36.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.673+0000 7f1e03fff640 1 -- 172.21.15.82:0/195279216 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f1e100588a0 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.881+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e1c06b050 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.881+0000 7f1e03fff640 1 -- 172.21.15.82:0/195279216 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f1e100581a0 con 0x7f1e1c06f7f0 2024-03-14T12:17:36.882 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:36.882 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0806bbd0 msgr2=0x7f1e0806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0806bbd0 0x7f1e0806e090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006fd0 tx=0x7f1e0c008040 comp rx=0 tx=0).stop 2024-03-14T12:17:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06f7f0 msgr2=0x7f1e1c06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06f7f0 0x7f1e1c06e980 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c500 tx=0x7f1e1000c9c0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 shutdown_connections 2024-03-14T12:17:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1c15df30 0x7f1e1c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1c06f7f0 0x7f1e1c06e980 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f1e0806bbd0 0x7f1e0806e090 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 --2- 172.21.15.82:0/195279216 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1c06b420 0x7f1e1c14c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 >> 172.21.15.82:0/195279216 conn(0x7f1e1c0d09d0 msgr2=0x7f1e1c156290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 shutdown_connections 2024-03-14T12:17:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:36.884+0000 7f1e21e2b640 1 -- 172.21.15.82:0/195279216 wait complete. 2024-03-14T12:17:36.895 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:37.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:38.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.007+0000 7f0772e12640 1 Processor -- start 2024-03-14T12:17:38.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.007+0000 7f0772e12640 1 -- start start 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f0772e12640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c15e670 0x7f076c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f0772e12640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f076c143f60 0x7f076c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f0772e12640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076c15ebe0 con 0x7f076c071740 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f0772e12640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c15eec0 con 0x7f076c143f60 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f0772e12640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c15f1a0 con 0x7f076c15e670 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f0770bae640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f076c071740 0x7f076c071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:49804/0 (socket says 172.21.15.82:49804) 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f0770bae640 1 -- 172.21.15.82:0/1561280209 learned_addr learned my addr 172.21.15.82:0/1561280209 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f076bfff640 1 --2- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c15e670 0x7f076c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:38.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f07713af640 1 --2- 172.21.15.82:0/1561280209 >> v2:172.21.15.164:3300/0 conn(0x7f076c143f60 0x7f076c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:38.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f076b7fe640 1 -- 172.21.15.82:0/1561280209 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181851812 0 0) 0x7f076c15ebe0 con 0x7f076c071740 2024-03-14T12:17:38.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f076b7fe640 1 -- 172.21.15.82:0/1561280209 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f075c003520 con 0x7f076c071740 2024-03-14T12:17:38.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f07713af640 1 -- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c15e670 msgr2=0x7f076c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:38.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f07713af640 1 --2- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c15e670 0x7f076c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f07713af640 1 -- 172.21.15.82:0/1561280209 >> v1:172.21.15.164:6789/0 conn(0x7f076c071740 legacy=0x7f076c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:38.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f07713af640 1 -- 172.21.15.82:0/1561280209 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c15f480 con 0x7f076c143f60 2024-03-14T12:17:38.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.008+0000 7f076bfff640 1 --2- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c15e670 0x7f076c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:38.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.009+0000 7f07713af640 1 --2- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 0x7f076c15b610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0764009d90 tx=0x7f076402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881c5e4f5c7806fe server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.009+0000 7f076b7fe640 1 -- 172.21.15.82:0/1561280209 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764039070 con 0x7f076c143f60 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.009+0000 7f076b7fe640 1 -- 172.21.15.82:0/1561280209 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076402cc50 con 0x7f076c143f60 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.009+0000 7f076b7fe640 1 -- 172.21.15.82:0/1561280209 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764034440 con 0x7f076c143f60 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 -- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 msgr2=0x7f076c15b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 --2- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 0x7f076c15b610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0764009d90 tx=0x7f076402bea0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 -- 172.21.15.82:0/1561280209 shutdown_connections 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 --2- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 0x7f076c15b610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 --2- 172.21.15.82:0/1561280209 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c15e670 0x7f076c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 -- 172.21.15.82:0/1561280209 >> 172.21.15.82:0/1561280209 conn(0x7f076c0b1c70 msgr2=0x7f076c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 -- 172.21.15.82:0/1561280209 shutdown_connections 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.010+0000 7f0772e12640 1 -- 172.21.15.82:0/1561280209 wait complete. 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.011+0000 7f0772e12640 1 Processor -- start 2024-03-14T12:17:38.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.011+0000 7f0772e12640 1 -- start start 2024-03-14T12:17:38.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f0772e12640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c071740 0x7f076c1f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:38.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f0772e12640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 0x7f076c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:38.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f0772e12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076c15e670 0x7f076c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:38.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f0772e12640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c144670 con 0x7f076c15e670 2024-03-14T12:17:38.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f0772e12640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c15b080 con 0x7f076c143f60 2024-03-14T12:17:38.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f0772e12640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c1447f0 con 0x7f076c071740 2024-03-14T12:17:38.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f07713af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076c15e670 0x7f076c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f07713af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076c15e670 0x7f076c14f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33100/0 (socket says 172.21.15.82:33100) 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f07713af640 1 -- 172.21.15.82:0/3966323215 learned_addr learned my addr 172.21.15.82:0/3966323215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f0770bae640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c071740 0x7f076c1f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.012+0000 7f076bfff640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 0x7f076c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07713af640 1 -- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c071740 msgr2=0x7f076c1f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07713af640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c071740 0x7f076c1f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07713af640 1 -- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 msgr2=0x7f076c1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07713af640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 0x7f076c1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07713af640 1 -- 172.21.15.82:0/3966323215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c13cc10 con 0x7f076c15e670 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07713af640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076c15e670 0x7f076c14f270 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0764009d90 tx=0x7f0764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07697fa640 1 -- 172.21.15.82:0/3966323215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764039050 con 0x7f076c15e670 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07697fa640 1 -- 172.21.15.82:0/3966323215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764045070 con 0x7f076c15e670 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c13cef0 con 0x7f076c15e670 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f07697fa640 1 -- 172.21.15.82:0/3966323215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764009450 con 0x7f076c15e670 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.013+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c13d4b0 con 0x7f076c15e670 2024-03-14T12:17:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.014+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c04f050 con 0x7f076c15e670 2024-03-14T12:17:38.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.014+0000 7f07697fa640 1 -- 172.21.15.82:0/3966323215 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f076403e610 con 0x7f076c15e670 2024-03-14T12:17:38.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.015+0000 7f07697fa640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f075006bbd0 0x7f075006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:38.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.015+0000 7f0770bae640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f075006bbd0 0x7f075006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:38.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.015+0000 7f07697fa640 1 -- 172.21.15.82:0/3966323215 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f07640ad320 con 0x7f076c15e670 2024-03-14T12:17:38.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.015+0000 7f0770bae640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f075006bbd0 0x7f075006e090 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0760005880 tx=0x7f0760009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:38.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.019+0000 7f07697fa640 1 -- 172.21.15.82:0/3966323215 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f076407b7c0 con 0x7f076c15e670 2024-03-14T12:17:38.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.225+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f076c143390 con 0x7f076c15e670 2024-03-14T12:17:38.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.225+0000 7f07697fa640 1 -- 172.21.15.82:0/3966323215 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f076407f860 con 0x7f076c15e670 2024-03-14T12:17:38.225 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:38.225 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:38.225 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:38.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.227+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f075006bbd0 msgr2=0x7f075006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:38.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.227+0000 7f0772e12640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f075006bbd0 0x7f075006e090 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0760005880 tx=0x7f0760009040 comp rx=0 tx=0).stop 2024-03-14T12:17:38.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.227+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076c15e670 msgr2=0x7f076c14f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:38.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.227+0000 7f0772e12640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076c15e670 0x7f076c14f270 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0764009d90 tx=0x7f0764004a40 comp rx=0 tx=0).stop 2024-03-14T12:17:38.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 shutdown_connections 2024-03-14T12:17:38.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076c15e670 0x7f076c14f270 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f076c143f60 0x7f076c1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f075006bbd0 0x7f075006e090 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 --2- 172.21.15.82:0/3966323215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f076c071740 0x7f076c1f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:38.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 >> 172.21.15.82:0/3966323215 conn(0x7f076c0b1c70 msgr2=0x7f076c0b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:38.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 shutdown_connections 2024-03-14T12:17:38.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:38.228+0000 7f0772e12640 1 -- 172.21.15.82:0/3966323215 wait complete. 2024-03-14T12:17:38.238 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:39.239 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:39.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.352+0000 7f507b2ab640 1 Processor -- start 2024-03-14T12:17:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.352+0000 7f507b2ab640 1 -- start start 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.353+0000 7f507b2ab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f507415e670 0x7f50740b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.353+0000 7f507b2ab640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507415ebe0 con 0x7f5074143f60 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.353+0000 7f507b2ab640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507415eec0 con 0x7f5074071740 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.353+0000 7f507b2ab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507415f1a0 con 0x7f507415e670 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.353+0000 7f5079848640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5074143f60 0x7f507415b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36964/0 (socket says 172.21.15.82:36964) 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.353+0000 7f5079848640 1 -- 172.21.15.82:0/419631104 learned_addr learned my addr 172.21.15.82:0/419631104 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.353+0000 7f5078846640 1 --2- 172.21.15.82:0/419631104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f507415e670 0x7f50740b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.354+0000 7f506bfff640 1 -- 172.21.15.82:0/419631104 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512514115 0 0) 0x7f507415ebe0 con 0x7f5074143f60 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.354+0000 7f506bfff640 1 -- 172.21.15.82:0/419631104 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5060003520 con 0x7f5074143f60 2024-03-14T12:17:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.354+0000 7f5078846640 1 -- 172.21.15.82:0/419631104 >> v1:172.21.15.164:6789/0 conn(0x7f5074071740 legacy=0x7f5074071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.354+0000 7f5078846640 1 -- 172.21.15.82:0/419631104 >> v1:172.21.15.82:6789/0 conn(0x7f5074143f60 legacy=0x7f507415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.354+0000 7f506bfff640 1 -- 172.21.15.82:0/419631104 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10954519 0 0) 0x7f507415eec0 con 0x7f5074071740 2024-03-14T12:17:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.354+0000 7f5078846640 1 -- 172.21.15.82:0/419631104 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507415f480 con 0x7f507415e670 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f5078846640 1 --2- 172.21.15.82:0/419631104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f507415e670 0x7f50740b2940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f50640097d0 tx=0x7f506402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726256d462dd9480 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f506bfff640 1 -- 172.21.15.82:0/419631104 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064037070 con 0x7f507415e670 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f506bfff640 1 -- 172.21.15.82:0/419631104 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064030710 con 0x7f507415e670 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f506bfff640 1 -- 172.21.15.82:0/419631104 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064030ac0 con 0x7f507415e670 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f507b2ab640 1 -- 172.21.15.82:0/419631104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f507415e670 msgr2=0x7f50740b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f507b2ab640 1 --2- 172.21.15.82:0/419631104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f507415e670 0x7f50740b2940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f50640097d0 tx=0x7f506402b800 comp rx=0 tx=0).stop 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f507b2ab640 1 -- 172.21.15.82:0/419631104 shutdown_connections 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f507b2ab640 1 --2- 172.21.15.82:0/419631104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f507415e670 0x7f50740b2940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f507b2ab640 1 -- 172.21.15.82:0/419631104 >> 172.21.15.82:0/419631104 conn(0x7f50740b1c70 msgr2=0x7f50740cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:39.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f507b2ab640 1 -- 172.21.15.82:0/419631104 shutdown_connections 2024-03-14T12:17:39.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.355+0000 7f507b2ab640 1 -- 172.21.15.82:0/419631104 wait complete. 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 Processor -- start 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 -- start start 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5074071740 0x7f5074137b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5074143f60 0x7f50740ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507415e670 0x7f507413b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074144670 con 0x7f507415e670 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507415b080 con 0x7f5074071740 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f507b2ab640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50741447f0 con 0x7f5074143f60 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f5079848640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507415e670 0x7f507413b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:39.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f5079848640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507415e670 0x7f507413b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33110/0 (socket says 172.21.15.82:33110) 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.357+0000 7f5079848640 1 -- 172.21.15.82:0/462554108 learned_addr learned my addr 172.21.15.82:0/462554108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5078846640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5074143f60 0x7f50740ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079047640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5074071740 0x7f5074137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079848640 1 -- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5074143f60 msgr2=0x7f50740ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079848640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5074143f60 0x7f50740ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079848640 1 -- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5074071740 msgr2=0x7f5074137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079848640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5074071740 0x7f5074137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079848640 1 -- 172.21.15.82:0/462554108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507415dab0 con 0x7f507415e670 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5078846640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5074143f60 0x7f50740ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079047640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5074071740 0x7f5074137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5079848640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507415e670 0x7f507413b750 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c780 tx=0x7f506c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5069ffb640 1 -- 172.21.15.82:0/462554108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c017070 con 0x7f507415e670 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5069ffb640 1 -- 172.21.15.82:0/462554108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c005a90 con 0x7f507415e670 2024-03-14T12:17:39.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f5069ffb640 1 -- 172.21.15.82:0/462554108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c007070 con 0x7f507415e670 2024-03-14T12:17:39.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5074152ba0 con 0x7f507415e670 2024-03-14T12:17:39.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.358+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5074158710 con 0x7f507415e670 2024-03-14T12:17:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.359+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5074068740 con 0x7f507415e670 2024-03-14T12:17:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.360+0000 7f5069ffb640 1 -- 172.21.15.82:0/462554108 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f506c01a070 con 0x7f507415e670 2024-03-14T12:17:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.360+0000 7f5069ffb640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f505806bc00 0x7f505806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.360+0000 7f5069ffb640 1 -- 172.21.15.82:0/462554108 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f506c08a050 con 0x7f507415e670 2024-03-14T12:17:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.360+0000 7f5079047640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f505806bc00 0x7f505806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.360+0000 7f5079047640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f505806bc00 0x7f505806e0c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5070007fd0 tx=0x7f5070009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.364+0000 7f5069ffb640 1 -- 172.21.15.82:0/462554108 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5074068740 con 0x7f507415e670 2024-03-14T12:17:39.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.572+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f507404f050 con 0x7f507415e670 2024-03-14T12:17:39.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.572+0000 7f5069ffb640 1 -- 172.21.15.82:0/462554108 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f506c058ab0 con 0x7f507415e670 2024-03-14T12:17:39.572 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:39.572 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:39.572 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f505806bc00 msgr2=0x7f505806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f505806bc00 0x7f505806e0c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5070007fd0 tx=0x7f5070009040 comp rx=0 tx=0).stop 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507415e670 msgr2=0x7f507413b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507415e670 0x7f507413b750 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c780 tx=0x7f506c00cc40 comp rx=0 tx=0).stop 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 shutdown_connections 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507415e670 0x7f507413b750 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f505806bc00 0x7f505806e0c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5074143f60 0x7f50740ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 --2- 172.21.15.82:0/462554108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5074071740 0x7f5074137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 >> 172.21.15.82:0/462554108 conn(0x7f50740b1c70 msgr2=0x7f5074070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 shutdown_connections 2024-03-14T12:17:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:39.574+0000 7f507b2ab640 1 -- 172.21.15.82:0/462554108 wait complete. 2024-03-14T12:17:39.584 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:40.585 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:40.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.698+0000 7fe5fedd4640 1 Processor -- start 2024-03-14T12:17:40.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.698+0000 7fe5fedd4640 1 -- start start 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.699+0000 7fe5fedd4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe5f80d0760 0x7fe5f80d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.699+0000 7fe5fedd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 0x7fe5f80d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.699+0000 7fe5fedd4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80d1080 con 0x7fe5f80d1da0 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.699+0000 7fe5fedd4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80cf8c0 con 0x7fe5f80d0760 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.699+0000 7fe5fedd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80cfba0 con 0x7fe5f80cf370 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.699+0000 7fe5fd371640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe5f80d1da0 0x7fe5f80d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36972/0 (socket says 172.21.15.82:36972) 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.699+0000 7fe5fd371640 1 -- 172.21.15.82:0/3437700241 learned_addr learned my addr 172.21.15.82:0/3437700241 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f7fff640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 0x7fe5f80d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5fcb70640 1 --2- 172.21.15.82:0/3437700241 >> v2:172.21.15.164:3300/0 conn(0x7fe5f80d0760 0x7fe5f80d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53028378 0 0) 0x7fe5f80d1080 con 0x7fe5f80d1da0 2024-03-14T12:17:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e0003520 con 0x7fe5f80d1da0 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2945528857 0 0) 0x7fe5e0003520 con 0x7fe5f80d1da0 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 msgr2=0x7fe5f80d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 0x7fe5f80d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 >> v2:172.21.15.164:3300/0 conn(0x7fe5f80d0760 msgr2=0x7fe5f80d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 --2- 172.21.15.82:0/3437700241 >> v2:172.21.15.164:3300/0 conn(0x7fe5f80d0760 0x7fe5f80d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80b6c40 con 0x7fe5f80d1da0 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fe5f00019f0 con 0x7fe5f80d1da0 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 >> v1:172.21.15.82:6789/0 conn(0x7fe5f80d1da0 legacy=0x7fe5f80d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e00039b0 0x7fe5e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5e0004800 0x7fe5e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 0x7fe5e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80cf8c0 con 0x7fe5e0004800 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80cfba0 con 0x7fe5e00039b0 2024-03-14T12:17:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e000b9f0 con 0x7fe5f80cf370 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5f7fff640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5e0004800 0x7fe5e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5fcb70640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e00039b0 0x7fe5e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.700+0000 7fe5fd371640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 0x7fe5e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.701+0000 7fe5f7fff640 1 -- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 msgr2=0x7fe5e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.701+0000 7fe5f7fff640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cf370 0x7fe5e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.701+0000 7fe5f7fff640 1 -- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e00039b0 msgr2=0x7fe5e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.701+0000 7fe5f7fff640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e00039b0 0x7fe5e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.701+0000 7fe5f7fff640 1 -- 172.21.15.82:0/3437700241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0003520 con 0x7fe5e0004800 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.701+0000 7fe5f7fff640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5e0004800 0x7fe5e0004c80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009ac0 tx=0x7fe5e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150650a8cf1a2312 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.701+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec039070 con 0x7fe5e0004800 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec02c6b0 con 0x7fe5e0004800 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5f77fe640 1 -- 172.21.15.82:0/3437700241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec009690 con 0x7fe5e0004800 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5fedd4640 1 -- 172.21.15.82:0/3437700241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5e0004800 msgr2=0x7fe5e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5fedd4640 1 --2- 172.21.15.82:0/3437700241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5e0004800 0x7fe5e0004c80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009ac0 tx=0x7fe5e0005830 comp rx=0 tx=0).stop 2024-03-14T12:17:40.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5fd371640 1 -- 172.21.15.82:0/3437700241 reap_dead start 2024-03-14T12:17:40.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5fedd4640 1 -- 172.21.15.82:0/3437700241 shutdown_connections 2024-03-14T12:17:40.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5fedd4640 1 -- 172.21.15.82:0/3437700241 >> 172.21.15.82:0/3437700241 conn(0x7fe5f809d030 msgr2=0x7fe5f80d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:40.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5fedd4640 1 -- 172.21.15.82:0/3437700241 shutdown_connections 2024-03-14T12:17:40.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.702+0000 7fe5fedd4640 1 -- 172.21.15.82:0/3437700241 wait complete. 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.703+0000 7fe5fedd4640 1 Processor -- start 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.703+0000 7fe5fedd4640 1 -- start start 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fedd4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e0004800 0x7fe5f81545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fedd4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cd2b0 0x7fe5f806c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fedd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80cd730 0x7fe5f806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fedd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f81617a0 con 0x7fe5f80cd730 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fedd4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80b7200 con 0x7fe5e0004800 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fedd4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f8161920 con 0x7fe5f80cd2b0 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fd371640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80cd730 0x7fe5f806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fd371640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80cd730 0x7fe5f806cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33130/0 (socket says 172.21.15.82:33130) 2024-03-14T12:17:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.704+0000 7fe5fd371640 1 -- 172.21.15.82:0/2407464944 learned_addr learned my addr 172.21.15.82:0/2407464944 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5f7fff640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cd2b0 0x7fe5f806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fcb70640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e0004800 0x7fe5f81545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fd371640 1 -- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cd2b0 msgr2=0x7fe5f806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fd371640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cd2b0 0x7fe5f806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fd371640 1 -- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e0004800 msgr2=0x7fe5f81545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fd371640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e0004800 0x7fe5f81545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fd371640 1 -- 172.21.15.82:0/2407464944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80cb1e0 con 0x7fe5f80cd730 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fd371640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80cd730 0x7fe5f806cd40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80cfe00 tx=0x7fe5f0007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5f57fa640 1 -- 172.21.15.82:0/2407464944 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000ab20 con 0x7fe5f80cd730 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5f57fa640 1 -- 172.21.15.82:0/2407464944 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f0018070 con 0x7fe5f80cd730 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f80cb4c0 con 0x7fe5f80cd730 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5f57fa640 1 -- 172.21.15.82:0/2407464944 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00133f0 con 0x7fe5f80cd730 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.705+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f80cba80 con 0x7fe5f80cd730 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.706+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f804f050 con 0x7fe5f80cd730 2024-03-14T12:17:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.707+0000 7fe5f57fa640 1 -- 172.21.15.82:0/2407464944 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe5f0013590 con 0x7fe5f80cd730 2024-03-14T12:17:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.707+0000 7fe5f57fa640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe5e406bbd0 0x7fe5e406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.707+0000 7fe5f57fa640 1 -- 172.21.15.82:0/2407464944 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe5f008b110 con 0x7fe5f80cd730 2024-03-14T12:17:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.707+0000 7fe5fcb70640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe5e406bbd0 0x7fe5e406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.707+0000 7fe5fcb70640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe5e406bbd0 0x7fe5e406e090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009b00 tx=0x7fe5e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.711+0000 7fe5f57fa640 1 -- 172.21.15.82:0/2407464944 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe5f0059b70 con 0x7fe5f80cd730 2024-03-14T12:17:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.922+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f80d0540 con 0x7fe5f80cd730 2024-03-14T12:17:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.922+0000 7fe5f57fa640 1 -- 172.21.15.82:0/2407464944 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fe5f0059470 con 0x7fe5f80cd730 2024-03-14T12:17:40.922 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:40.922 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe5e406bbd0 msgr2=0x7fe5e406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe5e406bbd0 0x7fe5e406e090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009b00 tx=0x7fe5e8008040 comp rx=0 tx=0).stop 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80cd730 msgr2=0x7fe5f806cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80cd730 0x7fe5f806cd40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80cfe00 tx=0x7fe5f0007b00 comp rx=0 tx=0).stop 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 shutdown_connections 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80cd730 0x7fe5f806cd40 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe5e406bbd0 0x7fe5e406e090 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5f80cd2b0 0x7fe5f806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 --2- 172.21.15.82:0/2407464944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5e0004800 0x7fe5f81545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 >> 172.21.15.82:0/2407464944 conn(0x7fe5f809d030 msgr2=0x7fe5f80b9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 shutdown_connections 2024-03-14T12:17:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:40.924+0000 7fe5fedd4640 1 -- 172.21.15.82:0/2407464944 wait complete. 2024-03-14T12:17:40.935 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:41.937 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:42.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.049+0000 7f825b0e8640 1 Processor -- start 2024-03-14T12:17:42.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.049+0000 7f825b0e8640 1 -- start start 2024-03-14T12:17:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.049+0000 7f825b0e8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f825415e670 0x7f82540b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.049+0000 7f825b0e8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825415ebe0 con 0x7f8254143f60 2024-03-14T12:17:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.049+0000 7f825b0e8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825415eec0 con 0x7f8254071740 2024-03-14T12:17:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.049+0000 7f825b0e8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825415f1a0 con 0x7f825415e670 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f8259685640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8254143f60 0x7f825415b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33442/0 (socket says 172.21.15.82:33442) 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f8259685640 1 -- 172.21.15.82:0/1601745049 learned_addr learned my addr 172.21.15.82:0/1601745049 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245174991 0 0) 0x7f825415ebe0 con 0x7f8254143f60 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f823c003520 con 0x7f8254143f60 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f8253fff640 1 --2- 172.21.15.82:0/1601745049 >> v2:172.21.15.164:3300/0 conn(0x7f825415e670 0x7f82540b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4159988539 0 0) 0x7f823c003520 con 0x7f8254143f60 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 >> v2:172.21.15.164:3300/0 conn(0x7f825415e670 msgr2=0x7f82540b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 --2- 172.21.15.82:0/1601745049 >> v2:172.21.15.164:3300/0 conn(0x7f825415e670 0x7f82540b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 >> v1:172.21.15.164:6789/0 conn(0x7f8254071740 legacy=0x7f8254071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825415f480 con 0x7f8254143f60 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f8244001670 con 0x7f8254143f60 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 >> v1:172.21.15.82:6789/0 conn(0x7f8254143f60 legacy=0x7f825415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f823c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f823c004800 0x7f823c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825415f1a0 con 0x7f823c0039e0 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c00c090 con 0x7f823c0051c0 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c00c290 con 0x7f823c004800 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f8258e84640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f823c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f8259685640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.050+0000 7f8253fff640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f823c004800 0x7f823c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.051+0000 7f8258e84640 1 -- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f823c004800 msgr2=0x7f823c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.051+0000 7f8258e84640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f823c004800 0x7f823c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.051+0000 7f8258e84640 1 -- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c0051c0 msgr2=0x7f823c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.051+0000 7f8258e84640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.051+0000 7f8259685640 1 -- 172.21.15.82:0/1601745049 reap_dead start 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.051+0000 7f8258e84640 1 -- 172.21.15.82:0/1601745049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c003520 con 0x7f823c0039e0 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f8258e84640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f823c003e00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f824c001e50 tx=0x7f823c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55eaf248101a8fe server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c001a70 con 0x7f823c0039e0 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c003b70 con 0x7f823c0039e0 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f82537fe640 1 -- 172.21.15.82:0/1601745049 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c0050d0 con 0x7f823c0039e0 2024-03-14T12:17:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f825b0e8640 1 -- 172.21.15.82:0/1601745049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 msgr2=0x7f823c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:42.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f823c003e00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f824c001e50 tx=0x7f823c005830 comp rx=0 tx=0).stop 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f825b0e8640 1 -- 172.21.15.82:0/1601745049 shutdown_connections 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1601745049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f823c003e00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f825b0e8640 1 -- 172.21.15.82:0/1601745049 >> 172.21.15.82:0/1601745049 conn(0x7f82540b1c70 msgr2=0x7f82540cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f825b0e8640 1 -- 172.21.15.82:0/1601745049 shutdown_connections 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.052+0000 7f825b0e8640 1 -- 172.21.15.82:0/1601745049 wait complete. 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.054+0000 7f825b0e8640 1 Processor -- start 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.054+0000 7f825b0e8640 1 -- start start 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f82541f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825412b620 0x7f82541f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82541f5f00 0x7f8254070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8254144670 con 0x7f823c0039e0 2024-03-14T12:17:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825415b080 con 0x7f825412b620 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82541447f0 con 0x7f82541f5f00 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f82541f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f82541f2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60200/0 (socket says 172.21.15.82:60200) 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 -- 172.21.15.82:0/1283650190 learned_addr learned my addr 172.21.15.82:0/1283650190 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8253fff640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825412b620 0x7f82541f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8259685640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82541f5f00 0x7f8254070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 -- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82541f5f00 msgr2=0x7f8254070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82541f5f00 0x7f8254070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 -- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825412b620 msgr2=0x7f82541f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825412b620 0x7f82541f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 -- 172.21.15.82:0/1283650190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82541f6580 con 0x7f823c0039e0 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f8258e84640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f82541f2fc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f824c000f80 tx=0x7f824c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f82517fa640 1 -- 172.21.15.82:0/1283650190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c003970 con 0x7f823c0039e0 2024-03-14T12:17:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f82517fa640 1 -- 172.21.15.82:0/1283650190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c006b50 con 0x7f823c0039e0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82541f6860 con 0x7f823c0039e0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f82517fa640 1 -- 172.21.15.82:0/1283650190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c007020 con 0x7f823c0039e0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.055+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82541f27c0 con 0x7f823c0039e0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.056+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82540b1460 con 0x7f823c0039e0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.057+0000 7f82517fa640 1 -- 172.21.15.82:0/1283650190 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f824c0071c0 con 0x7f823c0039e0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.057+0000 7f82517fa640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f824006bbd0 0x7f824006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.058+0000 7f8253fff640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f824006bbd0 0x7f824006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.058+0000 7f82517fa640 1 -- 172.21.15.82:0/1283650190 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f824c0ac310 con 0x7f823c0039e0 2024-03-14T12:17:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.058+0000 7f8253fff640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f824006bbd0 0x7f824006e090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82541f6ac0 tx=0x7f8248008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:42.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.062+0000 7f82517fa640 1 -- 172.21.15.82:0/1283650190 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f82540b1460 con 0x7f823c0039e0 2024-03-14T12:17:42.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.270+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82541f62e0 con 0x7f823c0039e0 2024-03-14T12:17:42.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.270+0000 7f82517fa640 1 -- 172.21.15.82:0/1283650190 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f824c033020 con 0x7f823c0039e0 2024-03-14T12:17:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:42.270 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:42.270 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f824006bbd0 msgr2=0x7f824006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f824006bbd0 0x7f824006e090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82541f6ac0 tx=0x7f8248008040 comp rx=0 tx=0).stop 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 msgr2=0x7f82541f2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f82541f2fc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f824c000f80 tx=0x7f824c0014a0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 shutdown_connections 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f824006bbd0 0x7f824006e090 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f82541f5f00 0x7f8254070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825412b620 0x7f82541f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.272+0000 7f825b0e8640 1 --2- 172.21.15.82:0/1283650190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f823c0039e0 0x7f82541f2fc0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:42.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.273+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 >> 172.21.15.82:0/1283650190 conn(0x7f82540b1c70 msgr2=0x7f82541583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:42.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.273+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 shutdown_connections 2024-03-14T12:17:42.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:42.273+0000 7f825b0e8640 1 -- 172.21.15.82:0/1283650190 wait complete. 2024-03-14T12:17:42.283 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:43.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:43.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.394+0000 7fd39862b640 1 Processor -- start 2024-03-14T12:17:43.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.394+0000 7fd39862b640 1 -- start start 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.395+0000 7fd39862b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 0x7fd3900d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.395+0000 7fd39862b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3900b7720 0x7fd3900b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.395+0000 7fd39862b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3900d1080 con 0x7fd3900cf370 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.395+0000 7fd39862b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3900cf8c0 con 0x7fd3900b7720 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.395+0000 7fd39862b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900cfba0 con 0x7fd3900d0760 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd395bc6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd3900cf370 0x7fd3900d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33446/0 (socket says 172.21.15.82:33446) 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd395bc6640 1 -- 172.21.15.82:0/2325028713 learned_addr learned my addr 172.21.15.82:0/2325028713 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd396bc8640 1 --2- 172.21.15.82:0/2325028713 >> v2:172.21.15.164:3300/0 conn(0x7fd3900b7720 0x7fd3900b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244487186 0 0) 0x7fd3900d1080 con 0x7fd3900cf370 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3963c7640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 0x7fd3900d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd378003520 con 0x7fd3900cf370 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58203649 0 0) 0x7fd378003520 con 0x7fd3900cf370 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 msgr2=0x7fd3900d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 0x7fd3900d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 >> v2:172.21.15.164:3300/0 conn(0x7fd3900b7720 msgr2=0x7fd3900b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 --2- 172.21.15.82:0/2325028713 >> v2:172.21.15.164:3300/0 conn(0x7fd3900b7720 0x7fd3900b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3900b6c40 con 0x7fd3900cf370 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fd384001670 con 0x7fd3900cf370 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 >> v1:172.21.15.82:6789/0 conn(0x7fd3900cf370 legacy=0x7fd3900d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3780039b0 0x7fd378003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 0x7fd378004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd378004d40 0x7fd3780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900cf8c0 con 0x7fd378004d40 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900cfba0 con 0x7fd3780039b0 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37800b9c0 con 0x7fd3900d0760 2024-03-14T12:17:43.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd396bc8640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd378004d40 0x7fd3780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd3963c7640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3780039b0 0x7fd378003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.396+0000 7fd395bc6640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 0x7fd378004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.397+0000 7fd396bc8640 1 -- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 msgr2=0x7fd378004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.397+0000 7fd396bc8640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900d0760 0x7fd378004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.397+0000 7fd396bc8640 1 -- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3780039b0 msgr2=0x7fd378003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.397+0000 7fd396bc8640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3780039b0 0x7fd378003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.397+0000 7fd396bc8640 1 -- 172.21.15.82:0/2325028713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378003520 con 0x7fd378004d40 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.397+0000 7fd396bc8640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd378004d40 0x7fd3780051c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd380009ac0 tx=0x7fd378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732e261768d6c7af server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380039070 con 0x7fd378004d40 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38002c710 con 0x7fd378004d40 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd3953c5640 1 -- 172.21.15.82:0/2325028713 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380009690 con 0x7fd378004d40 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd39862b640 1 -- 172.21.15.82:0/2325028713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd378004d40 msgr2=0x7fd3780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd39862b640 1 --2- 172.21.15.82:0/2325028713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd378004d40 0x7fd3780051c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd380009ac0 tx=0x7fd378005830 comp rx=0 tx=0).stop 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd396bc8640 1 -- 172.21.15.82:0/2325028713 reap_dead start 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd39862b640 1 -- 172.21.15.82:0/2325028713 shutdown_connections 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd39862b640 1 -- 172.21.15.82:0/2325028713 >> 172.21.15.82:0/2325028713 conn(0x7fd39009d030 msgr2=0x7fd3900d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:43.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd39862b640 1 -- 172.21.15.82:0/2325028713 shutdown_connections 2024-03-14T12:17:43.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.398+0000 7fd39862b640 1 -- 172.21.15.82:0/2325028713 wait complete. 2024-03-14T12:17:43.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.399+0000 7fd39862b640 1 Processor -- start 2024-03-14T12:17:43.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd39862b640 1 -- start start 2024-03-14T12:17:43.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd39862b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd378004d40 0x7fd39014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd39862b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd39014a020 0x7fd39014cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd39862b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900c9610 0x7fd3900c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd39862b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3901617a0 con 0x7fd39014a020 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd39862b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900b7200 con 0x7fd378004d40 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd39862b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd390161920 con 0x7fd3900c9610 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd395bc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd39014a020 0x7fd39014cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd395bc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd39014a020 0x7fd39014cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60216/0 (socket says 172.21.15.82:60216) 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.400+0000 7fd395bc6640 1 -- 172.21.15.82:0/2621205504 learned_addr learned my addr 172.21.15.82:0/2621205504 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd3963c7640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd378004d40 0x7fd39014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd395bc6640 1 -- 172.21.15.82:0/2621205504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900c9610 msgr2=0x7fd3900c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd395bc6640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900c9610 0x7fd3900c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd395bc6640 1 -- 172.21.15.82:0/2621205504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd378004d40 msgr2=0x7fd39014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd395bc6640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd378004d40 0x7fd39014c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd395bc6640 1 -- 172.21.15.82:0/2621205504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3900ca400 con 0x7fd39014a020 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd395bc6640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd39014a020 0x7fd39014cb10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd384007340 tx=0x7fd384007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd38effd640 1 -- 172.21.15.82:0/2621205504 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400f900 con 0x7fd39014a020 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd38effd640 1 -- 172.21.15.82:0/2621205504 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384018070 con 0x7fd39014a020 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3900ca6e0 con 0x7fd39014a020 2024-03-14T12:17:43.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd38effd640 1 -- 172.21.15.82:0/2621205504 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400fce0 con 0x7fd39014a020 2024-03-14T12:17:43.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.401+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3900ca000 con 0x7fd39014a020 2024-03-14T12:17:43.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.402+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39004f050 con 0x7fd39014a020 2024-03-14T12:17:43.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.402+0000 7fd38effd640 1 -- 172.21.15.82:0/2621205504 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd38401d460 con 0x7fd39014a020 2024-03-14T12:17:43.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.403+0000 7fd38effd640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd37c06bc00 0x7fd37c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:43.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.403+0000 7fd38effd640 1 -- 172.21.15.82:0/2621205504 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd38408ac30 con 0x7fd39014a020 2024-03-14T12:17:43.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.403+0000 7fd3963c7640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd37c06bc00 0x7fd37c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:43.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.403+0000 7fd3963c7640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd37c06bc00 0x7fd37c06e0c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd3900b4ed0 tx=0x7fd38800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:43.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.407+0000 7fd38effd640 1 -- 172.21.15.82:0/2621205504 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd384059690 con 0x7fd39014a020 2024-03-14T12:17:43.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.616+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3900cb600 con 0x7fd39014a020 2024-03-14T12:17:43.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.616+0000 7fd38effd640 1 -- 172.21.15.82:0/2621205504 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fd384058f90 con 0x7fd39014a020 2024-03-14T12:17:43.616 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:43.616 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:43.616 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd37c06bc00 msgr2=0x7fd37c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd37c06bc00 0x7fd37c06e0c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd3900b4ed0 tx=0x7fd38800b040 comp rx=0 tx=0).stop 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd39014a020 msgr2=0x7fd39014cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd39014a020 0x7fd39014cb10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd384007340 tx=0x7fd384007800 comp rx=0 tx=0).stop 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 shutdown_connections 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd37c06bc00 0x7fd37c06e0c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3900c9610 0x7fd3900c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd39014a020 0x7fd39014cb10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 --2- 172.21.15.82:0/2621205504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd378004d40 0x7fd39014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:43.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 >> 172.21.15.82:0/2621205504 conn(0x7fd39009d030 msgr2=0x7fd3900b7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:43.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 shutdown_connections 2024-03-14T12:17:43.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:43.618+0000 7fd39862b640 1 -- 172.21.15.82:0/2621205504 wait complete. 2024-03-14T12:17:43.628 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:44.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:44.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.741+0000 7f0969a06640 1 Processor -- start 2024-03-14T12:17:44.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.741+0000 7f0969a06640 1 -- start start 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f0969a06640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0964071740 0x7f0964071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f0969a06640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f096415e670 0x7f09640b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f0969a06640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0964143f60 0x7f096415b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f0969a06640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0964072060 con 0x7f0964143f60 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f0969a06640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096415ebe0 con 0x7f0964071740 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f0969a06640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096415eec0 con 0x7f096415e670 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0964143f60 0x7f096415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0964143f60 0x7f096415b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60222/0 (socket says 172.21.15.82:60222) 2024-03-14T12:17:44.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 -- 172.21.15.82:0/1811313642 learned_addr learned my addr 172.21.15.82:0/1811313642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:44.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f0962ffd640 1 --2- 172.21.15.82:0/1811313642 >> v2:172.21.15.164:3300/0 conn(0x7f0964071740 0x7f0964071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:44.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09627fc640 1 --2- 172.21.15.82:0/1811313642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f096415e670 0x7f09640b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:44.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 -- 172.21.15.82:0/1811313642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f096415e670 msgr2=0x7f09640b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:44.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 --2- 172.21.15.82:0/1811313642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f096415e670 0x7f09640b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 -- 172.21.15.82:0/1811313642 >> v2:172.21.15.164:3300/0 conn(0x7f0964071740 msgr2=0x7f0964071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:44.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 --2- 172.21.15.82:0/1811313642 >> v2:172.21.15.164:3300/0 conn(0x7f0964071740 0x7f0964071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.742+0000 7f09637fe640 1 -- 172.21.15.82:0/1811313642 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096415f1a0 con 0x7f0964143f60 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.743+0000 7f09637fe640 1 --2- 172.21.15.82:0/1811313642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096415b610 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f09640d04e0 tx=0x7f095c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcf07dc2edf9279 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.743+0000 7f0961ffb640 1 -- 172.21.15.82:0/1811313642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c035020 con 0x7f0964143f60 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.743+0000 7f0961ffb640 1 -- 172.21.15.82:0/1811313642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c031870 con 0x7f0964143f60 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.743+0000 7f0961ffb640 1 -- 172.21.15.82:0/1811313642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c031c40 con 0x7f0964143f60 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 -- 172.21.15.82:0/1811313642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 msgr2=0x7f096415b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 --2- 172.21.15.82:0/1811313642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096415b610 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f09640d04e0 tx=0x7f095c02b910 comp rx=0 tx=0).stop 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 -- 172.21.15.82:0/1811313642 shutdown_connections 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 --2- 172.21.15.82:0/1811313642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096415b610 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 --2- 172.21.15.82:0/1811313642 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f096415e670 0x7f09640b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 --2- 172.21.15.82:0/1811313642 >> v2:172.21.15.164:3300/0 conn(0x7f0964071740 0x7f0964071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 -- 172.21.15.82:0/1811313642 >> 172.21.15.82:0/1811313642 conn(0x7f09640b1c70 msgr2=0x7f09640cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 -- 172.21.15.82:0/1811313642 shutdown_connections 2024-03-14T12:17:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.744+0000 7f0969a06640 1 -- 172.21.15.82:0/1811313642 wait complete. 2024-03-14T12:17:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.745+0000 7f0969a06640 1 Processor -- start 2024-03-14T12:17:44.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.745+0000 7f0969a06640 1 -- start start 2024-03-14T12:17:44.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f0969a06640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0964071740 0x7f09640c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:44.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f0969a06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:44.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f0969a06640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f096415e670 0x7f096413d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:44.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f0969a06640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096415b080 con 0x7f0964143f60 2024-03-14T12:17:44.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f0969a06640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0964143ce0 con 0x7f096415e670 2024-03-14T12:17:44.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f0969a06640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0964144670 con 0x7f0964071740 2024-03-14T12:17:44.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f09627fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f09627fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096413ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60230/0 (socket says 172.21.15.82:60230) 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f09627fc640 1 -- 172.21.15.82:0/1081506797 learned_addr learned my addr 172.21.15.82:0/1081506797 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f09637fe640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f096415e670 0x7f096413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f0962ffd640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0964071740 0x7f09640c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f09627fc640 1 -- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0964071740 msgr2=0x7f09640c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f09627fc640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0964071740 0x7f09640c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.746+0000 7f09627fc640 1 -- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f096415e670 msgr2=0x7f096413d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f09627fc640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f096415e670 0x7f096413d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f09627fc640 1 -- 172.21.15.82:0/1081506797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096415c4b0 con 0x7f0964143f60 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f09627fc640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096413ccf0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f095800d830 tx=0x7f095800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f094bfff640 1 -- 172.21.15.82:0/1081506797 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800b6c0 con 0x7f0964143f60 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f094bfff640 1 -- 172.21.15.82:0/1081506797 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958019070 con 0x7f0964143f60 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096415c790 con 0x7f0964143f60 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f094bfff640 1 -- 172.21.15.82:0/1081506797 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800ba80 con 0x7f0964143f60 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.747+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096415cd50 con 0x7f0964143f60 2024-03-14T12:17:44.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.748+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096404f050 con 0x7f0964143f60 2024-03-14T12:17:44.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.748+0000 7f094bfff640 1 -- 172.21.15.82:0/1081506797 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f095800bc20 con 0x7f0964143f60 2024-03-14T12:17:44.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.749+0000 7f094bfff640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f095006bbd0 0x7f095006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:44.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.749+0000 7f094bfff640 1 -- 172.21.15.82:0/1081506797 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f095808aee0 con 0x7f0964143f60 2024-03-14T12:17:44.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.749+0000 7f0962ffd640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f095006bbd0 0x7f095006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:44.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.749+0000 7f0962ffd640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f095006bbd0 0x7f095006e090 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f096415d7e0 tx=0x7f09540039b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:44.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.753+0000 7f094bfff640 1 -- 172.21.15.82:0/1081506797 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f09580599d0 con 0x7f0964143f60 2024-03-14T12:17:44.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.959+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09640d07b0 con 0x7f0964143f60 2024-03-14T12:17:44.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.959+0000 7f094bfff640 1 -- 172.21.15.82:0/1081506797 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f09580592d0 con 0x7f0964143f60 2024-03-14T12:17:44.959 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:44.959 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:44.959 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f095006bbd0 msgr2=0x7f095006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f095006bbd0 0x7f095006e090 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f096415d7e0 tx=0x7f09540039b0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 msgr2=0x7f096413ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096413ccf0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f095800d830 tx=0x7f095800dcf0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 shutdown_connections 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f096415e670 0x7f096413d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0964143f60 0x7f096413ccf0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f095006bbd0 0x7f095006e090 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 --2- 172.21.15.82:0/1081506797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0964071740 0x7f09640c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:44.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.961+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 >> 172.21.15.82:0/1081506797 conn(0x7f09640b1c70 msgr2=0x7f0964142b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:44.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.962+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 shutdown_connections 2024-03-14T12:17:44.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:44.962+0000 7f0969a06640 1 -- 172.21.15.82:0/1081506797 wait complete. 2024-03-14T12:17:44.972 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:45.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:46.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.078+0000 7f1627f8b640 1 Processor -- start 2024-03-14T12:17:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.078+0000 7f1627f8b640 1 -- start start 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1627f8b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 0x7f16200ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1627f8b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16200ce8b0 con 0x7f1620144ac0 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1627f8b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16200ceb90 con 0x7f162015f7a0 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1627f8b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16200cf150 con 0x7f16200cdd60 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1626528640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1620144ac0 0x7f1620144f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33448/0 (socket says 172.21.15.82:33448) 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1626528640 1 -- 172.21.15.82:0/3209290135 learned_addr learned my addr 172.21.15.82:0/3209290135 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389168575 0 0) 0x7f16200ce8b0 con 0x7f1620144ac0 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1625526640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 0x7f16200ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.079+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f160c003520 con 0x7f1620144ac0 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330053277 0 0) 0x7f160c003520 con 0x7f1620144ac0 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 msgr2=0x7f16200ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 0x7f16200ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 >> v1:172.21.15.164:6789/0 conn(0x7f162015f7a0 legacy=0x7f162015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200cee70 con 0x7f1620144ac0 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f16180019f0 con 0x7f1620144ac0 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 >> v1:172.21.15.82:6789/0 conn(0x7f1620144ac0 legacy=0x7f1620144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c0039e0 0x7f160c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f160c004800 0x7f160c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 0x7f160c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16200cf150 con 0x7f160c004800 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c00bb20 con 0x7f160c0039e0 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c00bd20 con 0x7f16200cdd60 2024-03-14T12:17:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1625526640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f160c004800 0x7f160c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1626528640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 0x7f160c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1625d27640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c0039e0 0x7f160c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1625526640 1 -- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 msgr2=0x7f160c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1625526640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cdd60 0x7f160c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1625526640 1 -- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c0039e0 msgr2=0x7f160c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1625526640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c0039e0 0x7f160c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.080+0000 7f1625526640 1 -- 172.21.15.82:0/3209290135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c003520 con 0x7f160c004800 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.081+0000 7f1625526640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f160c004800 0x7f160c004c80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f1610009ac0 tx=0x7f160c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a48dbd4c1a5f1c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.081+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610039070 con 0x7f160c004800 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.081+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161002c6b0 con 0x7f160c004800 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.081+0000 7f1624d25640 1 -- 172.21.15.82:0/3209290135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610009690 con 0x7f160c004800 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.082+0000 7f1627f8b640 1 -- 172.21.15.82:0/3209290135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f160c004800 msgr2=0x7f160c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.082+0000 7f1627f8b640 1 --2- 172.21.15.82:0/3209290135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f160c004800 0x7f160c004c80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f1610009ac0 tx=0x7f160c005830 comp rx=0 tx=0).stop 2024-03-14T12:17:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.082+0000 7f1626528640 1 -- 172.21.15.82:0/3209290135 reap_dead start 2024-03-14T12:17:46.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.082+0000 7f1627f8b640 1 -- 172.21.15.82:0/3209290135 shutdown_connections 2024-03-14T12:17:46.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.082+0000 7f1627f8b640 1 -- 172.21.15.82:0/3209290135 >> 172.21.15.82:0/3209290135 conn(0x7f16200b1c70 msgr2=0x7f16200cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:46.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.082+0000 7f1627f8b640 1 -- 172.21.15.82:0/3209290135 shutdown_connections 2024-03-14T12:17:46.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.082+0000 7f1627f8b640 1 -- 172.21.15.82:0/3209290135 wait complete. 2024-03-14T12:17:46.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.083+0000 7f1627f8b640 1 Processor -- start 2024-03-14T12:17:46.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.083+0000 7f1627f8b640 1 -- start start 2024-03-14T12:17:46.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c004800 0x7f16201f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162015ad80 0x7f16201f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cb370 0x7f1620159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16200cc670 con 0x7f162015ad80 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16200cc4f0 con 0x7f160c004800 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1620070630 con 0x7f16200cb370 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162015ad80 0x7f16201f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162015ad80 0x7f16201f1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60248/0 (socket says 172.21.15.82:60248) 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 -- 172.21.15.82:0/2544294451 learned_addr learned my addr 172.21.15.82:0/2544294451 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625d27640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c004800 0x7f16201f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 -- 172.21.15.82:0/2544294451 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cb370 msgr2=0x7f1620159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cb370 0x7f1620159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 -- 172.21.15.82:0/2544294451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c004800 msgr2=0x7f16201f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c004800 0x7f16201f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 -- 172.21.15.82:0/2544294451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1620142260 con 0x7f162015ad80 2024-03-14T12:17:46.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1625526640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162015ad80 0x7f16201f1f90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c00 tx=0x7f1610004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f16167fc640 1 -- 172.21.15.82:0/2544294451 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610042020 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f16167fc640 1 -- 172.21.15.82:0/2544294451 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161003a040 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f16167fc640 1 -- 172.21.15.82:0/2544294451 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610039070 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16200cb800 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.084+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16201f2990 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.085+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162004f050 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.086+0000 7f16167fc640 1 -- 172.21.15.82:0/2544294451 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1610045020 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.086+0000 7f16167fc640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f160406bc00 0x7f160406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.086+0000 7f16167fc640 1 -- 172.21.15.82:0/2544294451 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f16100abf50 con 0x7f162015ad80 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.086+0000 7f1625d27640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f160406bc00 0x7f160406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:46.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.086+0000 7f1625d27640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f160406bc00 0x7f160406e0c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f161c007fd0 tx=0x7f161c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:46.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.090+0000 7f16167fc640 1 -- 172.21.15.82:0/2544294451 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f161004c070 con 0x7f162015ad80 2024-03-14T12:17:46.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.299+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1620140d10 con 0x7f162015ad80 2024-03-14T12:17:46.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.300+0000 7f16167fc640 1 -- 172.21.15.82:0/2544294451 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f161007aa40 con 0x7f162015ad80 2024-03-14T12:17:46.299 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:46.299 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:46.299 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:46.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f160406bc00 msgr2=0x7f160406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:46.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f160406bc00 0x7f160406e0c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f161c007fd0 tx=0x7f161c009040 comp rx=0 tx=0).stop 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162015ad80 msgr2=0x7f16201f1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162015ad80 0x7f16201f1f90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c00 tx=0x7f1610004730 comp rx=0 tx=0).stop 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 shutdown_connections 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f160406bc00 0x7f160406e0c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16200cb370 0x7f1620159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f162015ad80 0x7f16201f1f90 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 --2- 172.21.15.82:0/2544294451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f160c004800 0x7f16201f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 >> 172.21.15.82:0/2544294451 conn(0x7f16200b1c70 msgr2=0x7f162006e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 shutdown_connections 2024-03-14T12:17:46.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:46.302+0000 7f1627f8b640 1 -- 172.21.15.82:0/2544294451 wait complete. 2024-03-14T12:17:46.312 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:47.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.425+0000 7f6440444640 1 Processor -- start 2024-03-14T12:17:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.425+0000 7f6440444640 1 -- start start 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f6440444640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64380b0f60 0x7f64380cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f6440444640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f643806b420 0x7f64380d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f6440444640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64380b1340 con 0x7f64380afff0 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f6440444640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64380d0ee0 con 0x7f643806b420 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f6440444640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380b4ee0 con 0x7f64380b0f60 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f643e9e1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f643806b420 0x7f64380d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f643e9e1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f643806b420 0x7f64380d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60250/0 (socket says 172.21.15.82:60250) 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.426+0000 7f643e9e1640 1 -- 172.21.15.82:0/3405529299 learned_addr learned my addr 172.21.15.82:0/3405529299 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643e1e0640 1 --2- 172.21.15.82:0/3405529299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64380b0f60 0x7f64380cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643e9e1640 1 -- 172.21.15.82:0/3405529299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64380b0f60 msgr2=0x7f64380cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643e9e1640 1 --2- 172.21.15.82:0/3405529299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64380b0f60 0x7f64380cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643e9e1640 1 -- 172.21.15.82:0/3405529299 >> v1:172.21.15.164:6789/0 conn(0x7f64380afff0 legacy=0x7f643806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643e9e1640 1 -- 172.21.15.82:0/3405529299 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380cd030 con 0x7f643806b420 2024-03-14T12:17:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643e1e0640 1 --2- 172.21.15.82:0/3405529299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64380b0f60 0x7f64380cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643e9e1640 1 --2- 172.21.15.82:0/3405529299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f643806b420 0x7f64380d09a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6430009870 tx=0x7f643002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9698d61dbdf73d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.427+0000 7f643d1de640 1 -- 172.21.15.82:0/3405529299 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430035020 con 0x7f643806b420 2024-03-14T12:17:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f643d1de640 1 -- 172.21.15.82:0/3405529299 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6430031640 con 0x7f643806b420 2024-03-14T12:17:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f643d1de640 1 -- 172.21.15.82:0/3405529299 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643002d040 con 0x7f643806b420 2024-03-14T12:17:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 -- 172.21.15.82:0/3405529299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f643806b420 msgr2=0x7f64380d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 --2- 172.21.15.82:0/3405529299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f643806b420 0x7f64380d09a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6430009870 tx=0x7f643002b8f0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 -- 172.21.15.82:0/3405529299 shutdown_connections 2024-03-14T12:17:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 --2- 172.21.15.82:0/3405529299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f643806b420 0x7f64380d09a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 --2- 172.21.15.82:0/3405529299 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64380b0f60 0x7f64380cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 -- 172.21.15.82:0/3405529299 >> 172.21.15.82:0/3405529299 conn(0x7f6438072310 msgr2=0x7f6438072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 -- 172.21.15.82:0/3405529299 shutdown_connections 2024-03-14T12:17:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.428+0000 7f6440444640 1 -- 172.21.15.82:0/3405529299 wait complete. 2024-03-14T12:17:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.429+0000 7f6440444640 1 Processor -- start 2024-03-14T12:17:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.429+0000 7f6440444640 1 -- start start 2024-03-14T12:17:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f6440444640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f643806b420 0x7f6438151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f6440444640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64380afff0 0x7f64380999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f6440444640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64380b0f60 0x7f64380c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f6440444640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380cd8d0 con 0x7f64380afff0 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f6440444640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380cd5f0 con 0x7f64380b0f60 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f6440444640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380b0450 con 0x7f643806b420 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f643d9df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64380afff0 0x7f64380999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f643d9df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64380afff0 0x7f64380999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60254/0 (socket says 172.21.15.82:60254) 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f643d9df640 1 -- 172.21.15.82:0/3563679162 learned_addr learned my addr 172.21.15.82:0/3563679162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.430+0000 7f643e9e1640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64380b0f60 0x7f64380c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643e1e0640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f643806b420 0x7f6438151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643d9df640 1 -- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f643806b420 msgr2=0x7f6438151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643d9df640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f643806b420 0x7f6438151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643d9df640 1 -- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64380b0f60 msgr2=0x7f64380c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643d9df640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64380b0f60 0x7f64380c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643d9df640 1 -- 172.21.15.82:0/3563679162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380c6890 con 0x7f64380afff0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643e1e0640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f643806b420 0x7f6438151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f643d9df640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64380afff0 0x7f64380999a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f642800c430 tx=0x7f642800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f642effd640 1 -- 172.21.15.82:0/3563679162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428011790 con 0x7f64380afff0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f642effd640 1 -- 172.21.15.82:0/3563679162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6428017070 con 0x7f64380afff0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64380cf4c0 con 0x7f64380afff0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f642effd640 1 -- 172.21.15.82:0/3563679162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428011bb0 con 0x7f64380afff0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.431+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64380cfa80 con 0x7f64380afff0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.432+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643804f050 con 0x7f64380afff0 2024-03-14T12:17:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.432+0000 7f642effd640 1 -- 172.21.15.82:0/3563679162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f642801c460 con 0x7f64380afff0 2024-03-14T12:17:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.433+0000 7f642effd640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f641c06bbd0 0x7f641c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.433+0000 7f642effd640 1 -- 172.21.15.82:0/3563679162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f642808a950 con 0x7f64380afff0 2024-03-14T12:17:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.433+0000 7f643e1e0640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f641c06bbd0 0x7f641c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.433+0000 7f643e1e0640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f641c06bbd0 0x7f641c06e090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f643815e780 tx=0x7f64340074b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:47.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.437+0000 7f642effd640 1 -- 172.21.15.82:0/3563679162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6428059440 con 0x7f64380afff0 2024-03-14T12:17:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.642+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6438071200 con 0x7f64380afff0 2024-03-14T12:17:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.643+0000 7f642effd640 1 -- 172.21.15.82:0/3563679162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f6428058d40 con 0x7f64380afff0 2024-03-14T12:17:47.642 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:47.642 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.644+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f641c06bbd0 msgr2=0x7f641c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.644+0000 7f6440444640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f641c06bbd0 0x7f641c06e090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f643815e780 tx=0x7f64340074b0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.644+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64380afff0 msgr2=0x7f64380999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.644+0000 7f6440444640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64380afff0 0x7f64380999a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f642800c430 tx=0x7f642800c8f0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 shutdown_connections 2024-03-14T12:17:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64380b0f60 0x7f64380c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64380afff0 0x7f64380999a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f641c06bbd0 0x7f641c06e090 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 --2- 172.21.15.82:0/3563679162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f643806b420 0x7f6438151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 >> 172.21.15.82:0/3563679162 conn(0x7f6438072310 msgr2=0x7f643806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 shutdown_connections 2024-03-14T12:17:47.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:47.645+0000 7f6440444640 1 -- 172.21.15.82:0/3563679162 wait complete. 2024-03-14T12:17:47.655 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:48.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:48.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.762+0000 7f68113d4640 1 Processor -- start 2024-03-14T12:17:48.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.762+0000 7f68113d4640 1 -- start start 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f68113d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f680c0b4190 0x7f680c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f68113d4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0cd540 con 0x7f680c15b9e0 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f68113d4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0cd810 con 0x7f680c0cd0d0 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f68113d4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c0cf730 con 0x7f680c0b4190 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f680affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f680c15b9e0 0x7f680c0d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33456/0 (socket says 172.21.15.82:33456) 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f680a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f680c0b4190 0x7f680c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f680affd640 1 -- 172.21.15.82:0/3526097481 learned_addr learned my addr 172.21.15.82:0/3526097481 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f680a7fc640 1 -- 172.21.15.82:0/3526097481 >> v1:172.21.15.164:6789/0 conn(0x7f680c0cd0d0 legacy=0x7f680c15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f680a7fc640 1 -- 172.21.15.82:0/3526097481 >> v1:172.21.15.82:6789/0 conn(0x7f680c15b9e0 legacy=0x7f680c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f680a7fc640 1 -- 172.21.15.82:0/3526097481 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0cf980 con 0x7f680c0b4190 2024-03-14T12:17:48.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.763+0000 7f6809ffb640 1 -- 172.21.15.82:0/3526097481 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591084656 0 0) 0x7f680c0cd540 con 0x7f680c15b9e0 2024-03-14T12:17:48.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.764+0000 7f680a7fc640 1 --2- 172.21.15.82:0/3526097481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c15bdc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f68000097a0 tx=0x7f680002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20511e83da10c0f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.764+0000 7f6809ffb640 1 -- 172.21.15.82:0/3526097481 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000315a0 con 0x7f680c0b4190 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.764+0000 7f6809ffb640 1 -- 172.21.15.82:0/3526097481 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800037070 con 0x7f680c0b4190 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.764+0000 7f6809ffb640 1 -- 172.21.15.82:0/3526097481 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800031960 con 0x7f680c0b4190 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.765+0000 7f68113d4640 1 -- 172.21.15.82:0/3526097481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 msgr2=0x7f680c15bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.765+0000 7f68113d4640 1 --2- 172.21.15.82:0/3526097481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c15bdc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f68000097a0 tx=0x7f680002b840 comp rx=0 tx=0).stop 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.765+0000 7f68113d4640 1 -- 172.21.15.82:0/3526097481 shutdown_connections 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.765+0000 7f68113d4640 1 --2- 172.21.15.82:0/3526097481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c15bdc0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.765+0000 7f68113d4640 1 -- 172.21.15.82:0/3526097481 >> 172.21.15.82:0/3526097481 conn(0x7f680c0b3670 msgr2=0x7f680c0b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.765+0000 7f68113d4640 1 -- 172.21.15.82:0/3526097481 shutdown_connections 2024-03-14T12:17:48.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.765+0000 7f68113d4640 1 -- 172.21.15.82:0/3526097481 wait complete. 2024-03-14T12:17:48.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.766+0000 7f68113d4640 1 Processor -- start 2024-03-14T12:17:48.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.766+0000 7f68113d4640 1 -- start start 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f68113d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c1f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f68113d4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f680c0cd0d0 0x7f680c0c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f68113d4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f680c15b9e0 0x7f680c156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f68113d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c160050 con 0x7f680c0b4190 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f68113d4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0cff40 con 0x7f680c0cd0d0 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f68113d4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c1601d0 con 0x7f680c15b9e0 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c1f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:48.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c1f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60270/0 (socket says 172.21.15.82:60270) 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 -- 172.21.15.82:0/428806887 learned_addr learned my addr 172.21.15.82:0/428806887 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680a7fc640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f680c0cd0d0 0x7f680c0c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680b7fe640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f680c15b9e0 0x7f680c156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 -- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f680c15b9e0 msgr2=0x7f680c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f680c15b9e0 0x7f680c156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 -- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f680c0cd0d0 msgr2=0x7f680c0c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f680c0cd0d0 0x7f680c0c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680affd640 1 -- 172.21.15.82:0/428806887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c06bec0 con 0x7f680c0b4190 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.767+0000 7f680a7fc640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f680c0cd0d0 0x7f680c0c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.768+0000 7f680affd640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c1f41d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c890 tx=0x7f67fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.768+0000 7f67f37fe640 1 -- 172.21.15.82:0/428806887 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc018070 con 0x7f680c0b4190 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.768+0000 7f67f37fe640 1 -- 172.21.15.82:0/428806887 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc00a640 con 0x7f680c0b4190 2024-03-14T12:17:48.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.768+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c06c1a0 con 0x7f680c0b4190 2024-03-14T12:17:48.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.768+0000 7f67f37fe640 1 -- 172.21.15.82:0/428806887 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00aa30 con 0x7f680c0b4190 2024-03-14T12:17:48.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.768+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c0c4c60 con 0x7f680c0b4190 2024-03-14T12:17:48.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.769+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c04f050 con 0x7f680c0b4190 2024-03-14T12:17:48.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.769+0000 7f67f37fe640 1 -- 172.21.15.82:0/428806887 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f67fc01b070 con 0x7f680c0b4190 2024-03-14T12:17:48.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.770+0000 7f67f37fe640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f67f806bbd0 0x7f67f806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:48.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.770+0000 7f67f37fe640 1 -- 172.21.15.82:0/428806887 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f67fc08a1d0 con 0x7f680c0b4190 2024-03-14T12:17:48.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.770+0000 7f680a7fc640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f67f806bbd0 0x7f67f806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:48.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.770+0000 7f680a7fc640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f67f806bbd0 0x7f67f806e090 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6800036000 tx=0x7f6800008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:48.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.774+0000 7f67f37fe640 1 -- 172.21.15.82:0/428806887 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f67fc058c30 con 0x7f680c0b4190 2024-03-14T12:17:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.979+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f680c0b7ae0 con 0x7f680c0b4190 2024-03-14T12:17:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.979+0000 7f67f37fe640 1 -- 172.21.15.82:0/428806887 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f67fc058530 con 0x7f680c0b4190 2024-03-14T12:17:48.979 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:48.979 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f67f806bbd0 msgr2=0x7f67f806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f67f806bbd0 0x7f67f806e090 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6800036000 tx=0x7f6800008030 comp rx=0 tx=0).stop 2024-03-14T12:17:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 msgr2=0x7f680c1f41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c1f41d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c890 tx=0x7f67fc00cd50 comp rx=0 tx=0).stop 2024-03-14T12:17:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 shutdown_connections 2024-03-14T12:17:48.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f67f806bbd0 0x7f67f806e090 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:48.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f680c15b9e0 0x7f680c156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:48.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f680c0cd0d0 0x7f680c0c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:48.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 --2- 172.21.15.82:0/428806887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0b4190 0x7f680c1f41d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:48.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.981+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 >> 172.21.15.82:0/428806887 conn(0x7f680c0b3670 msgr2=0x7f680c15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:48.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.982+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 shutdown_connections 2024-03-14T12:17:48.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:48.982+0000 7f68113d4640 1 -- 172.21.15.82:0/428806887 wait complete. 2024-03-14T12:17:48.992 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:49.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:50.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.104+0000 7f6a5a412640 1 Processor -- start 2024-03-14T12:17:50.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.104+0000 7f6a5a412640 1 -- start start 2024-03-14T12:17:50.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.105+0000 7f6a5a412640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5415e670 0x7f6a540b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:50.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.105+0000 7f6a5a412640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a54143f60 0x7f6a5415b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:50.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.105+0000 7f6a5a412640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5415ebe0 con 0x7f6a54071740 2024-03-14T12:17:50.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.105+0000 7f6a5a412640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5415eec0 con 0x7f6a54143f60 2024-03-14T12:17:50.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.105+0000 7f6a5a412640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5415f1a0 con 0x7f6a5415e670 2024-03-14T12:17:50.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a589af640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a54143f60 0x7f6a5415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5415e670 0x7f6a540b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a589af640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6a54143f60 0x7f6a5415b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:59544/0 (socket says 172.21.15.82:59544) 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5415e670 0x7f6a540b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:38834/0 (socket says 172.21.15.82:38834) 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a589af640 1 -- 172.21.15.82:0/580836569 learned_addr learned my addr 172.21.15.82:0/580836569 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a589af640 1 -- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5415e670 msgr2=0x7f6a540b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a589af640 1 --2- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5415e670 0x7f6a540b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a589af640 1 -- 172.21.15.82:0/580836569 >> v1:172.21.15.164:6789/0 conn(0x7f6a54071740 legacy=0x7f6a54071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a589af640 1 -- 172.21.15.82:0/580836569 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5415f480 con 0x7f6a54143f60 2024-03-14T12:17:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.106+0000 7f6a537fe640 1 --2- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5415e670 0x7f6a540b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.107+0000 7f6a589af640 1 --2- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a54143f60 0x7f6a5415b610 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009e50 tx=0x7f6a4c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c92fc0c1977b07d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.107+0000 7f6a52ffd640 1 -- 172.21.15.82:0/580836569 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c02cb00 con 0x7f6a54143f60 2024-03-14T12:17:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.107+0000 7f6a52ffd640 1 -- 172.21.15.82:0/580836569 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c02cca0 con 0x7f6a54143f60 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.107+0000 7f6a52ffd640 1 -- 172.21.15.82:0/580836569 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c034610 con 0x7f6a54143f60 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 -- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a54143f60 msgr2=0x7f6a5415b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 --2- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a54143f60 0x7f6a5415b610 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009e50 tx=0x7f6a4c02bf60 comp rx=0 tx=0).stop 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 -- 172.21.15.82:0/580836569 shutdown_connections 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 --2- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a54143f60 0x7f6a5415b610 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 --2- 172.21.15.82:0/580836569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a5415e670 0x7f6a540b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 -- 172.21.15.82:0/580836569 >> 172.21.15.82:0/580836569 conn(0x7f6a540b1c70 msgr2=0x7f6a540cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 -- 172.21.15.82:0/580836569 shutdown_connections 2024-03-14T12:17:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.108+0000 7f6a5a412640 1 -- 172.21.15.82:0/580836569 wait complete. 2024-03-14T12:17:50.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.109+0000 7f6a5a412640 1 Processor -- start 2024-03-14T12:17:50.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.109+0000 7f6a5a412640 1 -- start start 2024-03-14T12:17:50.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a5a412640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a54071740 0x7f6a54158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:50.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a5a412640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a54143f60 0x7f6a541e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:50.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a5a412640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5415e670 0x7f6a54070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:50.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a5a412640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a54144670 con 0x7f6a54143f60 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a5a412640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5415b080 con 0x7f6a5415e670 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a5a412640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541447f0 con 0x7f6a54071740 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a54143f60 0x7f6a541e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a54143f60 0x7f6a541e44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60284/0 (socket says 172.21.15.82:60284) 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 -- 172.21.15.82:0/704629592 learned_addr learned my addr 172.21.15.82:0/704629592 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a589af640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5415e670 0x7f6a54070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a53fff640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a54071740 0x7f6a54158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 -- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a54071740 msgr2=0x7f6a54158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a54071740 0x7f6a54158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 -- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5415e670 msgr2=0x7f6a54070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5415e670 0x7f6a54070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 -- 172.21.15.82:0/704629592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a541e4f60 con 0x7f6a54143f60 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a537fe640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a54143f60 0x7f6a541e44a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400ec10 tx=0x7f6a4400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.110+0000 7f6a50ff9640 1 -- 172.21.15.82:0/704629592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44017070 con 0x7f6a54143f60 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.111+0000 7f6a50ff9640 1 -- 172.21.15.82:0/704629592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4400cc30 con 0x7f6a54143f60 2024-03-14T12:17:50.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.111+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a541e5240 con 0x7f6a54143f60 2024-03-14T12:17:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.111+0000 7f6a50ff9640 1 -- 172.21.15.82:0/704629592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44013470 con 0x7f6a54143f60 2024-03-14T12:17:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.111+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a541e5800 con 0x7f6a54143f60 2024-03-14T12:17:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.112+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5404f050 con 0x7f6a54143f60 2024-03-14T12:17:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.112+0000 7f6a50ff9640 1 -- 172.21.15.82:0/704629592 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6a4401a070 con 0x7f6a54143f60 2024-03-14T12:17:50.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.112+0000 7f6a50ff9640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4006bbd0 0x7f6a4006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:50.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.113+0000 7f6a53fff640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4006bbd0 0x7f6a4006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:50.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.113+0000 7f6a50ff9640 1 -- 172.21.15.82:0/704629592 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6a4408ad80 con 0x7f6a54143f60 2024-03-14T12:17:50.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.113+0000 7f6a53fff640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4006bbd0 0x7f6a4006e090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6a48007fd0 tx=0x7f6a48009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.117+0000 7f6a50ff9640 1 -- 172.21.15.82:0/704629592 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6a44059870 con 0x7f6a54143f60 2024-03-14T12:17:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.324+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a540d07b0 con 0x7f6a54143f60 2024-03-14T12:17:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.325+0000 7f6a50ff9640 1 -- 172.21.15.82:0/704629592 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f6a44059170 con 0x7f6a54143f60 2024-03-14T12:17:50.324 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:50.324 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4006bbd0 msgr2=0x7f6a4006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4006bbd0 0x7f6a4006e090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6a48007fd0 tx=0x7f6a48009040 comp rx=0 tx=0).stop 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a54143f60 msgr2=0x7f6a541e44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a54143f60 0x7f6a541e44a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400ec10 tx=0x7f6a4400c520 comp rx=0 tx=0).stop 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 shutdown_connections 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a5415e670 0x7f6a54070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a54143f60 0x7f6a541e44a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6a4006bbd0 0x7f6a4006e090 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 --2- 172.21.15.82:0/704629592 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a54071740 0x7f6a54158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 >> 172.21.15.82:0/704629592 conn(0x7f6a540b1c70 msgr2=0x7f6a540cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 shutdown_connections 2024-03-14T12:17:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:50.327+0000 7f6a5a412640 1 -- 172.21.15.82:0/704629592 wait complete. 2024-03-14T12:17:50.337 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:51.338 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:51.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.452+0000 7f24bd789640 1 Processor -- start 2024-03-14T12:17:51.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.452+0000 7f24bd789640 1 -- start start 2024-03-14T12:17:51.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.452+0000 7f24bd789640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f24b815e0e0 0x7f24b80d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:51.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24bd789640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b815e4c0 con 0x7f24b80b3e10 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24bd789640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b80cf2f0 con 0x7f24b80d4a20 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24bd789640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80d1640 con 0x7f24b815e0e0 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24b6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f24b815e0e0 0x7f24b80d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24b77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f24b80b3e10 0x7f24b80b4240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:33460/0 (socket says 172.21.15.82:33460) 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24b77fe640 1 -- 172.21.15.82:0/3065788214 learned_addr learned my addr 172.21.15.82:0/3065788214 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24b6ffd640 1 -- 172.21.15.82:0/3065788214 >> v1:172.21.15.164:6789/0 conn(0x7f24b80d4a20 legacy=0x7f24b80d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24b6ffd640 1 -- 172.21.15.82:0/3065788214 >> v1:172.21.15.82:6789/0 conn(0x7f24b80b3e10 legacy=0x7f24b80b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24b5ffb640 1 -- 172.21.15.82:0/3065788214 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730664124 0 0) 0x7f24b815e4c0 con 0x7f24b80b3e10 2024-03-14T12:17:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.453+0000 7f24b6ffd640 1 -- 172.21.15.82:0/3065788214 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80cf5d0 con 0x7f24b815e0e0 2024-03-14T12:17:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.454+0000 7f24b6ffd640 1 --2- 172.21.15.82:0/3065788214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b815e0e0 0x7f24b80d1910 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f24b80d17c0 tx=0x7f24a802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886d234fc8a03af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.454+0000 7f24b5ffb640 1 -- 172.21.15.82:0/3065788214 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8034070 con 0x7f24b815e0e0 2024-03-14T12:17:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.454+0000 7f24b5ffb640 1 -- 172.21.15.82:0/3065788214 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8030610 con 0x7f24b815e0e0 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.454+0000 7f24b5ffb640 1 -- 172.21.15.82:0/3065788214 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80309e0 con 0x7f24b815e0e0 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.455+0000 7f24bd789640 1 -- 172.21.15.82:0/3065788214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b815e0e0 msgr2=0x7f24b80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.455+0000 7f24bd789640 1 --2- 172.21.15.82:0/3065788214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b815e0e0 0x7f24b80d1910 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f24b80d17c0 tx=0x7f24a802b960 comp rx=0 tx=0).stop 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.455+0000 7f24bd789640 1 -- 172.21.15.82:0/3065788214 shutdown_connections 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.455+0000 7f24bd789640 1 --2- 172.21.15.82:0/3065788214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b815e0e0 0x7f24b80d1910 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.455+0000 7f24bd789640 1 -- 172.21.15.82:0/3065788214 >> 172.21.15.82:0/3065788214 conn(0x7f24b80d3d40 msgr2=0x7f24b80d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.455+0000 7f24bd789640 1 -- 172.21.15.82:0/3065788214 shutdown_connections 2024-03-14T12:17:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.455+0000 7f24bd789640 1 -- 172.21.15.82:0/3065788214 wait complete. 2024-03-14T12:17:51.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.456+0000 7f24bd789640 1 Processor -- start 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.456+0000 7f24bd789640 1 -- start start 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24bd789640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b80b3e10 0x7f24b80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24bd789640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24b80d4a20 0x7f24b8152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24bd789640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b815e0e0 0x7f24b80b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24bd789640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80b3610 con 0x7f24b80b3e10 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24bd789640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80cfb90 con 0x7f24b815e0e0 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24bd789640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80b38f0 con 0x7f24b80d4a20 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b80b3e10 0x7f24b80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b80b3e10 0x7f24b80ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49826/0 (socket says 172.21.15.82:49826) 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 -- 172.21.15.82:0/1905087820 learned_addr learned my addr 172.21.15.82:0/1905087820 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b77fe640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b815e0e0 0x7f24b80b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b67fc640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24b80d4a20 0x7f24b8152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 -- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24b80d4a20 msgr2=0x7f24b8152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24b80d4a20 0x7f24b8152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 -- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b815e0e0 msgr2=0x7f24b80b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b815e0e0 0x7f24b80b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 -- 172.21.15.82:0/1905087820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8153090 con 0x7f24b80b3e10 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b77fe640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b815e0e0 0x7f24b80b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f24b6ffd640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b80b3e10 0x7f24b80ca210 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f24a8000c00 tx=0x7f24a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f249f7fe640 1 -- 172.21.15.82:0/1905087820 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8034040 con 0x7f24b80b3e10 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.457+0000 7f249f7fe640 1 -- 172.21.15.82:0/1905087820 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8004030 con 0x7f24b80b3e10 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.458+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b8160d00 con 0x7f24b80b3e10 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.458+0000 7f249f7fe640 1 -- 172.21.15.82:0/1905087820 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8004420 con 0x7f24b80b3e10 2024-03-14T12:17:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.458+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b80b8ab0 con 0x7f24b80b3e10 2024-03-14T12:17:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.459+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b804f050 con 0x7f24b80b3e10 2024-03-14T12:17:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.459+0000 7f249f7fe640 1 -- 172.21.15.82:0/1905087820 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f24a8037070 con 0x7f24b80b3e10 2024-03-14T12:17:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.459+0000 7f249f7fe640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f24a406bc00 0x7f24a406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.460+0000 7f249f7fe640 1 -- 172.21.15.82:0/1905087820 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f24a80ac7c0 con 0x7f24b80b3e10 2024-03-14T12:17:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.460+0000 7f24b67fc640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f24a406bc00 0x7f24a406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.460+0000 7f24b67fc640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f24a406bc00 0x7f24a406e0c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f24b81538f0 tx=0x7f24ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:51.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.464+0000 7f249f7fe640 1 -- 172.21.15.82:0/1905087820 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f24a807b3d0 con 0x7f24b80b3e10 2024-03-14T12:17:51.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.671+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24b80cea80 con 0x7f24b80b3e10 2024-03-14T12:17:51.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.672+0000 7f249f7fe640 1 -- 172.21.15.82:0/1905087820 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f24a80810d0 con 0x7f24b80b3e10 2024-03-14T12:17:51.671 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:51.671 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:51.671 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.673+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f24a406bc00 msgr2=0x7f24a406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.673+0000 7f24bd789640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f24a406bc00 0x7f24a406e0c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f24b81538f0 tx=0x7f24ac009040 comp rx=0 tx=0).stop 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.673+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b80b3e10 msgr2=0x7f24b80ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.673+0000 7f24bd789640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b80b3e10 0x7f24b80ca210 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f24a8000c00 tx=0x7f24a80046c0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.673+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 shutdown_connections 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.674+0000 7f24bd789640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b815e0e0 0x7f24b80b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.674+0000 7f24bd789640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f24a406bc00 0x7f24a406e0c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.674+0000 7f24bd789640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24b80d4a20 0x7f24b8152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.674+0000 7f24bd789640 1 --2- 172.21.15.82:0/1905087820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b80b3e10 0x7f24b80ca210 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.674+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 >> 172.21.15.82:0/1905087820 conn(0x7f24b80d3d40 msgr2=0x7f24b806c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.674+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 shutdown_connections 2024-03-14T12:17:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:51.674+0000 7f24bd789640 1 -- 172.21.15.82:0/1905087820 wait complete. 2024-03-14T12:17:51.684 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:52.251 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:17:52.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:17:52.685 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:52.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.794+0000 7fdbc4f7e640 1 Processor -- start 2024-03-14T12:17:52.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.794+0000 7fdbc4f7e640 1 -- start start 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbc4f7e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbc00cf370 0x7fdbc00d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbc4f7e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdbc00b7720 0x7fdbc00b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbc4f7e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc00cf8c0 con 0x7fdbc00d0760 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbc4f7e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc00cfba0 con 0x7fdbc00cf370 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbc4f7e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc00b6c40 con 0x7fdbc00b7720 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbbdd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbc00cf370 0x7fdbc00d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbbdd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbc00cf370 0x7fdbc00d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49828/0 (socket says 172.21.15.82:49828) 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.795+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/3089196839 learned_addr learned my addr 172.21.15.82:0/3089196839 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:52.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.796+0000 7fdbbed9d640 1 --2- 172.21.15.82:0/3089196839 >> v2:172.21.15.164:3300/0 conn(0x7fdbc00b7720 0x7fdbc00b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.796+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/3089196839 >> v2:172.21.15.164:3300/0 conn(0x7fdbc00b7720 msgr2=0x7fdbc00b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.796+0000 7fdbbdd9b640 1 --2- 172.21.15.82:0/3089196839 >> v2:172.21.15.164:3300/0 conn(0x7fdbc00b7720 0x7fdbc00b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.796+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/3089196839 >> v1:172.21.15.164:6789/0 conn(0x7fdbc00d0760 legacy=0x7fdbc00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.796+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/3089196839 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc00b6f20 con 0x7fdbc00cf370 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.796+0000 7fdbbdd9b640 1 --2- 172.21.15.82:0/3089196839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00d4660 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009870 tx=0x7fdbb402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00f7c77d12e2479 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbbd59a640 1 -- 172.21.15.82:0/3089196839 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb402f840 con 0x7fdbc00cf370 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbbd59a640 1 -- 172.21.15.82:0/3089196839 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb402f9e0 con 0x7fdbc00cf370 2024-03-14T12:17:52.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbbd59a640 1 -- 172.21.15.82:0/3089196839 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb402fdb0 con 0x7fdbc00cf370 2024-03-14T12:17:52.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/3089196839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 msgr2=0x7fdbc00d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/3089196839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00d4660 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009870 tx=0x7fdbb402ba20 comp rx=0 tx=0).stop 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/3089196839 shutdown_connections 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/3089196839 >> v2:172.21.15.164:3300/0 conn(0x7fdbc00b7720 0x7fdbc00b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/3089196839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00d4660 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/3089196839 >> 172.21.15.82:0/3089196839 conn(0x7fdbc009d030 msgr2=0x7fdbc00d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/3089196839 shutdown_connections 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.797+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/3089196839 wait complete. 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 Processor -- start 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 -- start start 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc00b7720 0x7fdbc0152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc00d0760 0x7fdbc015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc01617a0 con 0x7fdbc00cf370 2024-03-14T12:17:52.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc00d4ba0 con 0x7fdbc00b7720 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbc4f7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc0161920 con 0x7fdbc00d0760 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.799+0000 7fdbbdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49832/0 (socket says 172.21.15.82:49832) 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/596997616 learned_addr learned my addr 172.21.15.82:0/596997616 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbe59c640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc00b7720 0x7fdbc0152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbed9d640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc00d0760 0x7fdbc015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc00d0760 msgr2=0x7fdbc015cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc00d0760 0x7fdbc015cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc00b7720 msgr2=0x7fdbc0152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc00b7720 0x7fdbc0152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 -- 172.21.15.82:0/596997616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc00cbf40 con 0x7fdbc00cf370 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbe59c640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc00b7720 0x7fdbc0152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbbdd9b640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00cd810 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000c00 tx=0x7fdbb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:52.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbaeffd640 1 -- 172.21.15.82:0/596997616 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4037c90 con 0x7fdbc00cf370 2024-03-14T12:17:52.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbaeffd640 1 -- 172.21.15.82:0/596997616 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb4046070 con 0x7fdbc00cf370 2024-03-14T12:17:52.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbaeffd640 1 -- 172.21.15.82:0/596997616 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4041640 con 0x7fdbc00cf370 2024-03-14T12:17:52.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc0158ab0 con 0x7fdbc00cf370 2024-03-14T12:17:52.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.800+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc0159070 con 0x7fdbc00cf370 2024-03-14T12:17:52.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.801+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc004f050 con 0x7fdbc00cf370 2024-03-14T12:17:52.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.801+0000 7fdbaeffd640 1 -- 172.21.15.82:0/596997616 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdbb4040070 con 0x7fdbc00cf370 2024-03-14T12:17:52.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.802+0000 7fdbaeffd640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdba006bbd0 0x7fdba006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:52.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.802+0000 7fdbaeffd640 1 -- 172.21.15.82:0/596997616 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdbb40ad280 con 0x7fdbc00cf370 2024-03-14T12:17:52.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.802+0000 7fdbbe59c640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdba006bbd0 0x7fdba006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:52.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.802+0000 7fdbbe59c640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdba006bbd0 0x7fdba006e090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001cd0 tx=0x7fdbb0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:52.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:52.806+0000 7fdbaeffd640 1 -- 172.21.15.82:0/596997616 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdbb4033030 con 0x7fdbc00cf370 2024-03-14T12:17:53.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.014+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbc00d0540 con 0x7fdbc00cf370 2024-03-14T12:17:53.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.014+0000 7fdbaeffd640 1 -- 172.21.15.82:0/596997616 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fdbb407bce0 con 0x7fdbc00cf370 2024-03-14T12:17:53.014 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:53.014 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:53.014 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdba006bbd0 msgr2=0x7fdba006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdba006bbd0 0x7fdba006e090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001cd0 tx=0x7fdbb0009040 comp rx=0 tx=0).stop 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 msgr2=0x7fdbc00cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00cd810 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000c00 tx=0x7fdbb4004730 comp rx=0 tx=0).stop 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 shutdown_connections 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdba006bbd0 0x7fdba006e090 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbc00d0760 0x7fdbc015cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbc00cf370 0x7fdbc00cd810 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 --2- 172.21.15.82:0/596997616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbc00b7720 0x7fdbc0152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 >> 172.21.15.82:0/596997616 conn(0x7fdbc009d030 msgr2=0x7fdbc00d23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:53.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 shutdown_connections 2024-03-14T12:17:53.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:53.016+0000 7fdbc4f7e640 1 -- 172.21.15.82:0/596997616 wait complete. 2024-03-14T12:17:53.027 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:54.028 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:54.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.134+0000 7fb435789640 1 Processor -- start 2024-03-14T12:17:54.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.134+0000 7fb435789640 1 -- start start 2024-03-14T12:17:54.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb435789640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4300d0760 0x7fb4300d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb435789640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 0x7fb4300b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb435789640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4300d1080 con 0x7fb4300cf370 2024-03-14T12:17:54.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb435789640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4300cf8c0 con 0x7fb4300d0760 2024-03-14T12:17:54.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb435789640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300cfba0 con 0x7fb4300b7720 2024-03-14T12:17:54.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb427fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb4300cf370 0x7fb4300d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38170/0 (socket says 172.21.15.82:38170) 2024-03-14T12:17:54.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb427fff640 1 -- 172.21.15.82:0/196749290 learned_addr learned my addr 172.21.15.82:0/196749290 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42effd640 1 --2- 172.21.15.82:0/196749290 >> v2:172.21.15.164:3300/0 conn(0x7fb4300d0760 0x7fb4300d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 0x7fb4300b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996882377 0 0) 0x7fb4300d1080 con 0x7fb4300cf370 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb41c003520 con 0x7fb4300cf370 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 114502544 0 0) 0x7fb41c003520 con 0x7fb4300cf370 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 msgr2=0x7fb4300b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 0x7fb4300b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 >> v2:172.21.15.164:3300/0 conn(0x7fb4300d0760 msgr2=0x7fb4300d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 --2- 172.21.15.82:0/196749290 >> v2:172.21.15.164:3300/0 conn(0x7fb4300d0760 0x7fb4300d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4300b6c40 con 0x7fb4300cf370 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fb418001670 con 0x7fb4300cf370 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 >> v1:172.21.15.82:6789/0 conn(0x7fb4300cf370 legacy=0x7fb4300d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0039b0 0x7fb41c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 0x7fb41c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41c004d40 0x7fb41c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300cf8c0 con 0x7fb41c004d40 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300cfba0 con 0x7fb41c0039b0 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.135+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c00b9c0 con 0x7fb4300b7720 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41c004d40 0x7fb41c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb42effd640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0039b0 0x7fb41c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb427fff640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 0x7fb41c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb42f7fe640 1 -- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 msgr2=0x7fb41c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4300b7720 0x7fb41c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb42f7fe640 1 -- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0039b0 msgr2=0x7fb41c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c0039b0 0x7fb41c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.136+0000 7fb42f7fe640 1 -- 172.21.15.82:0/196749290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c003520 con 0x7fb41c004d40 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41c004d40 0x7fb41c0051c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb428009fc0 tx=0x7fb41c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1605fdcd727ac0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428039070 con 0x7fb41c004d40 2024-03-14T12:17:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42802cc20 con 0x7fb41c004d40 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb42e7fc640 1 -- 172.21.15.82:0/196749290 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428009c10 con 0x7fb41c004d40 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb435789640 1 -- 172.21.15.82:0/196749290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41c004d40 msgr2=0x7fb41c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb435789640 1 --2- 172.21.15.82:0/196749290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41c004d40 0x7fb41c0051c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb428009fc0 tx=0x7fb41c005830 comp rx=0 tx=0).stop 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb42f7fe640 1 -- 172.21.15.82:0/196749290 reap_dead start 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb435789640 1 -- 172.21.15.82:0/196749290 shutdown_connections 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb435789640 1 -- 172.21.15.82:0/196749290 >> 172.21.15.82:0/196749290 conn(0x7fb43009d030 msgr2=0x7fb4300d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb435789640 1 -- 172.21.15.82:0/196749290 shutdown_connections 2024-03-14T12:17:54.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.137+0000 7fb435789640 1 -- 172.21.15.82:0/196749290 wait complete. 2024-03-14T12:17:54.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.138+0000 7fb435789640 1 Processor -- start 2024-03-14T12:17:54.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.138+0000 7fb435789640 1 -- start start 2024-03-14T12:17:54.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb435789640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c004d40 0x7fb43014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb435789640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb430156c50 0x7fb43006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb435789640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43006cb10 0x7fb430154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb435789640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4301617a0 con 0x7fb43006cb10 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb435789640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300b7200 con 0x7fb41c004d40 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb435789640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb430161920 con 0x7fb430156c50 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb42f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43006cb10 0x7fb430154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb42f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43006cb10 0x7fb430154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49850/0 (socket says 172.21.15.82:49850) 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb42f7fe640 1 -- 172.21.15.82:0/2327055206 learned_addr learned my addr 172.21.15.82:0/2327055206 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.139+0000 7fb42effd640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c004d40 0x7fb43014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb427fff640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb430156c50 0x7fb43006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb42f7fe640 1 -- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb430156c50 msgr2=0x7fb43006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb430156c50 0x7fb43006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb42f7fe640 1 -- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c004d40 msgr2=0x7fb43014b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c004d40 0x7fb43014b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb42f7fe640 1 -- 172.21.15.82:0/2327055206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb430155bb0 con 0x7fb43006cb10 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43006cb10 0x7fb430154ef0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e230 tx=0x7fb428009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb4277fe640 1 -- 172.21.15.82:0/2327055206 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428039030 con 0x7fb43006cb10 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb4277fe640 1 -- 172.21.15.82:0/2327055206 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb428043070 con 0x7fb43006cb10 2024-03-14T12:17:54.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb430155e90 con 0x7fb43006cb10 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb4277fe640 1 -- 172.21.15.82:0/2327055206 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42803d7f0 con 0x7fb43006cb10 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.140+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb430156450 con 0x7fb43006cb10 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.141+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43004f050 con 0x7fb43006cb10 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.141+0000 7fb4277fe640 1 -- 172.21.15.82:0/2327055206 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb428005d10 con 0x7fb43006cb10 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.142+0000 7fb4277fe640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb41006bc00 0x7fb41006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.142+0000 7fb4277fe640 1 -- 172.21.15.82:0/2327055206 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb4280aced0 con 0x7fb43006cb10 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.142+0000 7fb42effd640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb41006bc00 0x7fb41006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.142+0000 7fb42effd640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb41006bc00 0x7fb41006e0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb4200099c0 tx=0x7fb420008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:54.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.146+0000 7fb4277fe640 1 -- 172.21.15.82:0/2327055206 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb42804d020 con 0x7fb43006cb10 2024-03-14T12:17:54.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.352+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4300caa40 con 0x7fb43006cb10 2024-03-14T12:17:54.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.352+0000 7fb4277fe640 1 -- 172.21.15.82:0/2327055206 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fb42807b9c0 con 0x7fb43006cb10 2024-03-14T12:17:54.352 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:54.352 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:54.352 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb41006bc00 msgr2=0x7fb41006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb41006bc00 0x7fb41006e0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb4200099c0 tx=0x7fb420008040 comp rx=0 tx=0).stop 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43006cb10 msgr2=0x7fb430154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43006cb10 0x7fb430154ef0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e230 tx=0x7fb428009750 comp rx=0 tx=0).stop 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 shutdown_connections 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43006cb10 0x7fb430154ef0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb41006bc00 0x7fb41006e0c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb430156c50 0x7fb43006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 --2- 172.21.15.82:0/2327055206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c004d40 0x7fb43014b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 >> 172.21.15.82:0/2327055206 conn(0x7fb43009d030 msgr2=0x7fb4300b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 shutdown_connections 2024-03-14T12:17:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:54.354+0000 7fb435789640 1 -- 172.21.15.82:0/2327055206 wait complete. 2024-03-14T12:17:54.365 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:55.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:55.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.477+0000 7f3443bc4640 1 Processor -- start 2024-03-14T12:17:55.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.478+0000 7f3443bc4640 1 -- start start 2024-03-14T12:17:55.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3443bc4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f343c0cf370 0x7f343c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3443bc4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f343c0b7720 0x7f343c0b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3443bc4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c0cf8c0 con 0x7f343c0d0760 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3443bc4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c0cfba0 con 0x7f343c0b7720 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3443bc4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c0b6c40 con 0x7f343c0cf370 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3442161640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f343c0b7720 0x7f343c0b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3441960640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f343c0d0760 0x7f343c0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38186/0 (socket says 172.21.15.82:38186) 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3442161640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f343c0b7720 0x7f343c0b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49856/0 (socket says 172.21.15.82:49856) 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3441960640 1 -- 172.21.15.82:0/1148610188 learned_addr learned my addr 172.21.15.82:0/1148610188 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f344115f640 1 --2- 172.21.15.82:0/1148610188 >> v2:172.21.15.164:3300/0 conn(0x7f343c0cf370 0x7f343c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3442161640 1 -- 172.21.15.82:0/1148610188 >> v2:172.21.15.164:3300/0 conn(0x7f343c0cf370 msgr2=0x7f343c0d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3442161640 1 --2- 172.21.15.82:0/1148610188 >> v2:172.21.15.164:3300/0 conn(0x7f343c0cf370 0x7f343c0d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3442161640 1 -- 172.21.15.82:0/1148610188 >> v1:172.21.15.82:6789/0 conn(0x7f343c0d0760 legacy=0x7f343c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:55.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f3442161640 1 -- 172.21.15.82:0/1148610188 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0b6f20 con 0x7f343c0b7720 2024-03-14T12:17:55.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.479+0000 7f344095e640 1 -- 172.21.15.82:0/1148610188 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034543770 0 0) 0x7f343c0cf8c0 con 0x7f343c0d0760 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.480+0000 7f3442161640 1 --2- 172.21.15.82:0/1148610188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c0b7b70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f342c0097a0 tx=0x7f342c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfb9e71c2d19957 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.480+0000 7f344095e640 1 -- 172.21.15.82:0/1148610188 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c034070 con 0x7f343c0b7720 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.480+0000 7f344095e640 1 -- 172.21.15.82:0/1148610188 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c030610 con 0x7f343c0b7720 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.480+0000 7f344095e640 1 -- 172.21.15.82:0/1148610188 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c0309e0 con 0x7f343c0b7720 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 -- 172.21.15.82:0/1148610188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 msgr2=0x7f343c0b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 --2- 172.21.15.82:0/1148610188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c0b7b70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f342c0097a0 tx=0x7f342c02b960 comp rx=0 tx=0).stop 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 -- 172.21.15.82:0/1148610188 shutdown_connections 2024-03-14T12:17:55.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 --2- 172.21.15.82:0/1148610188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c0b7b70 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 --2- 172.21.15.82:0/1148610188 >> v2:172.21.15.164:3300/0 conn(0x7f343c0cf370 0x7f343c0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 -- 172.21.15.82:0/1148610188 >> 172.21.15.82:0/1148610188 conn(0x7f343c09d030 msgr2=0x7f343c0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 -- 172.21.15.82:0/1148610188 shutdown_connections 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.481+0000 7f3443bc4640 1 -- 172.21.15.82:0/1148610188 wait complete. 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.482+0000 7f3443bc4640 1 Processor -- start 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.482+0000 7f3443bc4640 1 -- start start 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.482+0000 7f3443bc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3443bc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f343c0cf370 0x7f343c0c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3443bc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f343c0d0760 0x7f343c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3443bc4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c1617a0 con 0x7f343c0b7720 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3443bc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0d4ba0 con 0x7f343c0cf370 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3443bc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c161920 con 0x7f343c0d0760 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49862/0 (socket says 172.21.15.82:49862) 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 -- 172.21.15.82:0/4143198591 learned_addr learned my addr 172.21.15.82:0/4143198591 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 -- 172.21.15.82:0/4143198591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f343c0d0760 msgr2=0x7f343c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f343c0d0760 0x7f343c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 -- 172.21.15.82:0/4143198591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f343c0cf370 msgr2=0x7f343c0c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f343c0cf370 0x7f343c0c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 -- 172.21.15.82:0/4143198591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0b7d50 con 0x7f343c0b7720 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3441960640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c147540 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f343400c7f0 tx=0x7f343400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f343a7fc640 1 -- 172.21.15.82:0/4143198591 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434018070 con 0x7f343c0b7720 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f343a7fc640 1 -- 172.21.15.82:0/4143198591 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343400a640 con 0x7f343c0b7720 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f343a7fc640 1 -- 172.21.15.82:0/4143198591 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400aa30 con 0x7f343c0b7720 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c0b8030 con 0x7f343c0b7720 2024-03-14T12:17:55.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.483+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c0b85f0 con 0x7f343c0b7720 2024-03-14T12:17:55.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.484+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c04f050 con 0x7f343c0b7720 2024-03-14T12:17:55.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.485+0000 7f343a7fc640 1 -- 172.21.15.82:0/4143198591 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f343401b070 con 0x7f343c0b7720 2024-03-14T12:17:55.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.485+0000 7f343a7fc640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f342806bbd0 0x7f342806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:55.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.485+0000 7f343a7fc640 1 -- 172.21.15.82:0/4143198591 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f343408a150 con 0x7f343c0b7720 2024-03-14T12:17:55.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.485+0000 7f344115f640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f342806bbd0 0x7f342806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:55.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.486+0000 7f344115f640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f342806bbd0 0x7f342806e090 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f343c0cffa0 tx=0x7f3430008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:55.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.490+0000 7f343a7fc640 1 -- 172.21.15.82:0/4143198591 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3434058bb0 con 0x7f343c0b7720 2024-03-14T12:17:55.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.696+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f343c0d0540 con 0x7f343c0b7720 2024-03-14T12:17:55.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.697+0000 7f343a7fc640 1 -- 172.21.15.82:0/4143198591 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f34340584b0 con 0x7f343c0b7720 2024-03-14T12:17:55.696 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:55.696 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:55.697 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:55.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f342806bbd0 msgr2=0x7f342806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:55.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f342806bbd0 0x7f342806e090 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f343c0cffa0 tx=0x7f3430008040 comp rx=0 tx=0).stop 2024-03-14T12:17:55.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 msgr2=0x7f343c147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c147540 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f343400c7f0 tx=0x7f343400ccb0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 shutdown_connections 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f342806bbd0 0x7f342806e090 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f343c0d0760 0x7f343c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f343c0cf370 0x7f343c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 --2- 172.21.15.82:0/4143198591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343c0b7720 0x7f343c147540 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 >> 172.21.15.82:0/4143198591 conn(0x7f343c09d030 msgr2=0x7f343c15e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 shutdown_connections 2024-03-14T12:17:55.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:55.699+0000 7f3443bc4640 1 -- 172.21.15.82:0/4143198591 wait complete. 2024-03-14T12:17:55.709 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:56.710 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.823+0000 7f9fc9c59640 1 Processor -- start 2024-03-14T12:17:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.823+0000 7f9fc9c59640 1 -- start start 2024-03-14T12:17:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.824+0000 7f9fc9c59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc41551d0 0x7f9fc40b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.824+0000 7f9fc9c59640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9fc4162190 0x7f9fc406f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.824+0000 7f9fc9c59640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40b60f0 con 0x7f9fc40b5bb0 2024-03-14T12:17:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.824+0000 7f9fc9c59640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc40b6d00 con 0x7f9fc4162190 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.824+0000 7f9fc9c59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40b63d0 con 0x7f9fc41551d0 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc3fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9fc4162190 0x7f9fc406f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc37fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc41551d0 0x7f9fc40b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc37fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc41551d0 0x7f9fc40b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:53316/0 (socket says 172.21.15.82:53316) 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc3fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9fc4162190 0x7f9fc406f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:36458/0 (socket says 172.21.15.82:36458) 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc37fe640 1 -- 172.21.15.82:0/3691236415 learned_addr learned my addr 172.21.15.82:0/3691236415 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc2ffd640 1 --1- 172.21.15.82:0/3691236415 >> v1:172.21.15.164:6789/0 conn(0x7f9fc40b5bb0 0x7f9fc4161a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:33886/0 (socket says 172.21.15.82:33886) 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc3fff640 1 -- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc41551d0 msgr2=0x7f9fc40b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc3fff640 1 --2- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc41551d0 0x7f9fc40b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc3fff640 1 -- 172.21.15.82:0/3691236415 >> v1:172.21.15.164:6789/0 conn(0x7f9fc40b5bb0 legacy=0x7f9fc4161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc3fff640 1 -- 172.21.15.82:0/3691236415 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40b66b0 con 0x7f9fc4162190 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc27fc640 1 -- 172.21.15.82:0/3691236415 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22008178 0 0) 0x7f9fc40b60f0 con 0x7f9fc40b5bb0 2024-03-14T12:17:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.825+0000 7f9fc37fe640 1 --2- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc41551d0 0x7f9fc40b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:17:56.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.826+0000 7f9fc3fff640 1 --2- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc4162190 0x7f9fc406f330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009d90 tx=0x7f9fbc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e196f19973479fc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.826+0000 7f9fc27fc640 1 -- 172.21.15.82:0/3691236415 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc039070 con 0x7f9fc4162190 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.826+0000 7f9fc27fc640 1 -- 172.21.15.82:0/3691236415 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc02cc50 con 0x7f9fc4162190 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.826+0000 7f9fc27fc640 1 -- 172.21.15.82:0/3691236415 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc034440 con 0x7f9fc4162190 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 -- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc4162190 msgr2=0x7f9fc406f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc4162190 0x7f9fc406f330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009d90 tx=0x7f9fbc02bea0 comp rx=0 tx=0).stop 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 -- 172.21.15.82:0/3691236415 shutdown_connections 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc4162190 0x7f9fc406f330 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/3691236415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc41551d0 0x7f9fc40b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 -- 172.21.15.82:0/3691236415 >> 172.21.15.82:0/3691236415 conn(0x7f9fc4072ee0 msgr2=0x7f9fc409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 -- 172.21.15.82:0/3691236415 shutdown_connections 2024-03-14T12:17:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.827+0000 7f9fc9c59640 1 -- 172.21.15.82:0/3691236415 wait complete. 2024-03-14T12:17:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.828+0000 7f9fc9c59640 1 Processor -- start 2024-03-14T12:17:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.828+0000 7f9fc9c59640 1 -- start start 2024-03-14T12:17:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc9c59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc40b5bb0 0x7f9fc4152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc9c59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc41551d0 0x7f9fc406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc9c59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc4162190 0x7f9fc406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc9c59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc406f9f0 con 0x7f9fc41551d0 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc9c59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc406f870 con 0x7f9fc40b5bb0 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc9c59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc406fb70 con 0x7f9fc4162190 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc41551d0 0x7f9fc406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc41551d0 0x7f9fc406c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49864/0 (socket says 172.21.15.82:49864) 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 -- 172.21.15.82:0/162435398 learned_addr learned my addr 172.21.15.82:0/162435398 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc37fe640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc40b5bb0 0x7f9fc4152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc3fff640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc4162190 0x7f9fc406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 -- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc4162190 msgr2=0x7f9fc406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc4162190 0x7f9fc406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 -- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc40b5bb0 msgr2=0x7f9fc4152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc40b5bb0 0x7f9fc4152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 -- 172.21.15.82:0/162435398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc41f50c0 con 0x7f9fc41551d0 2024-03-14T12:17:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.829+0000 7f9fc2ffd640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc41551d0 0x7f9fc406c2a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800c4d0 tx=0x7f9fb800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.830+0000 7f9fabfff640 1 -- 172.21.15.82:0/162435398 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8018070 con 0x7f9fc41551d0 2024-03-14T12:17:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.830+0000 7f9fabfff640 1 -- 172.21.15.82:0/162435398 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8007500 con 0x7f9fc41551d0 2024-03-14T12:17:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.830+0000 7f9fabfff640 1 -- 172.21.15.82:0/162435398 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8004a20 con 0x7f9fc41551d0 2024-03-14T12:17:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.830+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc41f4de0 con 0x7f9fc41551d0 2024-03-14T12:17:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.830+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc40cb1f0 con 0x7f9fc41551d0 2024-03-14T12:17:56.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.831+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc404f050 con 0x7f9fc41551d0 2024-03-14T12:17:56.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.831+0000 7f9fabfff640 1 -- 172.21.15.82:0/162435398 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9fb8004bc0 con 0x7f9fc41551d0 2024-03-14T12:17:56.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.831+0000 7f9fabfff640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fb006bbd0 0x7f9fb006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:56.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.832+0000 7f9fabfff640 1 -- 172.21.15.82:0/162435398 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9fb8089c80 con 0x7f9fc41551d0 2024-03-14T12:17:56.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.832+0000 7f9fc37fe640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fb006bbd0 0x7f9fb006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:56.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.832+0000 7f9fc37fe640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fb006bbd0 0x7f9fb006e090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009630 tx=0x7f9fb4009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:56.836+0000 7f9fabfff640 1 -- 172.21.15.82:0/162435398 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9fb8058120 con 0x7f9fc41551d0 2024-03-14T12:17:57.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.045+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fc409b3a0 con 0x7f9fc41551d0 2024-03-14T12:17:57.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.045+0000 7f9fabfff640 1 -- 172.21.15.82:0/162435398 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f9fb805c1c0 con 0x7f9fc41551d0 2024-03-14T12:17:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:57.045 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:57.045 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fb006bbd0 msgr2=0x7f9fb006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fb006bbd0 0x7f9fb006e090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009630 tx=0x7f9fb4009240 comp rx=0 tx=0).stop 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc41551d0 msgr2=0x7f9fc406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc41551d0 0x7f9fc406c2a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800c4d0 tx=0x7f9fb800c990 comp rx=0 tx=0).stop 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 shutdown_connections 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9fb006bbd0 0x7f9fb006e090 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fc4162190 0x7f9fc406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc41551d0 0x7f9fc406c2a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 --2- 172.21.15.82:0/162435398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc40b5bb0 0x7f9fc4152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 >> 172.21.15.82:0/162435398 conn(0x7f9fc4072ee0 msgr2=0x7f9fc406fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 shutdown_connections 2024-03-14T12:17:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:57.047+0000 7f9fc9c59640 1 -- 172.21.15.82:0/162435398 wait complete. 2024-03-14T12:17:57.057 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:58.059 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:58.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.170+0000 7fcd4bda9640 1 Processor -- start 2024-03-14T12:17:58.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.171+0000 7fcd4bda9640 1 -- start start 2024-03-14T12:17:58.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.171+0000 7fcd4bda9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcd440cf370 0x7fcd440d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:58.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.171+0000 7fcd4bda9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd440b7720 0x7fcd440b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:58.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.171+0000 7fcd4bda9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440cf8c0 con 0x7fcd440d0760 2024-03-14T12:17:58.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.171+0000 7fcd4bda9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd440cfba0 con 0x7fcd440b7720 2024-03-14T12:17:58.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.171+0000 7fcd4bda9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd440b6c40 con 0x7fcd440cf370 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd4a346640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd440b7720 0x7fcd440b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd4a346640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd440b7720 0x7fcd440b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49870/0 (socket says 172.21.15.82:49870) 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd4a346640 1 -- 172.21.15.82:0/3792002653 learned_addr learned my addr 172.21.15.82:0/3792002653 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd49344640 1 --2- 172.21.15.82:0/3792002653 >> v2:172.21.15.164:3300/0 conn(0x7fcd440cf370 0x7fcd440d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd4a346640 1 -- 172.21.15.82:0/3792002653 >> v2:172.21.15.164:3300/0 conn(0x7fcd440cf370 msgr2=0x7fcd440d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd4a346640 1 --2- 172.21.15.82:0/3792002653 >> v2:172.21.15.164:3300/0 conn(0x7fcd440cf370 0x7fcd440d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd4a346640 1 -- 172.21.15.82:0/3792002653 >> v1:172.21.15.164:6789/0 conn(0x7fcd440d0760 legacy=0x7fcd440d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:17:58.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.172+0000 7fcd4a346640 1 -- 172.21.15.82:0/3792002653 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440b6f20 con 0x7fcd440b7720 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4a346640 1 --2- 172.21.15.82:0/3792002653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440b7b70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e50 tx=0x7fcd3c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95298f09623799cb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd48b43640 1 -- 172.21.15.82:0/3792002653 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c02cb20 con 0x7fcd440b7720 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd48b43640 1 -- 172.21.15.82:0/3792002653 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c02ccc0 con 0x7fcd440b7720 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd48b43640 1 -- 172.21.15.82:0/3792002653 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c034610 con 0x7fcd440b7720 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 -- 172.21.15.82:0/3792002653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 msgr2=0x7fcd440b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/3792002653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440b7b70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e50 tx=0x7fcd3c02bf80 comp rx=0 tx=0).stop 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 -- 172.21.15.82:0/3792002653 shutdown_connections 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/3792002653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440b7b70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/3792002653 >> v2:172.21.15.164:3300/0 conn(0x7fcd440cf370 0x7fcd440d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 -- 172.21.15.82:0/3792002653 >> 172.21.15.82:0/3792002653 conn(0x7fcd4409d030 msgr2=0x7fcd440d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:58.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 -- 172.21.15.82:0/3792002653 shutdown_connections 2024-03-14T12:17:58.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.173+0000 7fcd4bda9640 1 -- 172.21.15.82:0/3792002653 wait complete. 2024-03-14T12:17:58.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 Processor -- start 2024-03-14T12:17:58.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 -- start start 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd440cf370 0x7fcd440a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd440d0760 0x7fcd441f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd441617a0 con 0x7fcd440b7720 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440d4ba0 con 0x7fcd440cf370 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd4bda9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44161920 con 0x7fcd440d0760 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd49b45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd49b45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49880/0 (socket says 172.21.15.82:49880) 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.175+0000 7fcd49b45640 1 -- 172.21.15.82:0/156395306 learned_addr learned my addr 172.21.15.82:0/156395306 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49344640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd440cf370 0x7fcd440a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49b45640 1 -- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd440d0760 msgr2=0x7fcd441f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:58.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49b45640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd440d0760 0x7fcd441f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49b45640 1 -- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd440cf370 msgr2=0x7fcd440a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49b45640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd440cf370 0x7fcd440a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49b45640 1 -- 172.21.15.82:0/156395306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440a0f40 con 0x7fcd440b7720 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49344640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd440cf370 0x7fcd440a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd49b45640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440a0170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000b3f0 tx=0x7fcd4000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd3a7fc640 1 -- 172.21.15.82:0/156395306 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40015020 con 0x7fcd440b7720 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd3a7fc640 1 -- 172.21.15.82:0/156395306 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40011830 con 0x7fcd440b7720 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd440a1220 con 0x7fcd440b7720 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd3a7fc640 1 -- 172.21.15.82:0/156395306 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40011c20 con 0x7fcd440b7720 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.176+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd440a17e0 con 0x7fcd440b7720 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.177+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4404f050 con 0x7fcd440b7720 2024-03-14T12:17:58.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.178+0000 7fcd3a7fc640 1 -- 172.21.15.82:0/156395306 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcd40018020 con 0x7fcd440b7720 2024-03-14T12:17:58.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.178+0000 7fcd3a7fc640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2806bbd0 0x7fcd2806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:58.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.178+0000 7fcd3a7fc640 1 -- 172.21.15.82:0/156395306 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcd4008a6a0 con 0x7fcd440b7720 2024-03-14T12:17:58.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.178+0000 7fcd49344640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2806bbd0 0x7fcd2806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:58.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.178+0000 7fcd49344640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2806bbd0 0x7fcd2806e090 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd440ca830 tx=0x7fcd34008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:58.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.182+0000 7fcd3a7fc640 1 -- 172.21.15.82:0/156395306 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fcd4001f030 con 0x7fcd440b7720 2024-03-14T12:17:58.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.391+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd440d0540 con 0x7fcd440b7720 2024-03-14T12:17:58.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.391+0000 7fcd3a7fc640 1 -- 172.21.15.82:0/156395306 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fcd400172c0 con 0x7fcd440b7720 2024-03-14T12:17:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:58.391 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:58.391 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2806bbd0 msgr2=0x7fcd2806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2806bbd0 0x7fcd2806e090 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd440ca830 tx=0x7fcd34008040 comp rx=0 tx=0).stop 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 msgr2=0x7fcd440a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440a0170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000b3f0 tx=0x7fcd4000b8b0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 shutdown_connections 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcd2806bbd0 0x7fcd2806e090 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd440d0760 0x7fcd441f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd440cf370 0x7fcd440a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 --2- 172.21.15.82:0/156395306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd440b7720 0x7fcd440a0170 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 >> 172.21.15.82:0/156395306 conn(0x7fcd4409d030 msgr2=0x7fcd440d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 shutdown_connections 2024-03-14T12:17:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:58.393+0000 7fcd4bda9640 1 -- 172.21.15.82:0/156395306 wait complete. 2024-03-14T12:17:58.403 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:17:59.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:17:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.517+0000 7feb9b28c640 1 Processor -- start 2024-03-14T12:17:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.517+0000 7feb9b28c640 1 -- start start 2024-03-14T12:17:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.518+0000 7feb9b28c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb94161a80 0x7feb94161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.518+0000 7feb9b28c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940b60f0 con 0x7feb941551d0 2024-03-14T12:17:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.518+0000 7feb9b28c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940b63d0 con 0x7feb9406f330 2024-03-14T12:17:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.518+0000 7feb9b28c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb940b6d00 con 0x7feb94161a80 2024-03-14T12:17:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.518+0000 7feb99028640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feb941551d0 0x7feb940b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38202/0 (socket says 172.21.15.82:38202) 2024-03-14T12:17:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.518+0000 7feb99028640 1 -- 172.21.15.82:0/407994671 learned_addr learned my addr 172.21.15.82:0/407994671 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:17:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.519+0000 7feb98827640 1 --2- 172.21.15.82:0/407994671 >> v2:172.21.15.164:3300/0 conn(0x7feb94161a80 0x7feb94161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.519+0000 7feb8bfff640 1 -- 172.21.15.82:0/407994671 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599496545 0 0) 0x7feb940b60f0 con 0x7feb941551d0 2024-03-14T12:17:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.519+0000 7feb8bfff640 1 -- 172.21.15.82:0/407994671 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb80003520 con 0x7feb941551d0 2024-03-14T12:17:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.519+0000 7feb8bfff640 1 -- 172.21.15.82:0/407994671 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153268561 0 0) 0x7feb80003520 con 0x7feb941551d0 2024-03-14T12:17:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.519+0000 7feb98827640 1 -- 172.21.15.82:0/407994671 >> v1:172.21.15.164:6789/0 conn(0x7feb9406f330 legacy=0x7feb9406f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.519+0000 7feb98827640 1 -- 172.21.15.82:0/407994671 >> v1:172.21.15.82:6789/0 conn(0x7feb941551d0 legacy=0x7feb940b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.519+0000 7feb98827640 1 -- 172.21.15.82:0/407994671 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940b66b0 con 0x7feb94161a80 2024-03-14T12:17:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.520+0000 7feb98827640 1 --2- 172.21.15.82:0/407994671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb94161a80 0x7feb94161ee0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb8402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4c808327e23a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.520+0000 7feb8bfff640 1 -- 172.21.15.82:0/407994671 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84038070 con 0x7feb94161a80 2024-03-14T12:17:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.520+0000 7feb8bfff640 1 -- 172.21.15.82:0/407994671 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84031500 con 0x7feb94161a80 2024-03-14T12:17:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.520+0000 7feb8bfff640 1 -- 172.21.15.82:0/407994671 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb900019f0 con 0x7feb94161a80 2024-03-14T12:17:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.521+0000 7feb9b28c640 1 -- 172.21.15.82:0/407994671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb94161a80 msgr2=0x7feb94161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.521+0000 7feb9b28c640 1 --2- 172.21.15.82:0/407994671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb94161a80 0x7feb94161ee0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb8402b820 comp rx=0 tx=0).stop 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.521+0000 7feb9b28c640 1 -- 172.21.15.82:0/407994671 shutdown_connections 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.521+0000 7feb9b28c640 1 --2- 172.21.15.82:0/407994671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb94161a80 0x7feb94161ee0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.521+0000 7feb9b28c640 1 -- 172.21.15.82:0/407994671 >> 172.21.15.82:0/407994671 conn(0x7feb94072ee0 msgr2=0x7feb9409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.521+0000 7feb9b28c640 1 -- 172.21.15.82:0/407994671 shutdown_connections 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.521+0000 7feb9b28c640 1 -- 172.21.15.82:0/407994671 wait complete. 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.522+0000 7feb9b28c640 1 Processor -- start 2024-03-14T12:17:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.522+0000 7feb9b28c640 1 -- start start 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb9b28c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb9406f330 0x7feb941f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb9b28c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb941551d0 0x7feb941f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb9b28c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb94161a80 0x7feb9414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb9b28c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940b5d80 con 0x7feb94161a80 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb9b28c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb94160b40 con 0x7feb941551d0 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb9b28c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb94160cc0 con 0x7feb9406f330 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb94161a80 0x7feb9414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb94161a80 0x7feb9414d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49896/0 (socket says 172.21.15.82:49896) 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 -- 172.21.15.82:0/1123462595 learned_addr learned my addr 172.21.15.82:0/1123462595 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:17:59.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb98827640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb941551d0 0x7feb941f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 -- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb9406f330 msgr2=0x7feb941f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99028640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb9406f330 0x7feb941f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb9406f330 0x7feb941f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 -- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb941551d0 msgr2=0x7feb941f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb941551d0 0x7feb941f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 -- 172.21.15.82:0/1123462595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940b1900 con 0x7feb94161a80 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb98827640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb941551d0 0x7feb941f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb99829640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb94161a80 0x7feb9414d540 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00c400 tx=0x7feb8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb89ffb640 1 -- 172.21.15.82:0/1123462595 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c017070 con 0x7feb94161a80 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb89ffb640 1 -- 172.21.15.82:0/1123462595 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8c0052d0 con 0x7feb94161a80 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb94150000 con 0x7feb94161a80 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.523+0000 7feb89ffb640 1 -- 172.21.15.82:0/1123462595 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c003810 con 0x7feb94161a80 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.524+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb940b1ba0 con 0x7feb94161a80 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.525+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9404f050 con 0x7feb94161a80 2024-03-14T12:17:59.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.525+0000 7feb89ffb640 1 -- 172.21.15.82:0/1123462595 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7feb8c01a070 con 0x7feb94161a80 2024-03-14T12:17:59.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.525+0000 7feb89ffb640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:17:59.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.525+0000 7feb89ffb640 1 -- 172.21.15.82:0/1123462595 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7feb8c089e90 con 0x7feb94161a80 2024-03-14T12:17:59.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.525+0000 7feb99028640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:17:59.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.526+0000 7feb99028640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feb90005c00 tx=0x7feb90009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:17:59.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.530+0000 7feb89ffb640 1 -- 172.21.15.82:0/1123462595 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7feb8c0588f0 con 0x7feb94161a80 2024-03-14T12:17:59.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.739+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb940b6ff0 con 0x7feb94161a80 2024-03-14T12:17:59.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.739+0000 7feb89ffb640 1 -- 172.21.15.82:0/1123462595 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7feb8c0581f0 con 0x7feb94161a80 2024-03-14T12:17:59.739 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:17:59.739 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:17:59.739 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 msgr2=0x7feb7806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feb90005c00 tx=0x7feb90009040 comp rx=0 tx=0).stop 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb94161a80 msgr2=0x7feb9414d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb94161a80 0x7feb9414d540 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00c400 tx=0x7feb8c00c8c0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 shutdown_connections 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb94161a80 0x7feb9414d540 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb941551d0 0x7feb941f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7feb7806bbd0 0x7feb7806e090 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 --2- 172.21.15.82:0/1123462595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb9406f330 0x7feb941f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:17:59.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.741+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 >> 172.21.15.82:0/1123462595 conn(0x7feb94072ee0 msgr2=0x7feb94154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:17:59.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.742+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 shutdown_connections 2024-03-14T12:17:59.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:17:59.742+0000 7feb9b28c640 1 -- 172.21.15.82:0/1123462595 wait complete. 2024-03-14T12:17:59.752 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:18:00.753 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.866+0000 7f8db4803640 1 Processor -- start 2024-03-14T12:18:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.866+0000 7f8db4803640 1 -- start start 2024-03-14T12:18:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.867+0000 7f8db4803640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8db00d1fe0 0x7f8db00d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.867+0000 7f8db4803640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00cfab0 con 0x7f8db015df30 2024-03-14T12:18:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.867+0000 7f8db4803640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00cea10 con 0x7f8db00b0690 2024-03-14T12:18:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.867+0000 7f8db4803640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db00d2980 con 0x7f8db00d1fe0 2024-03-14T12:18:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.867+0000 7f8dae59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8db00d1fe0 0x7f8db00d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.867+0000 7f8daed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8db015df30 0x7f8db00d4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38208/0 (socket says 172.21.15.82:38208) 2024-03-14T12:18:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.867+0000 7f8daed9d640 1 -- 172.21.15.82:0/2538183596 learned_addr learned my addr 172.21.15.82:0/2538183596 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.868+0000 7f8dae59c640 1 -- 172.21.15.82:0/2538183596 >> v1:172.21.15.164:6789/0 conn(0x7f8db00b0690 legacy=0x7f8db00b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.868+0000 7f8dae59c640 1 -- 172.21.15.82:0/2538183596 >> v1:172.21.15.82:6789/0 conn(0x7f8db015df30 legacy=0x7f8db00d4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.868+0000 7f8dadd9b640 1 -- 172.21.15.82:0/2538183596 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171821986 0 0) 0x7f8da0001110 con 0x7f8db015df30 2024-03-14T12:18:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.868+0000 7f8dae59c640 1 -- 172.21.15.82:0/2538183596 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00cecf0 con 0x7f8db00d1fe0 2024-03-14T12:18:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.868+0000 7f8dae59c640 1 --2- 172.21.15.82:0/2538183596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db00d1fe0 0x7f8db00d2440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097a0 tx=0x7f8da402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fcefa197c0b283 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.868+0000 7f8dadd9b640 1 -- 172.21.15.82:0/2538183596 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4035020 con 0x7f8db00d1fe0 2024-03-14T12:18:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8dadd9b640 1 -- 172.21.15.82:0/2538183596 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4031830 con 0x7f8db00d1fe0 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8dadd9b640 1 -- 172.21.15.82:0/2538183596 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4031c00 con 0x7f8db00d1fe0 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8db4803640 1 -- 172.21.15.82:0/2538183596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db00d1fe0 msgr2=0x7f8db00d2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8db4803640 1 --2- 172.21.15.82:0/2538183596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db00d1fe0 0x7f8db00d2440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097a0 tx=0x7f8da402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8db4803640 1 -- 172.21.15.82:0/2538183596 shutdown_connections 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8db4803640 1 --2- 172.21.15.82:0/2538183596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db00d1fe0 0x7f8db00d2440 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8db4803640 1 -- 172.21.15.82:0/2538183596 >> 172.21.15.82:0/2538183596 conn(0x7f8db009cea0 msgr2=0x7f8db00d3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8db4803640 1 -- 172.21.15.82:0/2538183596 shutdown_connections 2024-03-14T12:18:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.869+0000 7f8db4803640 1 -- 172.21.15.82:0/2538183596 wait complete. 2024-03-14T12:18:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 Processor -- start 2024-03-14T12:18:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 -- start start 2024-03-14T12:18:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8db00b0690 0x7f8db0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db00d1fe0 0x7f8db015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db015df30 0x7f8db00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00cf2b0 con 0x7f8db015df30 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00cf620 con 0x7f8db00d1fe0 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.871+0000 7f8db4803640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00ce6c0 con 0x7f8db00b0690 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db015df30 0x7f8db00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db015df30 0x7f8db00b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49918/0 (socket says 172.21.15.82:49918) 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 -- 172.21.15.82:0/1146905536 learned_addr learned my addr 172.21.15.82:0/1146905536 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daed9d640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8db00b0690 0x7f8db0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8dae59c640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db00d1fe0 0x7f8db015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 -- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8db00b0690 msgr2=0x7f8db0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8db00b0690 0x7f8db0152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 -- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db00d1fe0 msgr2=0x7f8db015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db00d1fe0 0x7f8db015e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 -- 172.21.15.82:0/1146905536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db015ed40 con 0x7f8db015df30 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8dae59c640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db00d1fe0 0x7f8db015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8daf59e640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db015df30 0x7f8db00b3c50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c780 tx=0x7f8da800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8d977fe640 1 -- 172.21.15.82:0/1146905536 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8004b60 con 0x7f8db015df30 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8d977fe640 1 -- 172.21.15.82:0/1146905536 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da8004d00 con 0x7f8db015df30 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db015f020 con 0x7f8db015df30 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8d977fe640 1 -- 172.21.15.82:0/1146905536 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800aa90 con 0x7f8db015df30 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.872+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db015f5e0 con 0x7f8db015df30 2024-03-14T12:18:00.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.873+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db0075210 con 0x7f8db015df30 2024-03-14T12:18:00.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.874+0000 7f8d977fe640 1 -- 172.21.15.82:0/1146905536 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8da800ac30 con 0x7f8db015df30 2024-03-14T12:18:00.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.874+0000 7f8d977fe640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8d9c06bbd0 0x7f8d9c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:00.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.874+0000 7f8daed9d640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8d9c06bbd0 0x7f8d9c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:00.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.874+0000 7f8d977fe640 1 -- 172.21.15.82:0/1146905536 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8da808adb0 con 0x7f8db015df30 2024-03-14T12:18:00.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.875+0000 7f8daed9d640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8d9c06bbd0 0x7f8d9c06e090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000f60 tx=0x7f8da0007c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:00.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:00.879+0000 7f8d977fe640 1 -- 172.21.15.82:0/1146905536 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8da8020070 con 0x7f8db015df30 2024-03-14T12:18:01.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.087+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db015fa70 con 0x7f8db015df30 2024-03-14T12:18:01.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.087+0000 7f8d977fe640 1 -- 172.21.15.82:0/1146905536 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f8da8059810 con 0x7f8db015df30 2024-03-14T12:18:01.087 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:01.087 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:01.087 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:18:01.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8d9c06bbd0 msgr2=0x7f8d9c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:01.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8d9c06bbd0 0x7f8d9c06e090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000f60 tx=0x7f8da0007c00 comp rx=0 tx=0).stop 2024-03-14T12:18:01.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db015df30 msgr2=0x7f8db00b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:01.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db015df30 0x7f8db00b3c50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c780 tx=0x7f8da800cc40 comp rx=0 tx=0).stop 2024-03-14T12:18:01.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 shutdown_connections 2024-03-14T12:18:01.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8db015df30 0x7f8db00b3c50 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:01.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db00d1fe0 0x7f8db015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:01.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f8d9c06bbd0 0x7f8d9c06e090 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:01.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.089+0000 7f8db4803640 1 --2- 172.21.15.82:0/1146905536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8db00b0690 0x7f8db0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:01.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.090+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 >> 172.21.15.82:0/1146905536 conn(0x7f8db009cea0 msgr2=0x7f8db015bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:01.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.090+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 shutdown_connections 2024-03-14T12:18:01.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:01.090+0000 7f8db4803640 1 -- 172.21.15.82:0/1146905536 wait complete. 2024-03-14T12:18:01.100 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:18:02.101 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:02.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.213+0000 7fe663bdf640 1 Processor -- start 2024-03-14T12:18:02.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.213+0000 7fe663bdf640 1 -- start start 2024-03-14T12:18:02.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.214+0000 7fe663bdf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c06b670 0x7fe65c072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:02.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.214+0000 7fe663bdf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe65c06eb20 0x7fe65c06efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:02.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.214+0000 7fe663bdf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65c0727a0 con 0x7fe65c0b0200 2024-03-14T12:18:02.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.214+0000 7fe663bdf640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c071810 con 0x7fe65c06eb20 2024-03-14T12:18:02.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.214+0000 7fe663bdf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c071dd0 con 0x7fe65c06b670 2024-03-14T12:18:02.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66197b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c06b670 0x7fe65c072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:02.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66217c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe65c06eb20 0x7fe65c06efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:02.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66197b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c06b670 0x7fe65c072260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:37742/0 (socket says 172.21.15.82:37742) 2024-03-14T12:18:02.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66217c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe65c06eb20 0x7fe65c06efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:36914/0 (socket says 172.21.15.82:36914) 2024-03-14T12:18:02.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66197b640 1 -- 172.21.15.82:0/1941332503 learned_addr learned my addr 172.21.15.82:0/1941332503 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:02.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66117a640 1 --1- 172.21.15.82:0/1941332503 >> v1:172.21.15.164:6789/0 conn(0x7fe65c0b0200 0x7fe65c0ce730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:50910/0 (socket says 172.21.15.82:50910) 2024-03-14T12:18:02.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe660979640 1 -- 172.21.15.82:0/1941332503 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578168598 0 0) 0x7fe65c0727a0 con 0x7fe65c0b0200 2024-03-14T12:18:02.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe660979640 1 -- 172.21.15.82:0/1941332503 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe648003520 con 0x7fe65c0b0200 2024-03-14T12:18:02.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66217c640 1 -- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c06b670 msgr2=0x7fe65c072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66217c640 1 --2- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c06b670 0x7fe65c072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66217c640 1 -- 172.21.15.82:0/1941332503 >> v1:172.21.15.164:6789/0 conn(0x7fe65c0b0200 legacy=0x7fe65c0ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66217c640 1 -- 172.21.15.82:0/1941332503 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0d0410 con 0x7fe65c06eb20 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.215+0000 7fe66197b640 1 --2- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c06b670 0x7fe65c072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.216+0000 7fe66217c640 1 --2- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06eb20 0x7fe65c06efd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe654009d90 tx=0x7fe65402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b701e5a45f1257f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.216+0000 7fe660979640 1 -- 172.21.15.82:0/1941332503 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654039070 con 0x7fe65c06eb20 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.216+0000 7fe660979640 1 -- 172.21.15.82:0/1941332503 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65402cc50 con 0x7fe65c06eb20 2024-03-14T12:18:02.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.216+0000 7fe660979640 1 -- 172.21.15.82:0/1941332503 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654034440 con 0x7fe65c06eb20 2024-03-14T12:18:02.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 -- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06eb20 msgr2=0x7fe65c06efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:02.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 --2- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06eb20 0x7fe65c06efd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe654009d90 tx=0x7fe65402bea0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 -- 172.21.15.82:0/1941332503 shutdown_connections 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 --2- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06eb20 0x7fe65c06efd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 --2- 172.21.15.82:0/1941332503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c06b670 0x7fe65c072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 -- 172.21.15.82:0/1941332503 >> 172.21.15.82:0/1941332503 conn(0x7fe65c0d0b60 msgr2=0x7fe65c0d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 -- 172.21.15.82:0/1941332503 shutdown_connections 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.217+0000 7fe663bdf640 1 -- 172.21.15.82:0/1941332503 wait complete. 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.218+0000 7fe663bdf640 1 Processor -- start 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.218+0000 7fe663bdf640 1 -- start start 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe663bdf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06b670 0x7fe65c1f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe663bdf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe65c06eb20 0x7fe65c0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe663bdf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c0b0200 0x7fe65c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe663bdf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0d0100 con 0x7fe65c06eb20 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe663bdf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c071a60 con 0x7fe65c06b670 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe663bdf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0cb1b0 con 0x7fe65c0b0200 2024-03-14T12:18:02.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe65c06eb20 0x7fe65c0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe65c06eb20 0x7fe65c0b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35834/0 (socket says 172.21.15.82:35834) 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 -- 172.21.15.82:0/226230461 learned_addr learned my addr 172.21.15.82:0/226230461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66197b640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06b670 0x7fe65c1f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66217c640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c0b0200 0x7fe65c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 -- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c0b0200 msgr2=0x7fe65c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c0b0200 0x7fe65c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 -- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06b670 msgr2=0x7fe65c1f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06b670 0x7fe65c1f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 -- 172.21.15.82:0/226230461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c1f3aa0 con 0x7fe65c06eb20 2024-03-14T12:18:02.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe66117a640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe65c06eb20 0x7fe65c0b5ce0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00c460 tx=0x7fe64c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:02.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.219+0000 7fe6527fc640 1 -- 172.21.15.82:0/226230461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c018070 con 0x7fe65c06eb20 2024-03-14T12:18:02.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.220+0000 7fe6527fc640 1 -- 172.21.15.82:0/226230461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c0074a0 con 0x7fe65c06eb20 2024-03-14T12:18:02.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.220+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c1f3d40 con 0x7fe65c06eb20 2024-03-14T12:18:02.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.220+0000 7fe6527fc640 1 -- 172.21.15.82:0/226230461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c004a20 con 0x7fe65c06eb20 2024-03-14T12:18:02.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.220+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c1f4300 con 0x7fe65c06eb20 2024-03-14T12:18:02.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.220+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c04f050 con 0x7fe65c06eb20 2024-03-14T12:18:02.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.221+0000 7fe6527fc640 1 -- 172.21.15.82:0/226230461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe64c004bc0 con 0x7fe65c06eb20 2024-03-14T12:18:02.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.221+0000 7fe6527fc640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe64006bc00 0x7fe64006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:02.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.221+0000 7fe6527fc640 1 -- 172.21.15.82:0/226230461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe64c089af0 con 0x7fe65c06eb20 2024-03-14T12:18:02.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.221+0000 7fe66197b640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe64006bc00 0x7fe64006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:02.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.222+0000 7fe66197b640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe64006bc00 0x7fe64006e0c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe658006ca0 tx=0x7fe658009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:02.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.226+0000 7fe6527fc640 1 -- 172.21.15.82:0/226230461 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe64c007650 con 0x7fe65c06eb20 2024-03-14T12:18:02.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.435+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe65c0b5ac0 con 0x7fe65c06eb20 2024-03-14T12:18:02.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.436+0000 7fe6527fc640 1 -- 172.21.15.82:0/226230461 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fe64c057f90 con 0x7fe65c06eb20 2024-03-14T12:18:02.436 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:02.436 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:18:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe64006bc00 msgr2=0x7fe64006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe64006bc00 0x7fe64006e0c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe658006ca0 tx=0x7fe658009240 comp rx=0 tx=0).stop 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe65c06eb20 msgr2=0x7fe65c0b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe65c06eb20 0x7fe65c0b5ce0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00c460 tx=0x7fe64c00c920 comp rx=0 tx=0).stop 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 shutdown_connections 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe64006bc00 0x7fe64006e0c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe65c0b0200 0x7fe65c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe65c06eb20 0x7fe65c0b5ce0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 --2- 172.21.15.82:0/226230461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c06b670 0x7fe65c1f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 >> 172.21.15.82:0/226230461 conn(0x7fe65c0d0b60 msgr2=0x7fe65c0afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 shutdown_connections 2024-03-14T12:18:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:02.438+0000 7fe663bdf640 1 -- 172.21.15.82:0/226230461 wait complete. 2024-03-14T12:18:02.448 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:18:03.450 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:03.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.563+0000 7f9d31737640 1 Processor -- start 2024-03-14T12:18:03.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.563+0000 7f9d31737640 1 -- start start 2024-03-14T12:18:03.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d31737640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0cf370 0x7f9d2c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d31737640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9d2c0b7720 0x7f9d2c0b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d31737640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0cf8c0 con 0x7f9d2c0d0760 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d31737640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c0cfba0 con 0x7f9d2c0b7720 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d31737640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b6c40 con 0x7f9d2c0cf370 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d2b7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9d2c0b7720 0x7f9d2c0b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d23fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0cf370 0x7f9d2c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d2affd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9d2c0d0760 0x7f9d2c0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:50926/0 (socket says 172.21.15.82:50926) 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d2b7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9d2c0b7720 0x7f9d2c0b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:36918/0 (socket says 172.21.15.82:36918) 2024-03-14T12:18:03.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.564+0000 7f9d2affd640 1 -- 172.21.15.82:0/2858768411 learned_addr learned my addr 172.21.15.82:0/2858768411 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.565+0000 7f9d2a7fc640 1 -- 172.21.15.82:0/2858768411 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340796046 0 0) 0x7f9d2c0cf8c0 con 0x7f9d2c0d0760 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.565+0000 7f9d2a7fc640 1 -- 172.21.15.82:0/2858768411 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d18003520 con 0x7f9d2c0d0760 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.565+0000 7f9d23fff640 1 -- 172.21.15.82:0/2858768411 >> v2:172.21.15.164:3300/0 conn(0x7f9d2c0b7720 msgr2=0x7f9d2c0b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.565+0000 7f9d23fff640 1 --2- 172.21.15.82:0/2858768411 >> v2:172.21.15.164:3300/0 conn(0x7f9d2c0b7720 0x7f9d2c0b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.565+0000 7f9d23fff640 1 -- 172.21.15.82:0/2858768411 >> v1:172.21.15.164:6789/0 conn(0x7f9d2c0d0760 legacy=0x7f9d2c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.565+0000 7f9d23fff640 1 -- 172.21.15.82:0/2858768411 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0b6f20 con 0x7f9d2c0cf370 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.565+0000 7f9d2b7fe640 1 --2- 172.21.15.82:0/2858768411 >> v2:172.21.15.164:3300/0 conn(0x7f9d2c0b7720 0x7f9d2c0b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:18:03.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d23fff640 1 --2- 172.21.15.82:0/2858768411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0cf370 0x7f9d2c0d4660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d140097a0 tx=0x7f9d1402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba35237f048b9c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d2a7fc640 1 -- 172.21.15.82:0/2858768411 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14034070 con 0x7f9d2c0cf370 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d2a7fc640 1 -- 172.21.15.82:0/2858768411 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14030610 con 0x7f9d2c0cf370 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d2a7fc640 1 -- 172.21.15.82:0/2858768411 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d140309e0 con 0x7f9d2c0cf370 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 -- 172.21.15.82:0/2858768411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0cf370 msgr2=0x7f9d2c0d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 --2- 172.21.15.82:0/2858768411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0cf370 0x7f9d2c0d4660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d140097a0 tx=0x7f9d1402b960 comp rx=0 tx=0).stop 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 -- 172.21.15.82:0/2858768411 shutdown_connections 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 --2- 172.21.15.82:0/2858768411 >> v2:172.21.15.164:3300/0 conn(0x7f9d2c0b7720 0x7f9d2c0b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 --2- 172.21.15.82:0/2858768411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0cf370 0x7f9d2c0d4660 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 -- 172.21.15.82:0/2858768411 >> 172.21.15.82:0/2858768411 conn(0x7f9d2c09d030 msgr2=0x7f9d2c0d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 -- 172.21.15.82:0/2858768411 shutdown_connections 2024-03-14T12:18:03.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.566+0000 7f9d31737640 1 -- 172.21.15.82:0/2858768411 wait complete. 2024-03-14T12:18:03.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 Processor -- start 2024-03-14T12:18:03.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 -- start start 2024-03-14T12:18:03.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c0b7720 0x7f9d2c1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:03.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d2c0cf370 0x7f9d2c1f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:03.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0d0760 0x7f9d2c0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1617a0 con 0x7f9d2c0cf370 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0d4ba0 con 0x7f9d2c0b7720 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d31737640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c161920 con 0x7f9d2c0d0760 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d23fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d2c0cf370 0x7f9d2c1f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d23fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d2c0cf370 0x7f9d2c1f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35838/0 (socket says 172.21.15.82:35838) 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.568+0000 7f9d23fff640 1 -- 172.21.15.82:0/16798070 learned_addr learned my addr 172.21.15.82:0/16798070 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d2b7fe640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0d0760 0x7f9d2c0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d2affd640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c0b7720 0x7f9d2c1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d23fff640 1 -- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0d0760 msgr2=0x7f9d2c0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d23fff640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0d0760 0x7f9d2c0ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d23fff640 1 -- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c0b7720 msgr2=0x7f9d2c1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d23fff640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c0b7720 0x7f9d2c1550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d23fff640 1 -- 172.21.15.82:0/16798070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c154640 con 0x7f9d2c0cf370 2024-03-14T12:18:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d23fff640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d2c0cf370 0x7f9d2c1f6f80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9d1402bf70 tx=0x7f9d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d237fe640 1 -- 172.21.15.82:0/16798070 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14034040 con 0x7f9d2c0cf370 2024-03-14T12:18:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d237fe640 1 -- 172.21.15.82:0/16798070 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14030410 con 0x7f9d2c0cf370 2024-03-14T12:18:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d237fe640 1 -- 172.21.15.82:0/16798070 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14006e10 con 0x7f9d2c0cf370 2024-03-14T12:18:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c15c120 con 0x7f9d2c0cf370 2024-03-14T12:18:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.569+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c154b90 con 0x7f9d2c0cf370 2024-03-14T12:18:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.570+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c072dc0 con 0x7f9d2c0cf370 2024-03-14T12:18:03.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.571+0000 7f9d237fe640 1 -- 172.21.15.82:0/16798070 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9d14037070 con 0x7f9d2c0cf370 2024-03-14T12:18:03.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.571+0000 7f9d237fe640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d0c06bbd0 0x7f9d0c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:03.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.571+0000 7f9d237fe640 1 -- 172.21.15.82:0/16798070 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9d140aca10 con 0x7f9d2c0cf370 2024-03-14T12:18:03.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.571+0000 7f9d2affd640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d0c06bbd0 0x7f9d0c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:03.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.572+0000 7f9d2affd640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d0c06bbd0 0x7f9d0c06e090 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c000f60 tx=0x7f9d1c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:03.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.575+0000 7f9d237fe640 1 -- 172.21.15.82:0/16798070 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f9d2c072dc0 con 0x7f9d2c0cf370 2024-03-14T12:18:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.784+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d2c04f050 con 0x7f9d2c0cf370 2024-03-14T12:18:03.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.785+0000 7f9d237fe640 1 -- 172.21.15.82:0/16798070 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f9d1407b410 con 0x7f9d2c0cf370 2024-03-14T12:18:03.784 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:03.784 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:03.784 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:18:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.786+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d0c06bbd0 msgr2=0x7f9d0c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.786+0000 7f9d31737640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d0c06bbd0 0x7f9d0c06e090 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c000f60 tx=0x7f9d1c009040 comp rx=0 tx=0).stop 2024-03-14T12:18:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.786+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d2c0cf370 msgr2=0x7f9d2c1f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.786+0000 7f9d31737640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d2c0cf370 0x7f9d2c1f6f80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9d1402bf70 tx=0x7f9d140046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 shutdown_connections 2024-03-14T12:18:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9d0c06bbd0 0x7f9d0c06e090 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9d2c0d0760 0x7f9d2c0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d2c0cf370 0x7f9d2c1f6f80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 --2- 172.21.15.82:0/16798070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c0b7720 0x7f9d2c1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 >> 172.21.15.82:0/16798070 conn(0x7f9d2c09d030 msgr2=0x7f9d2c15c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 shutdown_connections 2024-03-14T12:18:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:03.787+0000 7f9d31737640 1 -- 172.21.15.82:0/16798070 wait complete. 2024-03-14T12:18:03.797 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:18:04.798 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.901+0000 7fe81bfff640 1 Processor -- start 2024-03-14T12:18:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.901+0000 7fe81bfff640 1 -- start start 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe81bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe81c0ce920 0x7fe81c0cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe81bfff640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c06d910 con 0x7fe81c0cf730 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe81bfff640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c0ced00 con 0x7fe81c0d2470 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe81bfff640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c06e020 con 0x7fe81c0ce920 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe81b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe81c0cf730 0x7fe81c15dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39072/0 (socket says 172.21.15.82:39072) 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3420169415 learned_addr learned my addr 172.21.15.82:0/3420169415 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097148116 0 0) 0x7fe814001110 con 0x7fe81c0cf730 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe804003520 con 0x7fe81c0cf730 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.902+0000 7fe81affd640 1 --2- 172.21.15.82:0/3420169415 >> v2:172.21.15.164:3300/0 conn(0x7fe81c0ce920 0x7fe81c0cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853742614 0 0) 0x7fe804003520 con 0x7fe81c0cf730 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 >> v2:172.21.15.164:3300/0 conn(0x7fe81c0ce920 msgr2=0x7fe81c0cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 --2- 172.21.15.82:0/3420169415 >> v2:172.21.15.164:3300/0 conn(0x7fe81c0ce920 0x7fe81c0cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 >> v1:172.21.15.164:6789/0 conn(0x7fe81c0d2470 legacy=0x7fe81c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c06c870 con 0x7fe81c0cf730 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fe814001840 con 0x7fe81c0cf730 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 >> v1:172.21.15.82:6789/0 conn(0x7fe81c0cf730 legacy=0x7fe81c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8040039b0 0x7fe804003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe804004800 0x7fe804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8040051c0 0x7fe80400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80400c030 con 0x7fe8040051c0 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80400c1e0 con 0x7fe804004800 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80400c3e0 con 0x7fe8040039b0 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81b7fe640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8040051c0 0x7fe80400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81a7fc640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe804004800 0x7fe804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81affd640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8040039b0 0x7fe804003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8040039b0 msgr2=0x7fe804003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81b7fe640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8040039b0 0x7fe804003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe804004800 msgr2=0x7fe804004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81b7fe640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe804004800 0x7fe804004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3420169415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804003520 con 0x7fe8040051c0 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.903+0000 7fe81affd640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8040039b0 0x7fe804003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.904+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3420169415 reap_dead start 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.904+0000 7fe81b7fe640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8040051c0 0x7fe80400b9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c4c0 tx=0x7fe804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d70f8ed720b35a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.904+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81400c0e0 con 0x7fe8040051c0 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.904+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814030070 con 0x7fe8040051c0 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.904+0000 7fe819ffb640 1 -- 172.21.15.82:0/3420169415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8140058d0 con 0x7fe8040051c0 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.904+0000 7fe81bfff640 1 -- 172.21.15.82:0/3420169415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8040051c0 msgr2=0x7fe80400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.905+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8040051c0 0x7fe80400b9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c4c0 tx=0x7fe804005830 comp rx=0 tx=0).stop 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.905+0000 7fe81bfff640 1 -- 172.21.15.82:0/3420169415 shutdown_connections 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.905+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3420169415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8040051c0 0x7fe80400b9c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.905+0000 7fe81bfff640 1 -- 172.21.15.82:0/3420169415 >> 172.21.15.82:0/3420169415 conn(0x7fe81c0d1950 msgr2=0x7fe81c0d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.905+0000 7fe81bfff640 1 -- 172.21.15.82:0/3420169415 shutdown_connections 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.905+0000 7fe81bfff640 1 -- 172.21.15.82:0/3420169415 wait complete. 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.906+0000 7fe81bfff640 1 Processor -- start 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.906+0000 7fe81bfff640 1 -- start start 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8040051c0 0x7fe81c14fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe81c0af0a0 0x7fe81c0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe81c0c7ac0 0x7fe81c0b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06d110 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06ce30 con 0x7fe8040051c0 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06d3f0 con 0x7fe81c0af0a0 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe81c0c7ac0 0x7fe81c0b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe81c0c7ac0 0x7fe81c0b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35864/0 (socket says 172.21.15.82:35864) 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3158980162 learned_addr learned my addr 172.21.15.82:0/3158980162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81a7fc640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe81c0af0a0 0x7fe81c0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81affd640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8040051c0 0x7fe81c14fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe81c0af0a0 msgr2=0x7fe81c0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe81c0af0a0 0x7fe81c0c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8040051c0 msgr2=0x7fe81c14fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8040051c0 0x7fe81c14fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 -- 172.21.15.82:0/3158980162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0b0b40 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81b7fe640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe81c0c7ac0 0x7fe81c0b3670 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe814000f60 tx=0x7fe814001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe8037fe640 1 -- 172.21.15.82:0/3158980162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814004bf0 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe8037fe640 1 -- 172.21.15.82:0/3158980162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814004d90 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c0b0e20 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe8037fe640 1 -- 172.21.15.82:0/3158980162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8140072b0 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.907+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c0b3bb0 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.908+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c0c7f40 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.909+0000 7fe8037fe640 1 -- 172.21.15.82:0/3158980162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe814007450 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.909+0000 7fe8037fe640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe80806bbd0 0x7fe80806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:04.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.909+0000 7fe8037fe640 1 -- 172.21.15.82:0/3158980162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe8140abcc0 con 0x7fe81c0c7ac0 2024-03-14T12:18:04.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.909+0000 7fe81affd640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe80806bbd0 0x7fe80806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:04.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.910+0000 7fe81affd640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe80806bbd0 0x7fe80806e090 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe80c007440 tx=0x7fe80c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:04.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:04.914+0000 7fe8037fe640 1 -- 172.21.15.82:0/3158980162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe81c06d910 con 0x7fe81c0c7ac0 2024-03-14T12:18:05.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.121+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe81c075210 con 0x7fe81c0c7ac0 2024-03-14T12:18:05.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.121+0000 7fe8037fe640 1 -- 172.21.15.82:0/3158980162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fe81407a160 con 0x7fe81c0c7ac0 2024-03-14T12:18:05.121 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:05.121 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:17:16.057057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:05.121 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 38 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe80806bbd0 msgr2=0x7fe80806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe80806bbd0 0x7fe80806e090 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe80c007440 tx=0x7fe80c00b040 comp rx=0 tx=0).stop 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe81c0c7ac0 msgr2=0x7fe81c0b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe81c0c7ac0 0x7fe81c0b3670 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe814000f60 tx=0x7fe814001c10 comp rx=0 tx=0).stop 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 shutdown_connections 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe81c0c7ac0 0x7fe81c0b3670 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe80806bbd0 0x7fe80806e090 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe81c0af0a0 0x7fe81c0c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 --2- 172.21.15.82:0/3158980162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8040051c0 0x7fe81c14fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:05.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 >> 172.21.15.82:0/3158980162 conn(0x7fe81c0d1950 msgr2=0x7fe81c0cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:05.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 shutdown_connections 2024-03-14T12:18:05.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:05.123+0000 7fe81bfff640 1 -- 172.21.15.82:0/3158980162 wait complete. 2024-03-14T12:18:05.133 DEBUG:tasks.cephfs.filesystem:mapped states ['up:reconnect'] to None 2024-03-14T12:18:06.134 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:06.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.247+0000 7f3b630b2640 1 Processor -- start 2024-03-14T12:18:06.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.247+0000 7f3b630b2640 1 -- start start 2024-03-14T12:18:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b630b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b630b2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b5c15dde0 0x7f3b5c15e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b630b2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5c0b3da0 con 0x7f3b5c0b4b70 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b630b2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5c070d40 con 0x7f3b5c15dde0 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b630b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c15e790 con 0x7f3b5c151230 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b6164f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b5c15dde0 0x7f3b5c15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b6164f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b5c15dde0 0x7f3b5c15e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35876/0 (socket says 172.21.15.82:35876) 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b6164f640 1 -- 172.21.15.82:0/2838210889 learned_addr learned my addr 172.21.15.82:0/2838210889 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b6164f640 1 -- 172.21.15.82:0/2838210889 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 msgr2=0x7f3b5c0cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b6164f640 1 --2- 172.21.15.82:0/2838210889 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b6164f640 1 -- 172.21.15.82:0/2838210889 >> v1:172.21.15.164:6789/0 conn(0x7f3b5c0b4b70 legacy=0x7f3b5c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.248+0000 7f3b6164f640 1 -- 172.21.15.82:0/2838210889 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0b4080 con 0x7f3b5c15dde0 2024-03-14T12:18:06.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.249+0000 7f3b6164f640 1 --2- 172.21.15.82:0/2838210889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c15e250 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00aa40 tx=0x7f3b4c02cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2e942f394c3a67 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:06.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.249+0000 7f3b5b7fe640 1 -- 172.21.15.82:0/2838210889 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c03a070 con 0x7f3b5c15dde0 2024-03-14T12:18:06.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.249+0000 7f3b5b7fe640 1 -- 172.21.15.82:0/2838210889 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c02dae0 con 0x7f3b5c15dde0 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.249+0000 7f3b5b7fe640 1 -- 172.21.15.82:0/2838210889 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00b040 con 0x7f3b5c15dde0 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 -- 172.21.15.82:0/2838210889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 msgr2=0x7f3b5c15e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 --2- 172.21.15.82:0/2838210889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c15e250 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00aa40 tx=0x7f3b4c02cd30 comp rx=0 tx=0).stop 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 -- 172.21.15.82:0/2838210889 shutdown_connections 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 --2- 172.21.15.82:0/2838210889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c15e250 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 --2- 172.21.15.82:0/2838210889 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c0cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 -- 172.21.15.82:0/2838210889 >> 172.21.15.82:0/2838210889 conn(0x7f3b5c0b1c70 msgr2=0x7f3b5c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 -- 172.21.15.82:0/2838210889 shutdown_connections 2024-03-14T12:18:06.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.250+0000 7f3b630b2640 1 -- 172.21.15.82:0/2838210889 wait complete. 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.251+0000 7f3b630b2640 1 Processor -- start 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.251+0000 7f3b630b2640 1 -- start start 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b630b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b5c0b4b70 0x7f3b5c1f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b630b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c1f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b630b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c1f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b630b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c06c690 con 0x7f3b5c15dde0 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b630b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c06c3b0 con 0x7f3b5c0b4b70 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b630b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c151610 con 0x7f3b5c151230 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c1f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c1f3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35892/0 (socket says 172.21.15.82:35892) 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 -- 172.21.15.82:0/3202520256 learned_addr learned my addr 172.21.15.82:0/3202520256 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b5bfff640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c1f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b60e4e640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b5c0b4b70 0x7f3b5c1f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:06.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 -- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 msgr2=0x7f3b5c1f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c1f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 -- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b5c0b4b70 msgr2=0x7f3b5c1f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b5c0b4b70 0x7f3b5c1f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 -- 172.21.15.82:0/3202520256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c143190 con 0x7f3b5c15dde0 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b5bfff640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c1f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b60e4e640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b5c0b4b70 0x7f3b5c1f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b6164f640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c1f3b60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00a1c0 tx=0x7f3b4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.252+0000 7f3b597fa640 1 -- 172.21.15.82:0/3202520256 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c045070 con 0x7f3b5c15dde0 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.253+0000 7f3b597fa640 1 -- 172.21.15.82:0/3202520256 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c033550 con 0x7f3b5c15dde0 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.253+0000 7f3b597fa640 1 -- 172.21.15.82:0/3202520256 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c03a070 con 0x7f3b5c15dde0 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.253+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c1f43f0 con 0x7f3b5c15dde0 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.253+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c1f49b0 con 0x7f3b5c15dde0 2024-03-14T12:18:06.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.253+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c04f050 con 0x7f3b5c15dde0 2024-03-14T12:18:06.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.254+0000 7f3b597fa640 1 -- 172.21.15.82:0/3202520256 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3b4c048070 con 0x7f3b5c15dde0 2024-03-14T12:18:06.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.254+0000 7f3b597fa640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3b3c06bbd0 0x7f3b3c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:06.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.254+0000 7f3b597fa640 1 -- 172.21.15.82:0/3202520256 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3b4c0ac630 con 0x7f3b5c15dde0 2024-03-14T12:18:06.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.254+0000 7f3b60e4e640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3b3c06bbd0 0x7f3b3c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:06.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.255+0000 7f3b60e4e640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3b3c06bbd0 0x7f3b3c06e090 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b54007eb0 tx=0x7f3b540073c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:06.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.259+0000 7f3b597fa640 1 -- 172.21.15.82:0/3202520256 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3b4c07b0f0 con 0x7f3b5c15dde0 2024-03-14T12:18:06.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.468+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b5c0b2f40 con 0x7f3b5c15dde0 2024-03-14T12:18:06.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.468+0000 7f3b597fa640 1 -- 172.21.15.82:0/3202520256 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f3b4c07a9f0 con 0x7f3b5c15dde0 2024-03-14T12:18:06.468 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:06.468 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:18:05.357533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:06.468 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 39 2024-03-14T12:18:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.470+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3b3c06bbd0 msgr2=0x7f3b3c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.470+0000 7f3b630b2640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3b3c06bbd0 0x7f3b3c06e090 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b54007eb0 tx=0x7f3b540073c0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.470+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 msgr2=0x7f3b5c1f3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.470+0000 7f3b630b2640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c1f3b60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00a1c0 tx=0x7f3b4c004730 comp rx=0 tx=0).stop 2024-03-14T12:18:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 shutdown_connections 2024-03-14T12:18:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b5c15dde0 0x7f3b5c1f3b60 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3b3c06bbd0 0x7f3b3c06e090 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3b5c151230 0x7f3b5c1f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 --2- 172.21.15.82:0/3202520256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b5c0b4b70 0x7f3b5c1f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:06.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 >> 172.21.15.82:0/3202520256 conn(0x7f3b5c0b1c70 msgr2=0x7f3b5c1500c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:06.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 shutdown_connections 2024-03-14T12:18:06.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:06.471+0000 7f3b630b2640 1 -- 172.21.15.82:0/3202520256 wait complete. 2024-03-14T12:18:06.481 DEBUG:tasks.cephfs.filesystem:mapped states ['up:rejoin'] to None 2024-03-14T12:18:07.482 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:07.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.595+0000 7f60edf1b640 1 Processor -- start 2024-03-14T12:18:07.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.595+0000 7f60edf1b640 1 -- start start 2024-03-14T12:18:07.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.595+0000 7f60edf1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80d0760 0x7f60e80d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.595+0000 7f60edf1b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e80d1080 con 0x7f60e80cf370 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.595+0000 7f60edf1b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e80cf8c0 con 0x7f60e80b7720 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.595+0000 7f60edf1b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e80cfba0 con 0x7f60e80d0760 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e6ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f60e80cf370 0x7f60e80d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39084/0 (socket says 172.21.15.82:39084) 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e6ffd640 1 -- 172.21.15.82:0/1022757708 learned_addr learned my addr 172.21.15.82:0/1022757708 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e77fe640 1 --2- 172.21.15.82:0/1022757708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80d0760 0x7f60e80d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e67fc640 1 -- 172.21.15.82:0/1022757708 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727339219 0 0) 0x7f60e80d1080 con 0x7f60e80cf370 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e67fc640 1 -- 172.21.15.82:0/1022757708 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d0003520 con 0x7f60e80cf370 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e67fc640 1 -- 172.21.15.82:0/1022757708 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659702063 0 0) 0x7f60e80cf8c0 con 0x7f60e80b7720 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e67fc640 1 -- 172.21.15.82:0/1022757708 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60e80d1080 con 0x7f60e80b7720 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e77fe640 1 -- 172.21.15.82:0/1022757708 >> v1:172.21.15.164:6789/0 conn(0x7f60e80b7720 legacy=0x7f60e80b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e77fe640 1 -- 172.21.15.82:0/1022757708 >> v1:172.21.15.82:6789/0 conn(0x7f60e80cf370 legacy=0x7f60e80d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.596+0000 7f60e77fe640 1 -- 172.21.15.82:0/1022757708 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80b6c40 con 0x7f60e80d0760 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.597+0000 7f60e77fe640 1 --2- 172.21.15.82:0/1022757708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80d0760 0x7f60e80d0b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f60d80097d0 tx=0x7f60d802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da61ba05a274483b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.597+0000 7f60e67fc640 1 -- 172.21.15.82:0/1022757708 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8034070 con 0x7f60e80d0760 2024-03-14T12:18:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.597+0000 7f60e67fc640 1 -- 172.21.15.82:0/1022757708 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d8030880 con 0x7f60e80d0760 2024-03-14T12:18:07.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.597+0000 7f60e67fc640 1 -- 172.21.15.82:0/1022757708 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8030c30 con 0x7f60e80d0760 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.598+0000 7f60edf1b640 1 -- 172.21.15.82:0/1022757708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80d0760 msgr2=0x7f60e80d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.598+0000 7f60edf1b640 1 --2- 172.21.15.82:0/1022757708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80d0760 0x7f60e80d0b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f60d80097d0 tx=0x7f60d802b9c0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.598+0000 7f60edf1b640 1 -- 172.21.15.82:0/1022757708 shutdown_connections 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.598+0000 7f60edf1b640 1 --2- 172.21.15.82:0/1022757708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80d0760 0x7f60e80d0b40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.598+0000 7f60edf1b640 1 -- 172.21.15.82:0/1022757708 >> 172.21.15.82:0/1022757708 conn(0x7f60e809d030 msgr2=0x7f60e80d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.598+0000 7f60edf1b640 1 -- 172.21.15.82:0/1022757708 shutdown_connections 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.598+0000 7f60edf1b640 1 -- 172.21.15.82:0/1022757708 wait complete. 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.599+0000 7f60edf1b640 1 Processor -- start 2024-03-14T12:18:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.599+0000 7f60edf1b640 1 -- start start 2024-03-14T12:18:07.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60edf1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80b7720 0x7f60e81f8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60edf1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e80cf370 0x7f60e8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60edf1b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60e80d0760 0x7f60e80c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60edf1b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81617a0 con 0x7f60e80cf370 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60edf1b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e80b7200 con 0x7f60e80d0760 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60edf1b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8161920 con 0x7f60e80b7720 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e80cf370 0x7f60e8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e80cf370 0x7f60e8154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35908/0 (socket says 172.21.15.82:35908) 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 -- 172.21.15.82:0/215596218 learned_addr learned my addr 172.21.15.82:0/215596218 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e77fe640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80b7720 0x7f60e81f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e7fff640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60e80d0760 0x7f60e80c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 -- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80b7720 msgr2=0x7f60e81f8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80b7720 0x7f60e81f8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 -- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60e80d0760 msgr2=0x7f60e80c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60e80d0760 0x7f60e80c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.600+0000 7f60e6ffd640 1 -- 172.21.15.82:0/215596218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80ca4c0 con 0x7f60e80cf370 2024-03-14T12:18:07.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.601+0000 7f60e6ffd640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e80cf370 0x7f60e8154180 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c530 tx=0x7f60dc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:07.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.601+0000 7f60cffff640 1 -- 172.21.15.82:0/215596218 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc017070 con 0x7f60e80cf370 2024-03-14T12:18:07.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.601+0000 7f60cffff640 1 -- 172.21.15.82:0/215596218 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc005ce0 con 0x7f60e80cf370 2024-03-14T12:18:07.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.601+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e80ca7a0 con 0x7f60e80cf370 2024-03-14T12:18:07.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.601+0000 7f60cffff640 1 -- 172.21.15.82:0/215596218 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc007270 con 0x7f60e80cf370 2024-03-14T12:18:07.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.601+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e81f7a10 con 0x7f60e80cf370 2024-03-14T12:18:07.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.602+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e804f050 con 0x7f60e80cf370 2024-03-14T12:18:07.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.602+0000 7f60cffff640 1 -- 172.21.15.82:0/215596218 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f60dc01a070 con 0x7f60e80cf370 2024-03-14T12:18:07.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.602+0000 7f60cffff640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d406bbd0 0x7f60d406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.603+0000 7f60cffff640 1 -- 172.21.15.82:0/215596218 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f60dc089e60 con 0x7f60e80cf370 2024-03-14T12:18:07.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.603+0000 7f60e77fe640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d406bbd0 0x7f60d406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.603+0000 7f60e77fe640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d406bbd0 0x7f60d406e090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f60d8036000 tx=0x7f60d803b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.607+0000 7f60cffff640 1 -- 172.21.15.82:0/215596218 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f60dc058210 con 0x7f60e80cf370 2024-03-14T12:18:07.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.815+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e80d0540 con 0x7f60e80cf370 2024-03-14T12:18:07.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.816+0000 7f60cffff640 1 -- 172.21.15.82:0/215596218 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f60dc05c2b0 con 0x7f60e80cf370 2024-03-14T12:18:07.815 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:07.816 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:18:06.362029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":21,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:07.816 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 40 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d406bbd0 msgr2=0x7f60d406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d406bbd0 0x7f60d406e090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f60d8036000 tx=0x7f60d803b040 comp rx=0 tx=0).stop 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e80cf370 msgr2=0x7f60e8154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e80cf370 0x7f60e8154180 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c530 tx=0x7f60dc00c9f0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 shutdown_connections 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60e80d0760 0x7f60e80c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e80cf370 0x7f60e8154180 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f60d406bbd0 0x7f60d406e090 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 --2- 172.21.15.82:0/215596218 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60e80b7720 0x7f60e81f8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 >> 172.21.15.82:0/215596218 conn(0x7f60e809d030 msgr2=0x7f60e80b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:07.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 shutdown_connections 2024-03-14T12:18:07.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.818+0000 7f60edf1b640 1 -- 172.21.15.82:0/215596218 wait complete. 2024-03-14T12:18:07.828 DEBUG:tasks.cephfs.filesystem:mapped states ['up:active'] to up:active 2024-03-14T12:18:07.828 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 50.08023548126221s 2024-03-14T12:18:07.829 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T12:18:07.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.929+0000 7f099311b640 1 Processor -- start 2024-03-14T12:18:07.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.929+0000 7f099311b640 1 -- start start 2024-03-14T12:18:07.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f099311b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f098c06b420 0x7f098c157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f099311b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098c0b01d0 con 0x7f098c0af150 2024-03-14T12:18:07.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f099311b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098c15e710 con 0x7f098c15f1f0 2024-03-14T12:18:07.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f099311b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c0aff90 con 0x7f098c06b420 2024-03-14T12:18:07.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f09916b8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f098c06b420 0x7f098c157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f09916b8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f098c06b420 0x7f098c157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35916/0 (socket says 172.21.15.82:35916) 2024-03-14T12:18:07.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f09916b8640 1 -- 172.21.15.82:0/503622120 learned_addr learned my addr 172.21.15.82:0/503622120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:07.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.930+0000 7f0990eb7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f098c0af150 0x7f098c0af880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39094/0 (socket says 172.21.15.82:39094) 2024-03-14T12:18:07.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.931+0000 7f09916b8640 1 -- 172.21.15.82:0/503622120 >> v1:172.21.15.164:6789/0 conn(0x7f098c15f1f0 legacy=0x7f098c06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:07.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.931+0000 7f09916b8640 1 -- 172.21.15.82:0/503622120 >> v1:172.21.15.82:6789/0 conn(0x7f098c0af150 legacy=0x7f098c0af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.931+0000 7f09916b8640 1 -- 172.21.15.82:0/503622120 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c15e9f0 con 0x7f098c06b420 2024-03-14T12:18:07.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.931+0000 7f098b7fe640 1 -- 172.21.15.82:0/503622120 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239996874 0 0) 0x7f098c0b01d0 con 0x7f098c0af150 2024-03-14T12:18:07.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.931+0000 7f09916b8640 1 --2- 172.21.15.82:0/503622120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c06b420 0x7f098c157cd0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0984009e50 tx=0x7f098402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50499909de97b3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:07.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f098b7fe640 1 -- 172.21.15.82:0/503622120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098402ca80 con 0x7f098c06b420 2024-03-14T12:18:07.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f098b7fe640 1 -- 172.21.15.82:0/503622120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098402cc20 con 0x7f098c06b420 2024-03-14T12:18:07.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f098b7fe640 1 -- 172.21.15.82:0/503622120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840343f0 con 0x7f098c06b420 2024-03-14T12:18:07.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f099311b640 1 -- 172.21.15.82:0/503622120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c06b420 msgr2=0x7f098c157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f099311b640 1 --2- 172.21.15.82:0/503622120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c06b420 0x7f098c157cd0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0984009e50 tx=0x7f098402bee0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f099311b640 1 -- 172.21.15.82:0/503622120 shutdown_connections 2024-03-14T12:18:07.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f099311b640 1 --2- 172.21.15.82:0/503622120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c06b420 0x7f098c157cd0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f099311b640 1 -- 172.21.15.82:0/503622120 >> 172.21.15.82:0/503622120 conn(0x7f098c0d0a20 msgr2=0x7f098c0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:07.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f099311b640 1 -- 172.21.15.82:0/503622120 shutdown_connections 2024-03-14T12:18:07.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.932+0000 7f099311b640 1 -- 172.21.15.82:0/503622120 wait complete. 2024-03-14T12:18:07.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.933+0000 7f099311b640 1 Processor -- start 2024-03-14T12:18:07.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.933+0000 7f099311b640 1 -- start start 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f099311b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f098c06b420 0x7f098c1462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f099311b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f098c0af150 0x7f098c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f099311b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c15f1f0 0x7f098c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f099311b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0b58f0 con 0x7f098c15f1f0 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f099311b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c15e400 con 0x7f098c06b420 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f099311b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0b5a70 con 0x7f098c0af150 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c15f1f0 0x7f098c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c15f1f0 0x7f098c0ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35924/0 (socket says 172.21.15.82:35924) 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 -- 172.21.15.82:0/3106759844 learned_addr learned my addr 172.21.15.82:0/3106759844 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f098bfff640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f098c0af150 0x7f098c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f0990eb7640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f098c06b420 0x7f098c1462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 -- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f098c0af150 msgr2=0x7f098c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f098c0af150 0x7f098c0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 -- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f098c06b420 msgr2=0x7f098c1462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f098c06b420 0x7f098c1462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.934+0000 7f09916b8640 1 -- 172.21.15.82:0/3106759844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c0cea20 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.935+0000 7f09916b8640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c15f1f0 0x7f098c0ce350 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0984009e50 tx=0x7f0984004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.935+0000 7f09897fa640 1 -- 172.21.15.82:0/3106759844 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984043030 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.935+0000 7f09897fa640 1 -- 172.21.15.82:0/3106759844 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098403e450 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.935+0000 7f09897fa640 1 -- 172.21.15.82:0/3106759844 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984009510 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.935+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c0ced00 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.935+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c146f60 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.936+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c04f050 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.936+0000 7f09897fa640 1 -- 172.21.15.82:0/3106759844 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0984046070 con 0x7f098c15f1f0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.937+0000 7f09897fa640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f097806bbd0 0x7f097806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.937+0000 7f0990eb7640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f097806bbd0 0x7f097806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:07.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.937+0000 7f09897fa640 1 -- 172.21.15.82:0/3106759844 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f09840ad0a0 con 0x7f098c15f1f0 2024-03-14T12:18:07.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.937+0000 7f0990eb7640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f097806bbd0 0x7f097806e090 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f097c007fd0 tx=0x7f097c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:07.941+0000 7f09897fa640 1 -- 172.21.15.82:0/3106759844 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f098404d070 con 0x7f098c15f1f0 2024-03-14T12:18:08.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.148+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f098c161410 con 0x7f098c15f1f0 2024-03-14T12:18:08.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.149+0000 7f09897fa640 1 -- 172.21.15.82:0/3106759844 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f09840473b0 con 0x7f098c15f1f0 2024-03-14T12:18:08.148 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T12:18:08.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f097806bbd0 msgr2=0x7f097806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:08.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f097806bbd0 0x7f097806e090 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f097c007fd0 tx=0x7f097c009040 comp rx=0 tx=0).stop 2024-03-14T12:18:08.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c15f1f0 msgr2=0x7f098c0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:08.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c15f1f0 0x7f098c0ce350 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0984009e50 tx=0x7f0984004a40 comp rx=0 tx=0).stop 2024-03-14T12:18:08.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 shutdown_connections 2024-03-14T12:18:08.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c15f1f0 0x7f098c0ce350 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:08.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f097806bbd0 0x7f097806e090 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:08.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f098c0af150 0x7f098c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:08.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.150+0000 7f099311b640 1 --2- 172.21.15.82:0/3106759844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f098c06b420 0x7f098c1462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:08.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.151+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 >> 172.21.15.82:0/3106759844 conn(0x7f098c0d0a20 msgr2=0x7f098c0ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:08.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.151+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 shutdown_connections 2024-03-14T12:18:08.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:08.151+0000 7f099311b640 1 -- 172.21.15.82:0/3106759844 wait complete. 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T12:18:08.161 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:18:08.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.162 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:18:08.177 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42432sec preferred_lft 42432sec 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c443:69ff:fec0:3488/64 scope link 2024-03-14T12:18:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:18:08.179 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state DOWN group default qlen 1000 2024-03-14T12:18:08.179 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 8a:7e:60:47:43:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:18:08.179 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T12:18:08.179 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a6:e3:19:81:dc:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:18:08.179 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a4e3:19ff:fe81:dc41/64 scope link 2024-03-14T12:18:08.179 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-14T12:18:08.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.179 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T12:18:08.242 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-14T12:18:08.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.244 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-14T12:18:08.299 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-14T12:18:08.300 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T12:18:08.300 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T12:18:08.300 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T12:18:08.300 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T12:18:08.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.300 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-14T12:18:08.300 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-14T12:18:08.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T12:18:08.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T12:18:08.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T12:18:08.300 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-14T12:18:08.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.396 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:18:08.410 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-14T12:18:08.410 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:18:08.410 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:18:08.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.410 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-14T12:18:08.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:18:08.477 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T12:18:08.537 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T12:18:08.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T12:18:08.537 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:18:08.591 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T12:18:08.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.592 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:18:08.675 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:18:08.676 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:18:08.738 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T12:18:08.738 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T12:18:08.738 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.164:3300,172.21.15.164:3301" 2024-03-14T12:18:08.738 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2.cephfs=/ 2024-03-14T12:18:08.738 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.82:3300/172.21.15.164:3300/172.21.15.164:3301" will pass to kernel 2024-03-14T12:18:08.739 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T12:18:08.739 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-14T12:18:08.739 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T12:18:08.739 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T12:18:08.739 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T12:18:08.740 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T12:18:08.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T12:18:08.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.808 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:18:08.863 INFO:teuthology.orchestra.run.smithi082.stdout:client6136 2024-03-14T12:18:08.864 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:18:08.864 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:08.919 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:18:08.919 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:08.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:08.974 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T12:18:09.030 INFO:teuthology.orchestra.run.smithi082.stdout:client6136 2024-03-14T12:18:09.030 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:18:09.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 counter dump 2024-03-14T12:18:09.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.172+0000 7ff27d212640 1 Processor -- start 2024-03-14T12:18:09.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.172+0000 7ff27d212640 1 -- start start 2024-03-14T12:18:09.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff27d212640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff27815e250 0x7ff2780aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff27d212640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27806ad10 0x7ff2780af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff27d212640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2780b00e0 con 0x7ff2780af740 2024-03-14T12:18:09.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff27d212640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2780b03c0 con 0x7ff27815e250 2024-03-14T12:18:09.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff27d212640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780b06a0 con 0x7ff27806ad10 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff276d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff27815e250 0x7ff2780aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff276d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff27815e250 0x7ff2780aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35926/0 (socket says 172.21.15.82:35926) 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff276d9d640 1 -- 172.21.15.82:0/3759749270 learned_addr learned my addr 172.21.15.82:0/3759749270 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff276d9d640 1 -- 172.21.15.82:0/3759749270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27806ad10 msgr2=0x7ff2780af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff276d9d640 1 --2- 172.21.15.82:0/3759749270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27806ad10 0x7ff2780af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff276d9d640 1 -- 172.21.15.82:0/3759749270 >> v1:172.21.15.164:6789/0 conn(0x7ff2780af740 legacy=0x7ff2780acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.173+0000 7ff276d9d640 1 -- 172.21.15.82:0/3759749270 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780b0980 con 0x7ff27815e250 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff276d9d640 1 --2- 172.21.15.82:0/3759749270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27815e250 0x7ff2780aea80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff268009870 tx=0x7ff26802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e6f0870bd964d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27659c640 1 -- 172.21.15.82:0/3759749270 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268035020 con 0x7ff27815e250 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27659c640 1 -- 172.21.15.82:0/3759749270 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268031640 con 0x7ff27815e250 2024-03-14T12:18:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27659c640 1 -- 172.21.15.82:0/3759749270 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26802d040 con 0x7ff27815e250 2024-03-14T12:18:09.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27d212640 1 -- 172.21.15.82:0/3759749270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27815e250 msgr2=0x7ff2780aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27d212640 1 --2- 172.21.15.82:0/3759749270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27815e250 0x7ff2780aea80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff268009870 tx=0x7ff26802b8f0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27d212640 1 -- 172.21.15.82:0/3759749270 shutdown_connections 2024-03-14T12:18:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27d212640 1 --2- 172.21.15.82:0/3759749270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27806ad10 0x7ff2780af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27d212640 1 --2- 172.21.15.82:0/3759749270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27815e250 0x7ff2780aea80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.174+0000 7ff27d212640 1 -- 172.21.15.82:0/3759749270 >> 172.21.15.82:0/3759749270 conn(0x7ff2780b4e50 msgr2=0x7ff2780b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.175+0000 7ff27d212640 1 -- 172.21.15.82:0/3759749270 shutdown_connections 2024-03-14T12:18:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.175+0000 7ff27d212640 1 -- 172.21.15.82:0/3759749270 wait complete. 2024-03-14T12:18:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.176+0000 7ff27d212640 1 Processor -- start 2024-03-14T12:18:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.176+0000 7ff27d212640 1 -- start start 2024-03-14T12:18:09.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27806ad10 0x7ff278155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2780af740 0x7ff278156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27815e250 0x7ff278156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27806b2f0 con 0x7ff27806ad10 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27806b170 con 0x7ff2780af740 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780b1eb0 con 0x7ff27815e250 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27806ad10 0x7ff278155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27806ad10 0x7ff278155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35928/0 (socket says 172.21.15.82:35928) 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 -- 172.21.15.82:0/3239413994 learned_addr learned my addr 172.21.15.82:0/3239413994 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 -- 172.21.15.82:0/3239413994 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27815e250 msgr2=0x7ff278156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27815e250 0x7ff278156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 -- 172.21.15.82:0/3239413994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2780af740 msgr2=0x7ff278156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2780af740 0x7ff278156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 -- 172.21.15.82:0/3239413994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780ce5b0 con 0x7ff27806ad10 2024-03-14T12:18:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff276d9d640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27806ad10 0x7ff278155ee0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff268000c00 tx=0x7ff2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff26ffff640 1 -- 172.21.15.82:0/3239413994 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26803ac60 con 0x7ff27806ad10 2024-03-14T12:18:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff26ffff640 1 -- 172.21.15.82:0/3239413994 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26803ae20 con 0x7ff27806ad10 2024-03-14T12:18:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff26ffff640 1 -- 172.21.15.82:0/3239413994 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268044630 con 0x7ff27806ad10 2024-03-14T12:18:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2780d0a70 con 0x7ff27806ad10 2024-03-14T12:18:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.177+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2780d0d10 con 0x7ff27806ad10 2024-03-14T12:18:09.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.179+0000 7ff26ffff640 1 -- 172.21.15.82:0/3239413994 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff268038020 con 0x7ff27806ad10 2024-03-14T12:18:09.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.179+0000 7ff26ffff640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff25806bbd0 0x7ff25806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.179+0000 7ff26ffff640 1 -- 172.21.15.82:0/3239413994 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff2680ac9c0 con 0x7ff27806ad10 2024-03-14T12:18:09.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.179+0000 7ff26e59c640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff25806bbd0 0x7ff25806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.180+0000 7ff26e59c640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff25806bbd0 0x7ff25806e090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff2780cdb60 tx=0x7ff260006c90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.180+0000 7ff27fdf5740 1 Processor -- start 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.180+0000 7ff27fdf5740 1 -- start start 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf928090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf926ae0 0x561adf9285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf923210 0x561adf952d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561adf8f0360 con 0x561adf926ae0 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561adf88cec0 con 0x561adf923210 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x561adf835d30 con 0x561adf970a60 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf926ae0 0x561adf9285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf926ae0 0x561adf9285d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35936/0 (socket says 172.21.15.82:35936) 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 -- 172.21.15.82:0/1152590906 learned_addr learned my addr 172.21.15.82:0/1152590906 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff276d9d640 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf928090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27759e640 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf923210 0x561adf952d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 -- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 msgr2=0x561adf928090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf928090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 -- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf923210 msgr2=0x561adf952d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf923210 0x561adf952d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 -- 172.21.15.82:0/1152590906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561adf834bc0 con 0x561adf926ae0 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff27759e640 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf923210 0x561adf952d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff276d9d640 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf928090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.181+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf926ae0 0x561adf9285d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff2600158a0 tx=0x7ff26000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af5c3b94e333fe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff26cd99640 1 -- 172.21.15.82:0/1152590906 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260023070 con 0x561adf926ae0 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff26cd99640 1 -- 172.21.15.82:0/1152590906 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff260015e20 con 0x561adf926ae0 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff26cd99640 1 -- 172.21.15.82:0/1152590906 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000bb50 con 0x561adf926ae0 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1152590906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf926ae0 msgr2=0x561adf9285d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf926ae0 0x561adf9285d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff2600158a0 tx=0x7ff26000e000 comp rx=0 tx=0).stop 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1152590906 shutdown_connections 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf923210 0x561adf952d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf926ae0 0x561adf9285d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1152590906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf928090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1152590906 >> 172.21.15.82:0/1152590906 conn(0x561adf925480 msgr2=0x561adf96dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:09.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1152590906 shutdown_connections 2024-03-14T12:18:09.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.182+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1152590906 wait complete. 2024-03-14T12:18:09.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.183+0000 7ff27fdf5740 1 -- start start 2024-03-14T12:18:09.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.183+0000 7ff27fdf5740 1 Processor -- start 2024-03-14T12:18:09.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 0x561adf9515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf951b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf974f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561adf82b4a0 con 0x561adf94a950 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561adf888f60 con 0x561adf926ae0 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x561adf926ff0 con 0x561adf970a60 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 10 client.0 _ll_register_callbacks cb 0x561adf886d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf974f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf974f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35940/0 (socket says 172.21.15.82:35940) 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 -- 172.21.15.82:0/759271467 learned_addr learned my addr 172.21.15.82:0/759271467 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff26e59c640 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf951b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff276d9d640 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 0x561adf9515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 -- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 msgr2=0x561adf951b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf951b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 -- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 msgr2=0x561adf9515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 0x561adf9515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 -- 172.21.15.82:0/759271467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561adf5a13f0 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27759e640 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf974f10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff27000e4c0 tx=0x7ff27000e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d469058ba4356731 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff255ffb640 1 -- 172.21.15.82:0/759271467 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270003b80 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff255ffb640 1 -- 172.21.15.82:0/759271467 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff270010040 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.184+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff260015e20 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff255ffb640 1 -- 172.21.15.82:0/759271467 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27001f050 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 10 client.7011 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x561adf66b7d0 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff255ffb640 1 -- 172.21.15.82:0/759271467 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff260015e20 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff255ffb640 1 -- 172.21.15.82:0/759271467 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff270022020 con 0x561adf94a950 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 10 client.7011 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 10 client.7011 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 20 client.7011 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf975560 0x561adf975970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 4 client.7011 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561adf89a400 con 0x561adf975560 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff276d9d640 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf975560 0x561adf975970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff255ffb640 10 client.7011 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.185+0000 7ff276d9d640 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf975560 0x561adf975970 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c909aa6a0f13cc3 server_cookie=b47cd08c8acef470 in_seq=0 out_seq=0 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff255ffb640 1 -- 172.21.15.82:0/759271467 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561adf89a400 con 0x561adf975560 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff255ffb640 10 client.7011 handle_command_reply: tid=0 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 client.7011 shutdown 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 msgr2=0x561adf974f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf974f10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff27000e4c0 tx=0x7ff27000e980 comp rx=0 tx=0).stop 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 shutdown_connections 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf974f10 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf975560 0x561adf975970 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf951b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/759271467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 0x561adf9515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff255ffb640 0 client.7011 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:09.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 >> 172.21.15.82:0/759271467 conn(0x561adf923210 msgr2=0x561adf9227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:09.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 shutdown_connections 2024-03-14T12:18:09.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 1 -- 172.21.15.82:0/759271467 wait complete. 2024-03-14T12:18:09.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.186+0000 7ff27fdf5740 20 client.7011 trim_cache size 0 max 16384 2024-03-14T12:18:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 Processor -- start 2024-03-14T12:18:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 -- start start 2024-03-14T12:18:09.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 0x561adf9273b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf96db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf96e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2782004b0 con 0x561adf94a950 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2781ff680 con 0x561adf926ae0 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff278147a30 con 0x561adf970a60 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf96db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf96db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35950/0 (socket says 172.21.15.82:35950) 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.198+0000 7ff26e59c640 1 -- 172.21.15.82:0/1359703648 learned_addr learned my addr 172.21.15.82:0/1359703648 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff26e59c640 1 -- 172.21.15.82:0/1359703648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 msgr2=0x561adf96e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1359703648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf96e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff26e59c640 1 -- 172.21.15.82:0/1359703648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 msgr2=0x561adf9273b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1359703648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 0x561adf9273b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff26e59c640 1 -- 172.21.15.82:0/1359703648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561adf9288d0 con 0x561adf94a950 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1359703648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf96db70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff2600090d0 tx=0x7ff260010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894e156aa58b0f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff257fff640 1 -- 172.21.15.82:0/1359703648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26002c070 con 0x561adf94a950 2024-03-14T12:18:09.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff257fff640 1 -- 172.21.15.82:0/1359703648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26000b930 con 0x561adf94a950 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff257fff640 1 -- 172.21.15.82:0/1359703648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260023040 con 0x561adf94a950 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1359703648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 msgr2=0x561adf96db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1359703648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf96db70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff2600090d0 tx=0x7ff260010000 comp rx=0 tx=0).stop 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1359703648 shutdown_connections 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1359703648 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf970a60 0x561adf96e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1359703648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf96db70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1359703648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf926ae0 0x561adf9273b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1359703648 >> 172.21.15.82:0/1359703648 conn(0x561adf923210 msgr2=0x561adf950080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1359703648 shutdown_connections 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.199+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1359703648 wait complete. 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.200+0000 7ff27fdf5740 1 -- start start 2024-03-14T12:18:09.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.200+0000 7ff27fdf5740 1 Processor -- start 2024-03-14T12:18:09.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf926ae0 0x561adf9d8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf988470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf970a60 0x561adf9dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278148030 con 0x561adf94a950 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561adf9dcb10 con 0x561adf970a60 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x561adf929130 con 0x561adf926ae0 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 10 client.0 _ll_register_callbacks cb 0x7ff260009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf988470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf988470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35956/0 (socket says 172.21.15.82:35956) 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 -- 172.21.15.82:0/1653532522 learned_addr learned my addr 172.21.15.82:0/1653532522 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff276d9d640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf926ae0 0x561adf9d8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27759e640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf970a60 0x561adf9dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 -- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf926ae0 msgr2=0x561adf9d8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf926ae0 0x561adf9d8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 -- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf970a60 msgr2=0x561adf9dbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf970a60 0x561adf9dbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 -- 172.21.15.82:0/1653532522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561adf955ce0 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff276d9d640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf926ae0 0x561adf9d8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff26e59c640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf988470 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x561adf9236c0 tx=0x7ff260004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f28e7def0d719 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff255ffb640 1 -- 172.21.15.82:0/1653532522 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260037060 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff255ffb640 1 -- 172.21.15.82:0/1653532522 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff260027670 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.201+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff26000b930 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff255ffb640 1 -- 172.21.15.82:0/1653532522 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26003d050 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 10 client.7017 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x561adf928cf0 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff255ffb640 1 -- 172.21.15.82:0/1653532522 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26000b930 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff255ffb640 1 -- 172.21.15.82:0/1653532522 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff260040020 con 0x561adf94a950 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 10 client.7017 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 10 client.7017 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 20 client.7017 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf9dc410 0x561adf9806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 4 client.7017 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x561adf973da0 con 0x561adf9dc410 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff276d9d640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf9dc410 0x561adf9806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff255ffb640 10 client.7017 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.202+0000 7ff276d9d640 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf9dc410 0x561adf9806e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ca20ede54aab32 server_cookie=e999fb0ed1febf71 in_seq=0 out_seq=0 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff255ffb640 1 -- 172.21.15.82:0/1653532522 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25187 (crc 0 0 0) 0x561adf973da0 con 0x561adf9dc410 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff255ffb640 10 client.7017 handle_command_reply: tid=0 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 client.7017 shutdown 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 msgr2=0x561adf988470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf988470 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x561adf9236c0 tx=0x7ff260004840 comp rx=0 tx=0).stop 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 shutdown_connections 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561adf970a60 0x561adf9dbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561adf94a950 0x561adf988470 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x561adf9dc410 0x561adf9806e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 --2- 172.21.15.82:0/1653532522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x561adf926ae0 0x561adf9d8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff255ffb640 0 client.7017 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:09.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 >> 172.21.15.82:0/1653532522 conn(0x561adf923210 msgr2=0x561adf922100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:09.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 shutdown_connections 2024-03-14T12:18:09.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 1 -- 172.21.15.82:0/1653532522 wait complete. 2024-03-14T12:18:09.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27fdf5740 20 client.7017 trim_cache size 0 max 16384 2024-03-14T12:18:09.210 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:09.210 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.211 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.212 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 17, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13847, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 6430, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.004038918, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000427989, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002025070, 2024-03-14T12:18:09.213 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000387684, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002804180, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000164951 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.214 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 33, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 48, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 15606, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 117112, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.007791511, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001495453, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005617807, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000566561, 2024-03-14T12:18:09.215 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009864120, 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000209874 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 11, 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000002174, 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000197 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 105904 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.216 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 52, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 47, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 116864, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 39575, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.006607242, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001952247, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.004712413, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000556944, 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:09.217 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005014647, 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000106694 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000164, 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000082 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112688, 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 17440 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.218 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 51, 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003124936, 2024-03-14T12:18:09.219 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000061273 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000121504, 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000030376 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 5, 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 5, 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:18:09.221 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 12, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 10, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 7, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 3, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1, 2024-03-14T12:18:09.222 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 10, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 10, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:18:09.223 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:18:09.224 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.225 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:18:09.226 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:18:09.227 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "6471" 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 0 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 3, 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:18:09.228 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 9, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197281, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 4196487, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009482915, 2024-03-14T12:18:09.229 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003160971 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 6 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:18:09.230 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 40096, 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 231776 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.231 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 5, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 5, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 1, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:18:09.232 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.233 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:18:09.234 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.235 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.236 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.239 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.240 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.241 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.242 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 2, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 4, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 4, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 786, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:18:09.244 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:18:09.245 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 205630, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 54, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:18:09.246 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5736, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 94, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 9744, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 19, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:18:09.247 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 47, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 1076, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 0, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 47, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:09.248 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.055998963, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001191467 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 93 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 47, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 36, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 11, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 2, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 8, 2024-03-14T12:18:09.249 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 3, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 1, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 1, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 5, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 24, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:18:09.250 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:18:09.251 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 73, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 3, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 1, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 48, 2024-03-14T12:18:09.252 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.253 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:18:09.254 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 98, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 133836, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:09.255 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 98, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 98, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 133836, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.256 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 3251, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 3251, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.257 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:09.258 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 47, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 47, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.262 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:09.263 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x560d75e68a00": [ 2024-03-14T12:18:09.264 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 538, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 538, 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:09.265 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:09.266 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:09.266 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:09.266 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.266 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.266 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:09.266 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:18:09.266 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:09.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff25806bbd0 msgr2=0x7ff25806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27d212640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff25806bbd0 0x7ff25806e090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff2780cdb60 tx=0x7ff260006c90 comp rx=0 tx=0).stop 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27806ad10 msgr2=0x7ff278155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.203+0000 7ff27d212640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27806ad10 0x7ff278155ee0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff268000c00 tx=0x7ff2680046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 shutdown_connections 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff25806bbd0 0x7ff25806e090 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff27815e250 0x7ff278156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2780af740 0x7ff278156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 --2- 172.21.15.82:0/3239413994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff27806ad10 0x7ff278155ee0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 >> 172.21.15.82:0/3239413994 conn(0x7ff2780b4e50 msgr2=0x7ff27806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 shutdown_connections 2024-03-14T12:18:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:09.204+0000 7ff27d212640 1 -- 172.21.15.82:0/3239413994 wait complete. 2024-03-14T12:18:10.270 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:18:10.270 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 counter dump 2024-03-14T12:18:10.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.384+0000 7f76e4e68640 1 Processor -- start 2024-03-14T12:18:10.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.384+0000 7f76e4e68640 1 -- start start 2024-03-14T12:18:10.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76e4e68640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76e015e200 0x7f76e00d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76e4e68640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e00d2cf0 con 0x7f76e00d28e0 2024-03-14T12:18:10.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76e4e68640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e00cece0 con 0x7f76e015de20 2024-03-14T12:18:10.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76e4e68640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e00cfd80 con 0x7f76e015e200 2024-03-14T12:18:10.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76ddd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76e015e200 0x7f76e00d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76ded9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f76e00d28e0 0x7f76e00b4960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39106/0 (socket says 172.21.15.82:39106) 2024-03-14T12:18:10.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76ddd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76e015e200 0x7f76e00d23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35970/0 (socket says 172.21.15.82:35970) 2024-03-14T12:18:10.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.385+0000 7f76ded9d640 1 -- 172.21.15.82:0/2243666310 learned_addr learned my addr 172.21.15.82:0/2243666310 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:10.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.386+0000 7f76ddd9b640 1 -- 172.21.15.82:0/2243666310 >> v1:172.21.15.164:6789/0 conn(0x7f76e015de20 legacy=0x7f76e00d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.386+0000 7f76ddd9b640 1 -- 172.21.15.82:0/2243666310 >> v1:172.21.15.82:6789/0 conn(0x7f76e00d28e0 legacy=0x7f76e00b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.386+0000 7f76dd59a640 1 -- 172.21.15.82:0/2243666310 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124605442 0 0) 0x7f76d8001110 con 0x7f76e00d28e0 2024-03-14T12:18:10.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.386+0000 7f76ddd9b640 1 -- 172.21.15.82:0/2243666310 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00cefc0 con 0x7f76e015e200 2024-03-14T12:18:10.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76ddd9b640 1 --2- 172.21.15.82:0/2243666310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e00d23a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f76d40097a0 tx=0x7f76d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4248ea9a794af0e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:10.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76dd59a640 1 -- 172.21.15.82:0/2243666310 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d40315a0 con 0x7f76e015e200 2024-03-14T12:18:10.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76dd59a640 1 -- 172.21.15.82:0/2243666310 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d4037070 con 0x7f76e015e200 2024-03-14T12:18:10.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76dd59a640 1 -- 172.21.15.82:0/2243666310 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4031960 con 0x7f76e015e200 2024-03-14T12:18:10.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76e4e68640 1 -- 172.21.15.82:0/2243666310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 msgr2=0x7f76e00d23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76e4e68640 1 --2- 172.21.15.82:0/2243666310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e00d23a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f76d40097a0 tx=0x7f76d402b840 comp rx=0 tx=0).stop 2024-03-14T12:18:10.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76e4e68640 1 -- 172.21.15.82:0/2243666310 shutdown_connections 2024-03-14T12:18:10.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76e4e68640 1 --2- 172.21.15.82:0/2243666310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e00d23a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76e4e68640 1 -- 172.21.15.82:0/2243666310 >> 172.21.15.82:0/2243666310 conn(0x7f76e009d030 msgr2=0x7f76e00d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:10.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76e4e68640 1 -- 172.21.15.82:0/2243666310 shutdown_connections 2024-03-14T12:18:10.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.387+0000 7f76e4e68640 1 -- 172.21.15.82:0/2243666310 wait complete. 2024-03-14T12:18:10.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 Processor -- start 2024-03-14T12:18:10.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 -- start start 2024-03-14T12:18:10.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e00d28e0 0x7f76e00a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e015de20 0x7f76e0158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e01592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00ce940 con 0x7f76e015e200 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00cf580 con 0x7f76e00d28e0 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.389+0000 7f76e4e68640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00ceac0 con 0x7f76e015de20 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e01592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e01592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35986/0 (socket says 172.21.15.82:35986) 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 -- 172.21.15.82:0/1190738149 learned_addr learned my addr 172.21.15.82:0/1190738149 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76de59c640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e00d28e0 0x7f76e00a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ddd9b640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e015de20 0x7f76e0158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 -- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e015de20 msgr2=0x7f76e0158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e015de20 0x7f76e0158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 -- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e00d28e0 msgr2=0x7f76e00a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e00d28e0 0x7f76e00a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 -- 172.21.15.82:0/1190738149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00ca1e0 con 0x7f76e015e200 2024-03-14T12:18:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ddd9b640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e015de20 0x7f76e0158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e01592e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c780 tx=0x7f76d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ceffd640 1 -- 172.21.15.82:0/1190738149 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8018070 con 0x7f76e015e200 2024-03-14T12:18:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ceffd640 1 -- 172.21.15.82:0/1190738149 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d80070d0 con 0x7f76e015e200 2024-03-14T12:18:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76ceffd640 1 -- 172.21.15.82:0/1190738149 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d80074c0 con 0x7f76e015e200 2024-03-14T12:18:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e00cbcd0 con 0x7f76e015e200 2024-03-14T12:18:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.390+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e00c93b0 con 0x7f76e015e200 2024-03-14T12:18:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.392+0000 7f76ceffd640 1 -- 172.21.15.82:0/1190738149 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f76d801b070 con 0x7f76e015e200 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.392+0000 7f76ceffd640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76c006bbd0 0x7f76c006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.392+0000 7f76ceffd640 1 -- 172.21.15.82:0/1190738149 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f76d8089ca0 con 0x7f76e015e200 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.392+0000 7f76de59c640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76c006bbd0 0x7f76c006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.393+0000 7f76de59c640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76c006bbd0 0x7f76c006e090 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005660 tx=0x7f76d0003d90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.393+0000 7f76e7a47740 1 Processor -- start 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76e7a47740 1 -- start start 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489ed4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ed45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489ecf210 0x563489efed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76e7a47740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563489e9c360 con 0x563489ecf210 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563489e38ec0 con 0x563489f1ca60 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x563489de1d30 con 0x563489ed2ae0 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489ecf210 0x563489efed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489ecf210 0x563489efed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36002/0 (socket says 172.21.15.82:36002) 2024-03-14T12:18:10.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.394+0000 7f76ded9d640 1 -- 172.21.15.82:0/3853826653 learned_addr learned my addr 172.21.15.82:0/3853826653 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ddd9b640 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ed45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76de59c640 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489ed4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ded9d640 1 -- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 msgr2=0x563489ed45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ded9d640 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ed45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ded9d640 1 -- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 msgr2=0x563489ed4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ded9d640 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489ed4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ded9d640 1 -- 172.21.15.82:0/3853826653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563489de0bc0 con 0x563489ecf210 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ddd9b640 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ed45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76de59c640 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489ed4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76ded9d640 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489ecf210 0x563489efed70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f76d8088110 tx=0x7f76d805c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd94ee3282f48858 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76bffff640 1 -- 172.21.15.82:0/3853826653 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d80563e0 con 0x563489ecf210 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76bffff640 1 -- 172.21.15.82:0/3853826653 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8054750 con 0x563489ecf210 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.395+0000 7f76bffff640 1 -- 172.21.15.82:0/3853826653 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8057750 con 0x563489ecf210 2024-03-14T12:18:10.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 -- 172.21.15.82:0/3853826653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489ecf210 msgr2=0x563489efed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489ecf210 0x563489efed70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f76d8088110 tx=0x7f76d805c240 comp rx=0 tx=0).stop 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 -- 172.21.15.82:0/3853826653 shutdown_connections 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489ecf210 0x563489efed70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ed45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 --2- 172.21.15.82:0/3853826653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489ed4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 -- 172.21.15.82:0/3853826653 >> 172.21.15.82:0/3853826653 conn(0x563489ed1480 msgr2=0x563489f19d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 -- 172.21.15.82:0/3853826653 shutdown_connections 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 -- 172.21.15.82:0/3853826653 wait complete. 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 -- start start 2024-03-14T12:18:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.396+0000 7f76e7a47740 1 Processor -- start 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ef7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489f202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489efa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76e7a47740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563489dd74a0 con 0x563489f220a0 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563489e34f60 con 0x563489f1ca60 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x563489ed2ff0 con 0x563489ed2ae0 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76e7a47740 10 client.0 _ll_register_callbacks cb 0x563489e32d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489efa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489efa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36016/0 (socket says 172.21.15.82:36016) 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.397+0000 7f76ded9d640 1 -- 172.21.15.82:0/1051819634 learned_addr learned my addr 172.21.15.82:0/1051819634 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:10.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76de59c640 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ef7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76ddd9b640 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489f202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76ded9d640 1 -- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 msgr2=0x563489ef7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ef7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76ded9d640 1 -- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 msgr2=0x563489f202d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489f202d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76ded9d640 1 -- 172.21.15.82:0/1051819634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563489b4d3f0 con 0x563489f220a0 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489efa7d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f76d8001eb0 tx=0x7f76d8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6223d978b85ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1051819634 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8075060 con 0x563489f220a0 2024-03-14T12:18:10.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1051819634 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f76d805e320 con 0x563489f220a0 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76d8054750 con 0x563489f220a0 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1051819634 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d8053e30 con 0x563489f220a0 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 10 client.7032 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x563489c177d0 con 0x563489f220a0 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1051819634 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8054750 con 0x563489f220a0 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1051819634 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f76d8053fd0 con 0x563489f220a0 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 10 client.7032 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 10 client.7032 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 20 client.7032 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489f2a830 0x563489f2ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 4 client.7032 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.398+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563489e46400 con 0x563489f2a830 2024-03-14T12:18:10.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76de59c640 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489f2a830 0x563489f2ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76de59c640 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489f2a830 0x563489f2ac40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5ae7a002d1344e server_cookie=6405f5d8f688ca92 in_seq=0 out_seq=0 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76bd7fa640 10 client.7032 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1051819634 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563489e46400 con 0x563489f2a830 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76bd7fa640 10 client.7032 handle_command_reply: tid=0 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76e7a47740 1 client.7032 shutdown 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 msgr2=0x563489efa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.399+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489efa7d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f76d8001eb0 tx=0x7f76d8000ed0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 shutdown_connections 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489efa7d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489f1ca60 0x563489f202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489f2a830 0x563489f2ac40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1051819634 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489ed2ae0 0x563489ef7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76bd7fa640 0 client.7032 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:10.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 >> 172.21.15.82:0/1051819634 conn(0x563489ecf210 msgr2=0x563489ed4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:10.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 shutdown_connections 2024-03-14T12:18:10.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 1 -- 172.21.15.82:0/1051819634 wait complete. 2024-03-14T12:18:10.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.400+0000 7f76e7a47740 20 client.7032 trim_cache size 0 max 16384 2024-03-14T12:18:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.411+0000 7f76e7a47740 1 Processor -- start 2024-03-14T12:18:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.411+0000 7f76e7a47740 1 -- start start 2024-03-14T12:18:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 0x563489ed4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 0x563489ed50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f02630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76e7a47740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0200440 con 0x563489f220a0 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01ff690 con 0x563489ed2ae0 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00cd0a0 con 0x563489f1ca60 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f02630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f02630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36024/0 (socket says 172.21.15.82:36024) 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 -- 172.21.15.82:0/1945620171 learned_addr learned my addr 172.21.15.82:0/1945620171 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 -- 172.21.15.82:0/1945620171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 msgr2=0x563489ed50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1945620171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 0x563489ed50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 -- 172.21.15.82:0/1945620171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 msgr2=0x563489ed4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1945620171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 0x563489ed4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.412+0000 7f76ded9d640 1 -- 172.21.15.82:0/1945620171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563489ed65c0 con 0x563489f220a0 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1945620171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f02630 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f76d8054540 tx=0x7f76d8006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a69c401a1f3784 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76bf7fe640 1 -- 172.21.15.82:0/1945620171 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8053fd0 con 0x563489f220a0 2024-03-14T12:18:10.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76bf7fe640 1 -- 172.21.15.82:0/1945620171 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8054170 con 0x563489f220a0 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76bf7fe640 1 -- 172.21.15.82:0/1945620171 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d805e320 con 0x563489f220a0 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 -- 172.21.15.82:0/1945620171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 msgr2=0x563489f02630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1945620171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f02630 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f76d8054540 tx=0x7f76d8006040 comp rx=0 tx=0).stop 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 -- 172.21.15.82:0/1945620171 shutdown_connections 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1945620171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f02630 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1945620171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 0x563489ed50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1945620171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 0x563489ed4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 -- 172.21.15.82:0/1945620171 >> 172.21.15.82:0/1945620171 conn(0x563489ecf210 msgr2=0x563489ece960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 -- 172.21.15.82:0/1945620171 shutdown_connections 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.413+0000 7f76e7a47740 1 -- 172.21.15.82:0/1945620171 wait complete. 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.414+0000 7f76e7a47740 1 -- start start 2024-03-14T12:18:10.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.414+0000 7f76e7a47740 1 Processor -- start 2024-03-14T12:18:10.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 0x563489f85360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 0x563489f84310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76e7a47740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f01620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76e7a47740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563489ed2ec0 con 0x563489f220a0 2024-03-14T12:18:10.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00cd650 con 0x563489ed2ae0 2024-03-14T12:18:10.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76e7a47740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x563489f85930 con 0x563489f1ca60 2024-03-14T12:18:10.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76e7a47740 10 client.0 _ll_register_callbacks cb 0x7f76d8054400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f01620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f01620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36026/0 (socket says 172.21.15.82:36026) 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 -- 172.21.15.82:0/1955916679 learned_addr learned my addr 172.21.15.82:0/1955916679 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76de59c640 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 0x563489f85360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ddd9b640 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 0x563489f84310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 -- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 msgr2=0x563489f84310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 0x563489f84310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 -- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 msgr2=0x563489f85360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 0x563489f85360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 -- 172.21.15.82:0/1955916679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563489f186d0 con 0x563489f220a0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76ded9d640 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f01620 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000f10 tx=0x7f76d8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ce66e39850e0b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.415+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1955916679 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8053fd0 con 0x563489f220a0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1955916679 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f76d805e320 con 0x563489f220a0 2024-03-14T12:18:10.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76d8054170 con 0x563489f220a0 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1955916679 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d8081ad0 con 0x563489f220a0 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 10 client.7038 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x563489ecf760 con 0x563489f220a0 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1955916679 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8054170 con 0x563489f220a0 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1955916679 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f76d8081c70 con 0x563489f220a0 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 10 client.7038 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 10 client.7038 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 20 client.7038 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489efdfb0 0x563489f17740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 4 client.7038 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x563489e40750 con 0x563489efdfb0 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76de59c640 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489efdfb0 0x563489f17740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76bd7fa640 10 client.7038 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:10.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.416+0000 7f76de59c640 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489efdfb0 0x563489f17740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667433b3140d3c14 server_cookie=65b96108c43f168b in_seq=0 out_seq=0 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76bd7fa640 1 -- 172.21.15.82:0/1955916679 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25190 (crc 0 0 0) 0x563489e40750 con 0x563489efdfb0 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76bd7fa640 10 client.7038 handle_command_reply: tid=0 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 client.7038 shutdown 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 msgr2=0x563489f01620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f01620 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f76d8000f10 tx=0x7f76d8009040 comp rx=0 tx=0).stop 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 shutdown_connections 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563489f220a0 0x563489f01620 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x563489efdfb0 0x563489f17740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x563489f1ca60 0x563489f84310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 --2- 172.21.15.82:0/1955916679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563489ed2ae0 0x563489f85360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76bd7fa640 0 client.7038 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 >> 172.21.15.82:0/1955916679 conn(0x563489ecf210 msgr2=0x563489f19960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 shutdown_connections 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 1 -- 172.21.15.82:0/1955916679 wait complete. 2024-03-14T12:18:10.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.417+0000 7f76e7a47740 20 client.7038 trim_cache size 0 max 16384 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:10.424 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:10.425 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 17, 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19, 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13847, 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 6430, 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-03-14T12:18:10.426 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.004142440, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000427989, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002025070, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000387684, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002804180, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000164951 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.427 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 33, 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 48, 2024-03-14T12:18:10.428 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 15606, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 117112, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.007791511, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001495453, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005617807, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000566561, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:10.429 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009864120, 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000209874 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 11, 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000002174, 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000197 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 105904 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.430 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 55, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 50, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 117428, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 84655, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.007468045, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.002124304, 2024-03-14T12:18:10.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005308771, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000578404, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 50, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005064198, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000101283 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000331, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000082 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112960, 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 18064 2024-03-14T12:18:10.432 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.433 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 51, 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003124936, 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000061273 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.434 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000121504, 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000030376 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.435 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 5, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 5, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 12, 2024-03-14T12:18:10.436 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 10, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 7, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 3, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 10, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 10, 2024-03-14T12:18:10.437 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:18:10.438 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:18:10.439 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.440 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:18:10.441 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "6471" 2024-03-14T12:18:10.442 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 0 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 3, 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 9, 2024-03-14T12:18:10.443 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197281, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 4196487, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009482915, 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003160971 2024-03-14T12:18:10.444 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 6 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:18:10.445 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 40224, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 231776 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 5, 2024-03-14T12:18:10.446 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 5, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 1, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:18:10.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.450 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.451 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.452 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.453 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.454 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.455 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:18:10.456 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-14T12:18:10.457 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 2, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 4, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 4, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.458 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:18:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:18:10.460 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 222926, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 70, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:18:10.461 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5864, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 96, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 51672, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 186, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:18:10.462 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 47, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 1076, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 0, 2024-03-14T12:18:10.463 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 47, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.055998963, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001191467 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 93 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 47, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 36, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 11, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:18:10.464 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 2, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 8, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 3, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 1, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 1, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 5, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:18:10.465 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 24, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:18:10.466 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 73, 2024-03-14T12:18:10.467 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 3, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 1, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 48, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.468 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:18:10.469 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:18:10.470 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 101, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 134078, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 101, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 101, 2024-03-14T12:18:10.471 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 134078, 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:10.472 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 3251, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 3251, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.473 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:10.474 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 47, 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 47, 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.475 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x560d75e68a00": [ 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3, 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 538, 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:10.477 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3, 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3, 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 538, 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:18:10.478 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76c006bbd0 msgr2=0x7f76c006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76c006bbd0 0x7f76c006e090 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005660 tx=0x7f76d0003d90 comp rx=0 tx=0).stop 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 msgr2=0x7f76e01592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e01592e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c780 tx=0x7f76d800cc40 comp rx=0 tx=0).stop 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 shutdown_connections 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e015e200 0x7f76e01592e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f76c006bbd0 0x7f76c006e090 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f76e015de20 0x7f76e0158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 --2- 172.21.15.82:0/1190738149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e00d28e0 0x7f76e00a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:10.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 >> 172.21.15.82:0/1190738149 conn(0x7f76e009d030 msgr2=0x7f76e00d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:10.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 shutdown_connections 2024-03-14T12:18:10.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:10.418+0000 7f76e4e68640 1 -- 172.21.15.82:0/1190738149 wait complete. 2024-03-14T12:18:10.481 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-03-14T12:18:10.440+0000 7f27bed56640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:18:11.483 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:18:11.483 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 counter dump 2024-03-14T12:18:11.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.596+0000 7fcedb129640 1 Processor -- start 2024-03-14T12:18:11.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.596+0000 7fcedb129640 1 -- start start 2024-03-14T12:18:11.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.597+0000 7fcedb129640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fced415bed0 0x7fced40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.597+0000 7fcedb129640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced40d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.597+0000 7fcedb129640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced40cf830 con 0x7fced40d26e0 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.597+0000 7fcedb129640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced40cd7f0 con 0x7fced415bed0 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.597+0000 7fcedb129640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced40cfad0 con 0x7fced40cd270 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced8ec5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fced415bed0 0x7fced40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced3fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced40d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced96c6640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fced40d26e0 0x7fced40ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:36178/0 (socket says 172.21.15.82:36178) 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced96c6640 1 -- 172.21.15.82:0/3498950153 learned_addr learned my addr 172.21.15.82:0/3498950153 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced8ec5640 1 -- 172.21.15.82:0/3498950153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 msgr2=0x7fced40d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3498950153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced40d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced8ec5640 1 -- 172.21.15.82:0/3498950153 >> v1:172.21.15.164:6789/0 conn(0x7fced40d26e0 legacy=0x7fced40ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced8ec5640 1 -- 172.21.15.82:0/3498950153 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced40cfdb0 con 0x7fced415bed0 2024-03-14T12:18:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.598+0000 7fced37fe640 1 -- 172.21.15.82:0/3498950153 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177804321 0 0) 0x7fced40cf830 con 0x7fced40d26e0 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.599+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3498950153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 0x7fced40c5710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcec40097a0 tx=0x7fcec402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22f1b2c7c895a82 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.599+0000 7fced37fe640 1 -- 172.21.15.82:0/3498950153 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4035020 con 0x7fced415bed0 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.599+0000 7fced37fe640 1 -- 172.21.15.82:0/3498950153 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec4031830 con 0x7fced415bed0 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.599+0000 7fced37fe640 1 -- 172.21.15.82:0/3498950153 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4031c00 con 0x7fced415bed0 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 -- 172.21.15.82:0/3498950153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 msgr2=0x7fced40c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 --2- 172.21.15.82:0/3498950153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 0x7fced40c5710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcec40097a0 tx=0x7fcec402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 -- 172.21.15.82:0/3498950153 shutdown_connections 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 --2- 172.21.15.82:0/3498950153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced40d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 --2- 172.21.15.82:0/3498950153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 0x7fced40c5710 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 -- 172.21.15.82:0/3498950153 >> 172.21.15.82:0/3498950153 conn(0x7fced40d1bc0 msgr2=0x7fced40d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 -- 172.21.15.82:0/3498950153 shutdown_connections 2024-03-14T12:18:11.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.600+0000 7fcedb129640 1 -- 172.21.15.82:0/3498950153 wait complete. 2024-03-14T12:18:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.601+0000 7fcedb129640 1 Processor -- start 2024-03-14T12:18:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.601+0000 7fcedb129640 1 -- start start 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fcedb129640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced409bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fcedb129640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced40d26e0 0x7fced40ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fcedb129640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 0x7fced406c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fcedb129640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced415c2b0 con 0x7fced40d26e0 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fcedb129640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced406b770 con 0x7fced415bed0 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fcedb129640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced40ceba0 con 0x7fced40cd270 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced40d26e0 0x7fced40ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced40d26e0 0x7fced40ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55738/0 (socket says 172.21.15.82:55738) 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced3fff640 1 -- 172.21.15.82:0/886241925 learned_addr learned my addr 172.21.15.82:0/886241925 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced8ec5640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced409bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced96c6640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 0x7fced406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced3fff640 1 -- 172.21.15.82:0/886241925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 msgr2=0x7fced409bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced3fff640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced409bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.602+0000 7fced3fff640 1 -- 172.21.15.82:0/886241925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 msgr2=0x7fced406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fced3fff640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 0x7fced406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fced3fff640 1 -- 172.21.15.82:0/886241925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced406da00 con 0x7fced40d26e0 2024-03-14T12:18:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fced3fff640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced40d26e0 0x7fced40ce490 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e9e0 tx=0x7fcec800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fced17fa640 1 -- 172.21.15.82:0/886241925 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8017070 con 0x7fced40d26e0 2024-03-14T12:18:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fced17fa640 1 -- 172.21.15.82:0/886241925 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec800c9d0 con 0x7fced40d26e0 2024-03-14T12:18:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fced17fa640 1 -- 172.21.15.82:0/886241925 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec800cdc0 con 0x7fced40d26e0 2024-03-14T12:18:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced406dce0 con 0x7fced40d26e0 2024-03-14T12:18:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.603+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced406c5d0 con 0x7fced40d26e0 2024-03-14T12:18:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.604+0000 7fced17fa640 1 -- 172.21.15.82:0/886241925 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcec801a070 con 0x7fced40d26e0 2024-03-14T12:18:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.605+0000 7fced17fa640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcec006bc00 0x7fcec006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.605+0000 7fced17fa640 1 -- 172.21.15.82:0/886241925 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fcec808af20 con 0x7fced40d26e0 2024-03-14T12:18:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.605+0000 7fced8ec5640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcec006bc00 0x7fcec006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.605+0000 7fced8ec5640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcec006bc00 0x7fcec006e0c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcec4036000 tx=0x7fcec403c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.606+0000 7fceddde3740 1 Processor -- start 2024-03-14T12:18:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.606+0000 7fceddde3740 1 -- start start 2024-03-14T12:18:11.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e71a60 0x5620e9e29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 0x5620e9e295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceddde3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e24210 0x5620e9e53d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceddde3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e9df1360 con 0x5620e9e24210 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e9d8dec0 con 0x5620e9e71a60 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620e9d36d30 con 0x5620e9e27ae0 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e24210 0x5620e9e53d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e24210 0x5620e9e53d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55746/0 (socket says 172.21.15.82:55746) 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 -- 172.21.15.82:0/608368885 learned_addr learned my addr 172.21.15.82:0/608368885 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced8ec5640 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e71a60 0x5620e9e29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced3fff640 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 0x5620e9e295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 -- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 msgr2=0x5620e9e295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 0x5620e9e295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 -- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e71a60 msgr2=0x5620e9e29090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e71a60 0x5620e9e29090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 -- 172.21.15.82:0/608368885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e9d35bc0 con 0x5620e9e24210 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fced96c6640 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e24210 0x5620e9e53d70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcecc001c30 tx=0x7fcecc0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4317f340a66a6fc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceba7fc640 1 -- 172.21.15.82:0/608368885 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc007850 con 0x5620e9e24210 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceba7fc640 1 -- 172.21.15.82:0/608368885 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc018070 con 0x5620e9e24210 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.607+0000 7fceba7fc640 1 -- 172.21.15.82:0/608368885 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc013630 con 0x5620e9e24210 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 -- 172.21.15.82:0/608368885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e24210 msgr2=0x5620e9e53d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e24210 0x5620e9e53d70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcecc001c30 tx=0x7fcecc0020f0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 -- 172.21.15.82:0/608368885 shutdown_connections 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e24210 0x5620e9e53d70 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 0x5620e9e295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 --2- 172.21.15.82:0/608368885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e71a60 0x5620e9e29090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 -- 172.21.15.82:0/608368885 >> 172.21.15.82:0/608368885 conn(0x5620e9e26480 msgr2=0x5620e9e6ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 -- 172.21.15.82:0/608368885 shutdown_connections 2024-03-14T12:18:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.608+0000 7fceddde3740 1 -- 172.21.15.82:0/608368885 wait complete. 2024-03-14T12:18:11.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.609+0000 7fceddde3740 1 -- start start 2024-03-14T12:18:11.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.609+0000 7fceddde3740 1 Processor -- start 2024-03-14T12:18:11.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e27ae0 0x5620e9e4f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e75290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e9d2c4a0 con 0x5620e9e27ae0 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e9d89f60 con 0x5620e9e76de0 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620e9e27ff0 con 0x5620e9e71a60 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 10 client.0 _ll_register_callbacks cb 0x5620e9d87d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e27ae0 0x5620e9e4f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e27ae0 0x5620e9e4f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55748/0 (socket says 172.21.15.82:55748) 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 -- 172.21.15.82:0/3834082105 learned_addr learned my addr 172.21.15.82:0/3834082105 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced3fff640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e75290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced96c6640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 -- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 msgr2=0x5620e9e75290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e75290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 -- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 msgr2=0x5620e9e533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 -- 172.21.15.82:0/3834082105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620e9aa23f0 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced96c6640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e27ae0 0x5620e9e4f470 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fcec402bee0 tx=0x7fcec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c9dcd7ba5fc725 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceb37fe640 1 -- 172.21.15.82:0/3834082105 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4044600 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceb37fe640 1 -- 172.21.15.82:0/3834082105 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fcec40447a0 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.610+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcecc018070 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceb37fe640 1 -- 172.21.15.82:0/3834082105 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcec405b050 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 10 client.7047 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5620e9b6c7d0 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceb37fe640 1 -- 172.21.15.82:0/3834082105 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc018070 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceb37fe640 1 -- 172.21.15.82:0/3834082105 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcec405e020 con 0x5620e9e27ae0 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 10 client.7047 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 10 client.7047 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 20 client.7047 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:11.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e4a860 0x5620e9e722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 4 client.7047 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620e9d9b400 con 0x5620e9e4a860 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fced3fff640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e4a860 0x5620e9e722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fced3fff640 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e4a860 0x5620e9e722f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7cd34b263578b2 server_cookie=9310a77d8888052c in_seq=0 out_seq=0 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.611+0000 7fceb37fe640 10 client.7047 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceb37fe640 1 -- 172.21.15.82:0/3834082105 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620e9d9b400 con 0x5620e9e4a860 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceb37fe640 10 client.7047 handle_command_reply: tid=0 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 client.7047 shutdown 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e27ae0 msgr2=0x5620e9e4f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e27ae0 0x5620e9e4f470 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fcec402bee0 tx=0x7fcec4004730 comp rx=0 tx=0).stop 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 shutdown_connections 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e4a860 0x5620e9e722f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e75290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 --2- 172.21.15.82:0/3834082105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e27ae0 0x5620e9e4f470 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceb37fe640 0 client.7047 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:11.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 >> 172.21.15.82:0/3834082105 conn(0x5620e9e24210 msgr2=0x5620e9e239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:11.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 shutdown_connections 2024-03-14T12:18:11.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 1 -- 172.21.15.82:0/3834082105 wait complete. 2024-03-14T12:18:11.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.612+0000 7fceddde3740 20 client.7047 trim_cache size 0 max 16384 2024-03-14T12:18:11.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.624+0000 7fceddde3740 1 Processor -- start 2024-03-14T12:18:11.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.624+0000 7fceddde3740 1 -- start start 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 0x5620e9e752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceddde3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e71a60 0x5620e9e28140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e28c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceddde3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced4200270 con 0x5620e9e71a60 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced409e730 con 0x5620e9e76de0 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced40b6460 con 0x5620e9e27ae0 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e71a60 0x5620e9e28140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e71a60 0x5620e9e28140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55750/0 (socket says 172.21.15.82:55750) 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 -- 172.21.15.82:0/4255236622 learned_addr learned my addr 172.21.15.82:0/4255236622 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 -- 172.21.15.82:0/4255236622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 msgr2=0x5620e9e752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 --2- 172.21.15.82:0/4255236622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 0x5620e9e752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 -- 172.21.15.82:0/4255236622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 msgr2=0x5620e9e28c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 --2- 172.21.15.82:0/4255236622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e28c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 -- 172.21.15.82:0/4255236622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e9e92370 con 0x5620e9e71a60 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fced3fff640 1 --2- 172.21.15.82:0/4255236622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e71a60 0x5620e9e28140 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcec80097a0 tx=0x7fcec8078040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b5d1ebc6fc322e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceb9ffb640 1 -- 172.21.15.82:0/4255236622 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8021040 con 0x5620e9e71a60 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceb9ffb640 1 -- 172.21.15.82:0/4255236622 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec8061810 con 0x5620e9e71a60 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.625+0000 7fceb9ffb640 1 -- 172.21.15.82:0/4255236622 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec805f040 con 0x5620e9e71a60 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 -- 172.21.15.82:0/4255236622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e71a60 msgr2=0x5620e9e28140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 --2- 172.21.15.82:0/4255236622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e71a60 0x5620e9e28140 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcec80097a0 tx=0x7fcec8078040 comp rx=0 tx=0).stop 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 -- 172.21.15.82:0/4255236622 shutdown_connections 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 --2- 172.21.15.82:0/4255236622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e76de0 0x5620e9e28c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 --2- 172.21.15.82:0/4255236622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e71a60 0x5620e9e28140 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 --2- 172.21.15.82:0/4255236622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e27ae0 0x5620e9e752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 -- 172.21.15.82:0/4255236622 >> 172.21.15.82:0/4255236622 conn(0x5620e9e24210 msgr2=0x5620e9e246f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:11.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 -- 172.21.15.82:0/4255236622 shutdown_connections 2024-03-14T12:18:11.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 -- 172.21.15.82:0/4255236622 wait complete. 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.626+0000 7fceddde3740 1 -- start start 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 1 Processor -- start 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e27ae0 0x5620e9e28c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e29140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e76de0 0x5620e9e505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e9e4f760 con 0x5620e9e76de0 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced409e130 con 0x5620e9e27ae0 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620e9ed98f0 con 0x5620e9e71a60 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fceddde3740 10 client.0 _ll_register_callbacks cb 0x7fcecc004c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fced96c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e76de0 0x5620e9e505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fced96c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e76de0 0x5620e9e505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55754/0 (socket says 172.21.15.82:55754) 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.627+0000 7fced96c6640 1 -- 172.21.15.82:0/3206574407 learned_addr learned my addr 172.21.15.82:0/3206574407 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced3fff640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e29140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e27ae0 0x5620e9e28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced96c6640 1 -- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 msgr2=0x5620e9e29140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced96c6640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e29140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced96c6640 1 -- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e27ae0 msgr2=0x5620e9e28c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced96c6640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e27ae0 0x5620e9e28c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced96c6640 1 -- 172.21.15.82:0/3206574407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620e9eddad0 con 0x5620e9e76de0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced3fff640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e29140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fced96c6640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e76de0 0x5620e9e505e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcecc014000 tx=0x7fcecc001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672423514e865a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceb37fe640 1 -- 172.21.15.82:0/3206574407 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc004cf0 con 0x5620e9e76de0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceb37fe640 1 -- 172.21.15.82:0/3206574407 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fcecc01c660 con 0x5620e9e76de0 2024-03-14T12:18:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcec8061810 con 0x5620e9e76de0 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceb37fe640 1 -- 172.21.15.82:0/3206574407 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcecc02f050 con 0x5620e9e76de0 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 10 client.7053 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5620e9e51650 con 0x5620e9e76de0 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceb37fe640 1 -- 172.21.15.82:0/3206574407 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec8061810 con 0x5620e9e76de0 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceb37fe640 1 -- 172.21.15.82:0/3206574407 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcecc032020 con 0x5620e9e76de0 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 10 client.7053 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 10 client.7053 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 20 client.7053 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e29680 0x5620e9e50b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 4 client.7053 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.628+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x5620e9d95750 con 0x5620e9e29680 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.629+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e29680 0x5620e9e50b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.629+0000 7fceb37fe640 10 client.7053 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.629+0000 7fced8ec5640 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e29680 0x5620e9e50b20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fe40032e436158 server_cookie=a5ff2e5c907f73b3 in_seq=0 out_seq=0 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.629+0000 7fceb37fe640 1 -- 172.21.15.82:0/3206574407 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25192 (crc 0 0 0) 0x5620e9d95750 con 0x5620e9e29680 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.629+0000 7fceb37fe640 10 client.7053 handle_command_reply: tid=0 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.629+0000 7fceddde3740 1 client.7053 shutdown 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e76de0 msgr2=0x5620e9e505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e76de0 0x5620e9e505e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcecc014000 tx=0x7fcecc001250 comp rx=0 tx=0).stop 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 shutdown_connections 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620e9e76de0 0x5620e9e505e0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x5620e9e29680 0x5620e9e50b20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x5620e9e71a60 0x5620e9e29140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 --2- 172.21.15.82:0/3206574407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5620e9e27ae0 0x5620e9e28c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceb37fe640 0 client.7053 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 >> 172.21.15.82:0/3206574407 conn(0x5620e9e24210 msgr2=0x5620e9e29f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 shutdown_connections 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 1 -- 172.21.15.82:0/3206574407 wait complete. 2024-03-14T12:18:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fceddde3740 20 client.7053 trim_cache size 0 max 16384 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:11.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.637 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 17, 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19, 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13847, 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 6430, 2024-03-14T12:18:11.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.004225230, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000427989, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002025070, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000387684, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002804180, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000164951 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 35, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 49, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 16442, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 118904, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:18:11.640 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008066151, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001626150, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005682107, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000629336, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 48, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009912259, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000206505 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 13, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000002449, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000188 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.641 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 107696 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 57, 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 52, 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 117720, 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 129114, 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 10, 2024-03-14T12:18:11.642 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008167591, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.002204930, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005865432, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000578404, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 52, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005085675, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000097801 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000530, 2024-03-14T12:18:11.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000088 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112960, 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 18064 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.644 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 51, 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003124936, 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000061273 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.645 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000121504, 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000030376 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.646 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 5, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 5, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 12, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 10, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 7, 2024-03-14T12:18:11.647 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 3, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 10, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 10, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:18:11.648 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:18:11.649 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:18:11.650 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:18:11.651 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:18:11.652 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "6471" 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 0 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.653 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 3, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 9, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:18:11.654 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197281, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 4196487, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009482915, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003160971 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 6 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:18:11.655 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 40480, 2024-03-14T12:18:11.656 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 231776 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 5, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 5, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 1, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.659 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.660 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.661 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.663 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.664 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:18:11.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 2, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 4, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 4, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1, 2024-03-14T12:18:11.668 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:18:11.669 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:18:11.670 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 222926, 2024-03-14T12:18:11.671 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 70, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5864, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 96, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:18:11.672 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 51672, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 186, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.673 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 47, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 1076, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 0, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 47, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.055998963, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001191467 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:11.674 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 93 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 47, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 36, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 11, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 2, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 8, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 3, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 1, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 1, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:18:11.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 5, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 24, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:18:11.676 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:18:11.677 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 73, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 3, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 1, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 48, 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:18:11.678 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:18:11.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.680 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 105, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 134846, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 105, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 105, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 134846, 2024-03-14T12:18:11.681 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:11.682 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 3251, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 3251, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.683 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 47, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 47, 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:11.684 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:11.685 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:11.686 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x560d75e68a00": [ 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 538, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 538, 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:11.687 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:11.688 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:11.688 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:11.688 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.688 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.688 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:11.688 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:18:11.688 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcec006bc00 msgr2=0x7fcec006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcec006bc00 0x7fcec006e0c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcec4036000 tx=0x7fcec403c040 comp rx=0 tx=0).stop 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced40d26e0 msgr2=0x7fced40ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced40d26e0 0x7fced40ce490 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e9e0 tx=0x7fcec800eea0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 shutdown_connections 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415bed0 0x7fced406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced40d26e0 0x7fced40ce490 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcec006bc00 0x7fcec006e0c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 --2- 172.21.15.82:0/886241925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced40cd270 0x7fced409bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 >> 172.21.15.82:0/886241925 conn(0x7fced40d1bc0 msgr2=0x7fced40c56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.630+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 shutdown_connections 2024-03-14T12:18:11.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:11.631+0000 7fcedb129640 1 -- 172.21.15.82:0/886241925 wait complete. 2024-03-14T12:18:12.692 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:18:12.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 counter dump 2024-03-14T12:18:12.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.805+0000 7fad186e8640 1 Processor -- start 2024-03-14T12:18:12.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.805+0000 7fad186e8640 1 -- start start 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad186e8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad1006c440 0x7fad10070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad186e8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 0x7fad1015c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad186e8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad10070fe0 con 0x7fad100706c0 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad186e8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1015b2b0 con 0x7fad1006c440 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad186e8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1015b590 con 0x7fad1015ace0 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad15c83640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad1006c440 0x7fad10070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad15c83640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad1006c440 0x7fad10070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55758/0 (socket says 172.21.15.82:55758) 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad15c83640 1 -- 172.21.15.82:0/2743132032 learned_addr learned my addr 172.21.15.82:0/2743132032 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad16c85640 1 --2- 172.21.15.82:0/2743132032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 0x7fad1015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad15c83640 1 -- 172.21.15.82:0/2743132032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 msgr2=0x7fad1015c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad15c83640 1 --2- 172.21.15.82:0/2743132032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 0x7fad1015c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad15c83640 1 -- 172.21.15.82:0/2743132032 >> v1:172.21.15.164:6789/0 conn(0x7fad100706c0 legacy=0x7fad1006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.806+0000 7fad15c83640 1 -- 172.21.15.82:0/2743132032 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1015b870 con 0x7fad1006c440 2024-03-14T12:18:12.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.807+0000 7fad15c83640 1 --2- 172.21.15.82:0/2743132032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1006c440 0x7fad10070aa0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fad00009870 tx=0x7fad0002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0803184fe6917dd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:12.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.807+0000 7fad15482640 1 -- 172.21.15.82:0/2743132032 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00034070 con 0x7fad1006c440 2024-03-14T12:18:12.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.807+0000 7fad15482640 1 -- 172.21.15.82:0/2743132032 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad00030840 con 0x7fad1006c440 2024-03-14T12:18:12.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.807+0000 7fad15482640 1 -- 172.21.15.82:0/2743132032 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00030c10 con 0x7fad1006c440 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.807+0000 7fad186e8640 1 -- 172.21.15.82:0/2743132032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1006c440 msgr2=0x7fad10070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.808+0000 7fad186e8640 1 --2- 172.21.15.82:0/2743132032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1006c440 0x7fad10070aa0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fad00009870 tx=0x7fad0002b980 comp rx=0 tx=0).stop 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.808+0000 7fad186e8640 1 -- 172.21.15.82:0/2743132032 shutdown_connections 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.808+0000 7fad186e8640 1 --2- 172.21.15.82:0/2743132032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 0x7fad1015c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.808+0000 7fad186e8640 1 --2- 172.21.15.82:0/2743132032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1006c440 0x7fad10070aa0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.808+0000 7fad186e8640 1 -- 172.21.15.82:0/2743132032 >> 172.21.15.82:0/2743132032 conn(0x7fad100cf910 msgr2=0x7fad100cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.808+0000 7fad186e8640 1 -- 172.21.15.82:0/2743132032 shutdown_connections 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.808+0000 7fad186e8640 1 -- 172.21.15.82:0/2743132032 wait complete. 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.809+0000 7fad186e8640 1 Processor -- start 2024-03-14T12:18:12.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.809+0000 7fad186e8640 1 -- start start 2024-03-14T12:18:12.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad1006c440 0x7fad100cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100706c0 0x7fad1013ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 0x7fad10152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1015cab0 con 0x7fad100706c0 2024-03-14T12:18:12.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1015be30 con 0x7fad1006c440 2024-03-14T12:18:12.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1015cc30 con 0x7fad1015ace0 2024-03-14T12:18:12.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100706c0 0x7fad1013ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100706c0 0x7fad1013ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55764/0 (socket says 172.21.15.82:55764) 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 -- 172.21.15.82:0/1183730901 learned_addr learned my addr 172.21.15.82:0/1183730901 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 -- 172.21.15.82:0/1183730901 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 msgr2=0x7fad10152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 0x7fad10152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 -- 172.21.15.82:0/1183730901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad1006c440 msgr2=0x7fad100cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad1006c440 0x7fad100cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 -- 172.21.15.82:0/1183730901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100cdf00 con 0x7fad100706c0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad15c83640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100706c0 0x7fad1013ecb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fad0002b9c0 tx=0x7fad00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad06ffd640 1 -- 172.21.15.82:0/1183730901 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00048070 con 0x7fad100706c0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad06ffd640 1 -- 172.21.15.82:0/1183730901 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad00030640 con 0x7fad100706c0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad100ce1e0 con 0x7fad100706c0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad06ffd640 1 -- 172.21.15.82:0/1183730901 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00034040 con 0x7fad100706c0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.810+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1013f850 con 0x7fad100706c0 2024-03-14T12:18:12.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.812+0000 7fad06ffd640 1 -- 172.21.15.82:0/1183730901 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fad0004b070 con 0x7fad100706c0 2024-03-14T12:18:12.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.812+0000 7fad06ffd640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7facf406bbd0 0x7facf406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.813+0000 7fad16484640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7facf406bbd0 0x7facf406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.813+0000 7fad06ffd640 1 -- 172.21.15.82:0/1183730901 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fad000acc40 con 0x7fad100706c0 2024-03-14T12:18:12.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.813+0000 7fad16484640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7facf406bbd0 0x7facf406e090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fad0c000f60 tx=0x7fad0c0079d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:12.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.813+0000 7fad1b39e740 1 Processor -- start 2024-03-14T12:18:12.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.814+0000 7fad1b39e740 1 -- start start 2024-03-14T12:18:12.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.814+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74b5c0 0x56257e76a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.814+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e7697b0 0x56257e723aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.814+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e74dd70 0x56257e74e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.814+0000 7fad1b39e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257e6eb360 con 0x56257e74dd70 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.814+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257e687ec0 con 0x56257e74b5c0 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.814+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56257e630d30 con 0x56257e7697b0 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e74dd70 0x56257e74e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e74dd70 0x56257e74e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55768/0 (socket says 172.21.15.82:55768) 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 -- 172.21.15.82:0/2795960519 learned_addr learned my addr 172.21.15.82:0/2795960519 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad15c83640 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e7697b0 0x56257e723aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16484640 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74b5c0 0x56257e76a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 -- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e7697b0 msgr2=0x56257e723aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e7697b0 0x56257e723aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 -- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74b5c0 msgr2=0x56257e76a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74b5c0 0x56257e76a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 -- 172.21.15.82:0/2795960519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56257e62fbc0 con 0x56257e74dd70 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad15c83640 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e7697b0 0x56257e723aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16484640 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74b5c0 0x56257e76a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7fad16c85640 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e74dd70 0x56257e74e1f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fad08009f80 tx=0x7fad080077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f1780a483b1f3b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7facf3fff640 1 -- 172.21.15.82:0/2795960519 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08019070 con 0x56257e74dd70 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7facf3fff640 1 -- 172.21.15.82:0/2795960519 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad08012730 con 0x56257e74dd70 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.815+0000 7facf3fff640 1 -- 172.21.15.82:0/2795960519 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08012b60 con 0x56257e74dd70 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 -- 172.21.15.82:0/2795960519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e74dd70 msgr2=0x56257e74e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e74dd70 0x56257e74e1f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fad08009f80 tx=0x7fad080077c0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 -- 172.21.15.82:0/2795960519 shutdown_connections 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e74dd70 0x56257e74e1f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e7697b0 0x56257e723aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 --2- 172.21.15.82:0/2795960519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74b5c0 0x56257e76a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 -- 172.21.15.82:0/2795960519 >> 172.21.15.82:0/2795960519 conn(0x56257e720480 msgr2=0x56257e768d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:12.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 -- 172.21.15.82:0/2795960519 shutdown_connections 2024-03-14T12:18:12.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.816+0000 7fad1b39e740 1 -- 172.21.15.82:0/2795960519 wait complete. 2024-03-14T12:18:12.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.817+0000 7fad1b39e740 1 -- start start 2024-03-14T12:18:12.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.817+0000 7fad1b39e740 1 Processor -- start 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74dd70 0x56257e77d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e749790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e749e00 0x56257e770020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257e6264a0 con 0x56257e7697b0 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257e683f60 con 0x56257e74dd70 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56257e74b9a0 con 0x56257e749e00 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 10 client.0 _ll_register_callbacks cb 0x56257e681d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e749790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e749790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55774/0 (socket says 172.21.15.82:55774) 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 -- 172.21.15.82:0/3300191355 learned_addr learned my addr 172.21.15.82:0/3300191355 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad16c85640 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e749e00 0x56257e770020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad16484640 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74dd70 0x56257e77d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 -- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e749e00 msgr2=0x56257e770020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e749e00 0x56257e770020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 -- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74dd70 msgr2=0x56257e77d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74dd70 0x56257e77d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 -- 172.21.15.82:0/3300191355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56257e39c3f0 con 0x56257e7697b0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad15c83640 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e749790 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fad000ab0b0 tx=0x7fad00030810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09d8c452c55a3e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7facf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7facf17fa640 1 -- 172.21.15.82:0/3300191355 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0007a740 con 0x56257e7697b0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7facf17fa640 1 -- 172.21.15.82:0/3300191355 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fad000811d0 con 0x56257e7697b0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad08012730 con 0x56257e7697b0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7facf17fa640 1 -- 172.21.15.82:0/3300191355 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad0007f240 con 0x56257e7697b0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.818+0000 7fad1b39e740 10 client.7065 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x56257e4667d0 con 0x56257e7697b0 2024-03-14T12:18:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7facf17fa640 1 -- 172.21.15.82:0/3300191355 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad08012730 con 0x56257e7697b0 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7facf17fa640 1 -- 172.21.15.82:0/3300191355 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fad00076e40 con 0x56257e7697b0 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad1b39e740 10 client.7065 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad1b39e740 10 client.7065 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad1b39e740 20 client.7065 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76e7b0 0x56257e76ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad1b39e740 4 client.7065 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56257e695400 con 0x56257e76e7b0 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad16484640 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76e7b0 0x56257e76ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7facf17fa640 10 client.7065 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.819+0000 7fad16484640 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76e7b0 0x56257e76ebf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b9221ee3fac66e server_cookie=90a512cb4857f13d in_seq=0 out_seq=0 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7facf17fa640 1 -- 172.21.15.82:0/3300191355 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56257e695400 con 0x56257e76e7b0 2024-03-14T12:18:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7facf17fa640 10 client.7065 handle_command_reply: tid=0 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 client.7065 shutdown 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 msgr2=0x56257e749790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e749790 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fad000ab0b0 tx=0x7fad00030810 comp rx=0 tx=0).stop 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 shutdown_connections 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76e7b0 0x56257e76ebf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e749e00 0x56257e770020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e749790 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3300191355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e74dd70 0x56257e77d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7facf17fa640 0 client.7065 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 >> 172.21.15.82:0/3300191355 conn(0x56257e74b5c0 msgr2=0x56257e71e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 shutdown_connections 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 1 -- 172.21.15.82:0/3300191355 wait complete. 2024-03-14T12:18:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.820+0000 7fad1b39e740 20 client.7065 trim_cache size 0 max 16384 2024-03-14T12:18:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 Processor -- start 2024-03-14T12:18:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 -- start start 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e71d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e71d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b3000 con 0x56257e7697b0 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10142260 con 0x56257e749e00 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad10141660 con 0x56257e74dd70 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad16c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.832+0000 7fad16c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55784/0 (socket says 172.21.15.82:55784) 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16c85640 1 -- 172.21.15.82:0/3411981378 learned_addr learned my addr 172.21.15.82:0/3411981378 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad15c83640 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e71d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16c85640 1 -- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 msgr2=0x56257e71d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16484640 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e71d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16c85640 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e71d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16c85640 1 -- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 msgr2=0x56257e71d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16c85640 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e71d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16c85640 1 -- 172.21.15.82:0/3411981378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56257e771d00 con 0x56257e7697b0 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16484640 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e71d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad15c83640 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e71d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7fad16c85640 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76f1d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fad08014000 tx=0x7fad08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47892795dd25086 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7facf37fe640 1 -- 172.21.15.82:0/3411981378 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08019040 con 0x56257e7697b0 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7facf37fe640 1 -- 172.21.15.82:0/3411981378 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad08023070 con 0x56257e7697b0 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.833+0000 7facf37fe640 1 -- 172.21.15.82:0/3411981378 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08003df0 con 0x56257e7697b0 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 -- 172.21.15.82:0/3411981378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 msgr2=0x56257e76f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76f1d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fad08014000 tx=0x7fad08012000 comp rx=0 tx=0).stop 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 -- 172.21.15.82:0/3411981378 shutdown_connections 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76f1d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e71d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 --2- 172.21.15.82:0/3411981378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e71d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 -- 172.21.15.82:0/3411981378 >> 172.21.15.82:0/3411981378 conn(0x56257e74b5c0 msgr2=0x56257e74acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 -- 172.21.15.82:0/3411981378 shutdown_connections 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 -- 172.21.15.82:0/3411981378 wait complete. 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 -- start start 2024-03-14T12:18:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.834+0000 7fad1b39e740 1 Processor -- start 2024-03-14T12:18:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e750dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e7864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad1b39e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad1b39e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257e74d080 con 0x56257e7697b0 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10141c60 con 0x56257e749e00 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad1b39e740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x56257e68d1b0 con 0x56257e74dd70 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad1b39e740 10 client.0 _ll_register_callbacks cb 0x7fad1006c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad16c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad16c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55796/0 (socket says 172.21.15.82:55796) 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.835+0000 7fad16c85640 1 -- 172.21.15.82:0/815785211 learned_addr learned my addr 172.21.15.82:0/815785211 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad15c83640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e7864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16484640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e750dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16c85640 1 -- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 msgr2=0x56257e7864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16c85640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e7864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16c85640 1 -- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 msgr2=0x56257e750dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16c85640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e750dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16c85640 1 -- 172.21.15.82:0/815785211 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56257e786a10 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16484640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e750dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad15c83640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e7864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad16c85640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76a630 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fad08020040 tx=0x7fad08009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2859463b58bf2abd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7facf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7facf17fa640 1 -- 172.21.15.82:0/815785211 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0802d070 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7facf17fa640 1 -- 172.21.15.82:0/815785211 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fad08004440 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad08023070 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7facf17fa640 1 -- 172.21.15.82:0/815785211 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad08038050 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 10 client.7071 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x56257e751390 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7facf17fa640 1 -- 172.21.15.82:0/815785211 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad08023070 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7facf17fa640 1 -- 172.21.15.82:0/815785211 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fad0803b020 con 0x56257e7697b0 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 10 client.7071 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 10 client.7071 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 20 client.7071 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76ab70 0x56257e7d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 4 client.7071 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.836+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x56257e68f750 con 0x56257e76ab70 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.837+0000 7fad16484640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76ab70 0x56257e7d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.837+0000 7facf17fa640 10 client.7071 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.837+0000 7fad16484640 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76ab70 0x56257e7d7320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c629f4dcc39a8e server_cookie=43d462e4ca00cfb9 in_seq=0 out_seq=0 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.837+0000 7facf17fa640 1 -- 172.21.15.82:0/815785211 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25192 (crc 0 0 0) 0x56257e68f750 con 0x56257e76ab70 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.837+0000 7facf17fa640 10 client.7071 handle_command_reply: tid=0 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.837+0000 7fad1b39e740 1 client.7071 shutdown 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 msgr2=0x56257e76a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76a630 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fad08020040 tx=0x7fad08009570 comp rx=0 tx=0).stop 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 shutdown_connections 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56257e7697b0 0x56257e76a630 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x56257e76ab70 0x56257e7d7320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56257e74dd70 0x56257e7864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 --2- 172.21.15.82:0/815785211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56257e749e00 0x56257e750dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7facf17fa640 0 client.7071 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 >> 172.21.15.82:0/815785211 conn(0x56257e74b5c0 msgr2=0x56257e71d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 shutdown_connections 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 1 -- 172.21.15.82:0/815785211 wait complete. 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad1b39e740 20 client.7071 trim_cache size 0 max 16384 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:12.845 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.846 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:18:12.847 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 17, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13847, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 6430, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.004320212, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000427989, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002025070, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000387684, 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:12.848 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17, 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002804180, 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000164951 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:18:12.849 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 36, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 49, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 16860, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 118904, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008193976, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001669023, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005711417, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000682123, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 48, 2024-03-14T12:18:12.850 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009912259, 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000206505 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 14, 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000002600, 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000185 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 107696 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.851 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 59, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 54, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 118012, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 173575, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 12, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 8, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008877373, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.002271593, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.006441740, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000578404, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 54, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005103283, 2024-03-14T12:18:12.852 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000094505 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 8, 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000698, 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000087 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112960, 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 18064 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.853 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 51, 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003124936, 2024-03-14T12:18:12.854 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000061273 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000121504, 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000030376 2024-03-14T12:18:12.855 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 5, 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 5, 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:18:12.856 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 12, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 10, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 7, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 3, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:18:12.857 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 10, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 10, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:18:12.858 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:18:12.859 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:18:12.860 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:18:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.862 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "6471" 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 0 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.863 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 3, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 9, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197281, 2024-03-14T12:18:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 4196487, 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009482915, 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003160971 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 6 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:18:12.865 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:18:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 40736, 2024-03-14T12:18:12.869 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 231776 2024-03-14T12:18:12.869 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.869 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.869 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.869 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:18:12.869 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.869 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 5, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 5, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 1, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:12.870 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.871 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.872 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.873 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.877 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.878 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.879 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.880 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 2, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 4, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 4, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.881 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:18:12.882 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:18:12.883 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 222926, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 70, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:18:12.884 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5864, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 96, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 51672, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 186, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:18:12.885 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 47, 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 1076, 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 0, 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 47, 2024-03-14T12:18:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.055998963, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001191467 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 93 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 47, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 36, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 11, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:18:12.887 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 2, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 8, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 3, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 1, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 1, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 5, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:18:12.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 24, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:18:12.889 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:18:12.890 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 73, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 3, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 1, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 48, 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.891 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.892 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:18:12.893 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:12.894 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 108, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 135286, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 108, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 108, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 135286, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.895 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:12.896 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 3251, 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 3251, 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.897 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 47, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 47, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.898 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:12.899 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x560d75e68a00": [ 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:12.900 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 538, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 538, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:12.901 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:12.902 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:12.902 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.902 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.902 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:12.902 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:18:12.902 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7facf406bbd0 msgr2=0x7facf406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7facf406bbd0 0x7facf406e090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fad0c000f60 tx=0x7fad0c0079d0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100706c0 msgr2=0x7fad1013ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100706c0 0x7fad1013ecb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fad0002b9c0 tx=0x7fad00004730 comp rx=0 tx=0).stop 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 shutdown_connections 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7facf406bbd0 0x7facf406e090 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fad1015ace0 0x7fad10152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100706c0 0x7fad1013ecb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 --2- 172.21.15.82:0/1183730901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad1006c440 0x7fad100cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.838+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 >> 172.21.15.82:0/1183730901 conn(0x7fad100cf910 msgr2=0x7fad10153280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.839+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 shutdown_connections 2024-03-14T12:18:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:12.839+0000 7fad186e8640 1 -- 172.21.15.82:0/1183730901 wait complete. 2024-03-14T12:18:13.906 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-14T12:18:13.906 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 counter dump 2024-03-14T12:18:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.018+0000 7fefa1a0c640 1 Processor -- start 2024-03-14T12:18:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.018+0000 7fefa1a0c640 1 -- start start 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fefa1a0c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fef9c15bd10 0x7fef9c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fefa1a0c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c160920 con 0x7fef9c06e9e0 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fefa1a0c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c160360 con 0x7fef9c0b5090 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fefa1a0c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c0af6a0 con 0x7fef9c15bd10 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fef9affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fef9c15bd10 0x7fef9c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fef9affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fef9c15bd10 0x7fef9c15f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55800/0 (socket says 172.21.15.82:55800) 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fef9a7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fef9c06e9e0 0x7fef9c0b4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44734/0 (socket says 172.21.15.82:44734) 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.019+0000 7fef9affd640 1 -- 172.21.15.82:0/613484485 learned_addr learned my addr 172.21.15.82:0/613484485 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.020+0000 7fef9affd640 1 -- 172.21.15.82:0/613484485 >> v1:172.21.15.164:6789/0 conn(0x7fef9c0b5090 legacy=0x7fef9c0afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.020+0000 7fef9affd640 1 -- 172.21.15.82:0/613484485 >> v1:172.21.15.82:6789/0 conn(0x7fef9c06e9e0 legacy=0x7fef9c0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.020+0000 7fef99ffb640 1 -- 172.21.15.82:0/613484485 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428826077 0 0) 0x7fef9c160920 con 0x7fef9c06e9e0 2024-03-14T12:18:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.020+0000 7fef9affd640 1 -- 172.21.15.82:0/613484485 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c160640 con 0x7fef9c15bd10 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.020+0000 7fef9affd640 1 --2- 172.21.15.82:0/613484485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c15bd10 0x7fef9c15f9f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009e50 tx=0x7fef8c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565e10f099dd5635 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.020+0000 7fef99ffb640 1 -- 172.21.15.82:0/613484485 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c02cb20 con 0x7fef9c15bd10 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fef99ffb640 1 -- 172.21.15.82:0/613484485 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c02ccc0 con 0x7fef9c15bd10 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fef99ffb640 1 -- 172.21.15.82:0/613484485 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c034610 con 0x7fef9c15bd10 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fefa1a0c640 1 -- 172.21.15.82:0/613484485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c15bd10 msgr2=0x7fef9c15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/613484485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c15bd10 0x7fef9c15f9f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009e50 tx=0x7fef8c02bf80 comp rx=0 tx=0).stop 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fefa1a0c640 1 -- 172.21.15.82:0/613484485 shutdown_connections 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/613484485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c15bd10 0x7fef9c15f9f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fefa1a0c640 1 -- 172.21.15.82:0/613484485 >> 172.21.15.82:0/613484485 conn(0x7fef9c09a9e0 msgr2=0x7fef9c09adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fefa1a0c640 1 -- 172.21.15.82:0/613484485 shutdown_connections 2024-03-14T12:18:14.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.021+0000 7fefa1a0c640 1 -- 172.21.15.82:0/613484485 wait complete. 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.022+0000 7fefa1a0c640 1 Processor -- start 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.022+0000 7fefa1a0c640 1 -- start start 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fefa1a0c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef9c06e9e0 0x7fef9c14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fefa1a0c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c0b5090 0x7fef9c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fefa1a0c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef9c15bd10 0x7fef9c1f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fefa1a0c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1600b0 con 0x7fef9c0b5090 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fefa1a0c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15ff30 con 0x7fef9c06e9e0 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fefa1a0c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15d100 con 0x7fef9c15bd10 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c0b5090 0x7fef9c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c0b5090 0x7fef9c15af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55816/0 (socket says 172.21.15.82:55816) 2024-03-14T12:18:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.023+0000 7fef9a7fc640 1 -- 172.21.15.82:0/894313330 learned_addr learned my addr 172.21.15.82:0/894313330 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef9a7fc640 1 -- 172.21.15.82:0/894313330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef9c15bd10 msgr2=0x7fef9c1f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef9c15bd10 0x7fef9c1f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef9a7fc640 1 -- 172.21.15.82:0/894313330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef9c06e9e0 msgr2=0x7fef9c14dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef9c06e9e0 0x7fef9c14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef9a7fc640 1 -- 172.21.15.82:0/894313330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c1f2540 con 0x7fef9c0b5090 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c0b5090 0x7fef9c15af00 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c4d0 tx=0x7fef9000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef83fff640 1 -- 172.21.15.82:0/894313330 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90018070 con 0x7fef9c0b5090 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef83fff640 1 -- 172.21.15.82:0/894313330 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90007500 con 0x7fef9c0b5090 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c1f2820 con 0x7fef9c0b5090 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fef83fff640 1 -- 172.21.15.82:0/894313330 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90004a20 con 0x7fef9c0b5090 2024-03-14T12:18:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.024+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c14e120 con 0x7fef9c0b5090 2024-03-14T12:18:14.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.026+0000 7fef83fff640 1 -- 172.21.15.82:0/894313330 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fef9001b070 con 0x7fef9c0b5090 2024-03-14T12:18:14.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.026+0000 7fef83fff640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8806bbd0 0x7fef8806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.026+0000 7fef83fff640 1 -- 172.21.15.82:0/894313330 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fef90089e30 con 0x7fef9c0b5090 2024-03-14T12:18:14.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.026+0000 7fef9affd640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8806bbd0 0x7fef8806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.027+0000 7fef9affd640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8806bbd0 0x7fef8806e090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fef8c035000 tx=0x7fef8c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.027+0000 7fefa45df740 1 Processor -- start 2024-03-14T12:18:14.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.027+0000 7fefa45df740 1 -- start start 2024-03-14T12:18:14.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fefa45df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d6229f210 0x557d622ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fefa45df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6226c360 con 0x557d622a2ae0 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d62208ec0 con 0x557d6229f210 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d621b1d30 con 0x557d622eca60 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55818/0 (socket says 172.21.15.82:55818) 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 -- 172.21.15.82:0/1036700233 learned_addr learned my addr 172.21.15.82:0/1036700233 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9b7fe640 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d6229f210 0x557d622ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9affd640 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 -- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 msgr2=0x557d622a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 -- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d6229f210 msgr2=0x557d622ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d6229f210 0x557d622ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 -- 172.21.15.82:0/1036700233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d621b0bc0 con 0x557d622a2ae0 2024-03-14T12:18:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9affd640 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622a45d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fef90054620 tx=0x7fef90054590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f2135c59e45083 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.028+0000 7fef817fa640 1 -- 172.21.15.82:0/1036700233 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900564c0 con 0x557d622a2ae0 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fef817fa640 1 -- 172.21.15.82:0/1036700233 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90054830 con 0x557d622a2ae0 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fef817fa640 1 -- 172.21.15.82:0/1036700233 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90004dc0 con 0x557d622a2ae0 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 -- 172.21.15.82:0/1036700233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 msgr2=0x557d622a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622a45d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fef90054620 tx=0x7fef90054590 comp rx=0 tx=0).stop 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 -- 172.21.15.82:0/1036700233 shutdown_connections 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d6229f210 0x557d622ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622a45d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 --2- 172.21.15.82:0/1036700233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 -- 172.21.15.82:0/1036700233 >> 172.21.15.82:0/1036700233 conn(0x557d622a1480 msgr2=0x557d622e9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 -- 172.21.15.82:0/1036700233 shutdown_connections 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.029+0000 7fefa45df740 1 -- 172.21.15.82:0/1036700233 wait complete. 2024-03-14T12:18:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.030+0000 7fefa45df740 1 -- start start 2024-03-14T12:18:14.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.030+0000 7fefa45df740 1 Processor -- start 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622a2ae0 0x557d622b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622eca60 0x557d622ee440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d621a74a0 con 0x557d622eca60 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d62204f60 con 0x557d622a2ae0 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d622a2ff0 con 0x557d622eff30 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 10 client.0 _ll_register_callbacks cb 0x557d62202d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622eca60 0x557d622ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622eca60 0x557d622ee440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55820/0 (socket says 172.21.15.82:55820) 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 -- 172.21.15.82:0/4288860947 learned_addr learned my addr 172.21.15.82:0/4288860947 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9affd640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622a2ae0 0x557d622b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9b7fe640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 -- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 msgr2=0x557d622cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 -- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622a2ae0 msgr2=0x557d622b9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622a2ae0 0x557d622b9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 -- 172.21.15.82:0/4288860947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d61f1d3f0 con 0x557d622eca60 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9affd640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622a2ae0 0x557d622b9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9b7fe640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622eca60 0x557d622ee440 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fef90088350 tx=0x7fef90000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77f2ece3fba4eab server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef7a7fc640 1 -- 172.21.15.82:0/4288860947 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90086060 con 0x557d622eca60 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fef7a7fc640 1 -- 172.21.15.82:0/4288860947 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fef9005e4d0 con 0x557d622eca60 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.031+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef90054830 con 0x557d622eca60 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fef7a7fc640 1 -- 172.21.15.82:0/4288860947 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef90057610 con 0x557d622eca60 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 10 client.7086 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x557d61fe77d0 con 0x557d622eca60 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fef7a7fc640 1 -- 172.21.15.82:0/4288860947 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90054830 con 0x557d622eca60 2024-03-14T12:18:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fef7a7fc640 1 -- 172.21.15.82:0/4288860947 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fef900577b0 con 0x557d622eca60 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 10 client.7086 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 10 client.7086 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 20 client.7086 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d622faf10 0x557d622fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 4 client.7086 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d62216400 con 0x557d622faf10 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fef9affd640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d622faf10 0x557d622fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fef9affd640 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d622faf10 0x557d622fb310 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04ff97363c7fe86 server_cookie=4780d2cb171c1f9f in_seq=0 out_seq=0 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.032+0000 7fef7a7fc640 10 client.7086 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fef7a7fc640 1 -- 172.21.15.82:0/4288860947 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d62216400 con 0x557d622faf10 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fef7a7fc640 10 client.7086 handle_command_reply: tid=0 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 client.7086 shutdown 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622eca60 msgr2=0x557d622ee440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622eca60 0x557d622ee440 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fef90088350 tx=0x7fef90000ed0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 shutdown_connections 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d622faf10 0x557d622fb310 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622eca60 0x557d622ee440 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 --2- 172.21.15.82:0/4288860947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622a2ae0 0x557d622b9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fef7a7fc640 0 client.7086 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 >> 172.21.15.82:0/4288860947 conn(0x557d6229f210 msgr2=0x557d622a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 shutdown_connections 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 1 -- 172.21.15.82:0/4288860947 wait complete. 2024-03-14T12:18:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.033+0000 7fefa45df740 20 client.7086 trim_cache size 0 max 16384 2024-03-14T12:18:14.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 Processor -- start 2024-03-14T12:18:14.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 -- start start 2024-03-14T12:18:14.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eff30 0x557d622a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1fdf80 con 0x557d622a2ae0 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f5e40 con 0x557d622eff30 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.045+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f5240 con 0x557d622eca60 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55834/0 (socket says 172.21.15.82:55834) 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 -- 172.21.15.82:0/3045975925 learned_addr learned my addr 172.21.15.82:0/3045975925 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 -- 172.21.15.82:0/3045975925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 msgr2=0x557d622cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 --2- 172.21.15.82:0/3045975925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 -- 172.21.15.82:0/3045975925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eff30 msgr2=0x557d622a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 --2- 172.21.15.82:0/3045975925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eff30 0x557d622a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 -- 172.21.15.82:0/3045975925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d6230dba0 con 0x557d622a2ae0 2024-03-14T12:18:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef9affd640 1 --2- 172.21.15.82:0/3045975925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622ccc50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009f70 tx=0x7fef8c02cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62604d97378c5fcd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef80ff9640 1 -- 172.21.15.82:0/3045975925 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c054070 con 0x557d622a2ae0 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef80ff9640 1 -- 172.21.15.82:0/3045975925 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c040b10 con 0x557d622a2ae0 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.046+0000 7fef80ff9640 1 -- 172.21.15.82:0/3045975925 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c04f0d0 con 0x557d622a2ae0 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 -- 172.21.15.82:0/3045975925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 msgr2=0x557d622ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 --2- 172.21.15.82:0/3045975925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622ccc50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009f70 tx=0x7fef8c02cf90 comp rx=0 tx=0).stop 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 -- 172.21.15.82:0/3045975925 shutdown_connections 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 --2- 172.21.15.82:0/3045975925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eff30 0x557d622a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 --2- 172.21.15.82:0/3045975925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eca60 0x557d622cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 --2- 172.21.15.82:0/3045975925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d622ccc50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 -- 172.21.15.82:0/3045975925 >> 172.21.15.82:0/3045975925 conn(0x557d6229f210 msgr2=0x557d622ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 -- 172.21.15.82:0/3045975925 shutdown_connections 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 -- 172.21.15.82:0/3045975925 wait complete. 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 -- start start 2024-03-14T12:18:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.047+0000 7fefa45df740 1 Processor -- start 2024-03-14T12:18:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fefa45df740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d6229f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eca60 0x557d6229fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fefa45df740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fefa45df740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d622a5e50 con 0x557d622a2ae0 2024-03-14T12:18:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f5840 con 0x557d622eca60 2024-03-14T12:18:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fefa45df740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d622ea480 con 0x557d622eff30 2024-03-14T12:18:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fefa45df740 10 client.0 _ll_register_callbacks cb 0x7fef900544e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fef9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d6229f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fef9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d6229f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55842/0 (socket says 172.21.15.82:55842) 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.048+0000 7fef9affd640 1 -- 172.21.15.82:0/1547674885 learned_addr learned my addr 172.21.15.82:0/1547674885 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef9affd640 1 -- 172.21.15.82:0/1547674885 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 msgr2=0x557d622eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef9affd640 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef9affd640 1 -- 172.21.15.82:0/1547674885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eca60 msgr2=0x557d6229fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef9affd640 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eca60 0x557d6229fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef9affd640 1 -- 172.21.15.82:0/1547674885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d622f1910 con 0x557d622a2ae0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef9affd640 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d6229f5f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fef8c02c590 tx=0x7fef8c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e74ac8a62071d34 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef7a7fc640 1 -- 172.21.15.82:0/1547674885 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c054070 con 0x557d622a2ae0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef7a7fc640 1 -- 172.21.15.82:0/1547674885 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fef8c003c60 con 0x557d622a2ae0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef8c040b10 con 0x557d622a2ae0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef7a7fc640 1 -- 172.21.15.82:0/1547674885 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8c063050 con 0x557d622a2ae0 2024-03-14T12:18:14.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 10 client.7092 fetch_fsmap learned FSMap version 40 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x557d622ef110 con 0x557d622a2ae0 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef7a7fc640 1 -- 172.21.15.82:0/1547674885 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c040b10 con 0x557d622a2ae0 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef7a7fc640 1 -- 172.21.15.82:0/1547674885 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fef8c066020 con 0x557d622a2ae0 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 10 client.7092 fetch_fsmap finished waiting for FSMap version 40 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 10 client.7092 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 20 client.7092 populate_metadata read hostname 'smithi082' 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d62354ed0 0x557d622eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 4 client.7092 mds_command: new command op to 6471 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 --> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x557d62210750 con 0x557d62354ed0 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.049+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d62354ed0 0x557d622eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fef9a7fc640 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d62354ed0 0x557d622eb520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd71d30f4204c7d server_cookie=7066a49783e6d5d5 in_seq=0 out_seq=0 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fef7a7fc640 10 client.7092 ms_handle_connect on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fef7a7fc640 1 -- 172.21.15.82:0/1547674885 <== mds.0 v2:172.21.15.82:6836/2268648862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25990 (crc 0 0 0) 0x557d62210750 con 0x557d62354ed0 2024-03-14T12:18:14.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fef7a7fc640 10 client.7092 handle_command_reply: tid=0 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 client.7092 shutdown 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 msgr2=0x557d6229f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d6229f5f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fef8c02c590 tx=0x7fef8c02e070 comp rx=0 tx=0).stop 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 shutdown_connections 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:6836/2268648862,v1:172.21.15.82:6837/2268648862] conn(0x557d62354ed0 0x557d622eb520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x557d622eff30 0x557d622eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557d622eca60 0x557d6229fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 --2- 172.21.15.82:0/1547674885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d622a2ae0 0x557d6229f5f0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fef7a7fc640 0 client.7092 ms_handle_reset on v2:172.21.15.82:6836/2268648862 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 >> 172.21.15.82:0/1547674885 conn(0x557d6229f210 msgr2=0x557d622ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 shutdown_connections 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 1 -- 172.21.15.82:0/1547674885 wait complete. 2024-03-14T12:18:14.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.050+0000 7fefa45df740 20 client.7092 trim_cache size 0 max 16384 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker": [ 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "0" 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "1" 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:14.058 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "2" 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_connection_idle_timeouts": 0 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": [ 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 17, 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19, 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13847, 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 6430, 2024-03-14T12:18:14.059 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.004410842, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000427989, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002025070, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000387684, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002804180, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000164951 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.060 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": [ 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 37, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 49, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 17278, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 118904, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-14T12:18:14.061 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.008281693, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.001701405, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.005732958, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000713848, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 48, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009912259, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000206505 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 15, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000002813, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000187 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.062 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 107696 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": [ 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 61, 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 56, 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 118304, 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 218036, 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 14, 2024-03-14T12:18:14.063 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.009510883, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.002343085, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.006940460, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000578404, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 56, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005123254, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000091486 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000867, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000086 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 112960, 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 18064 2024-03-14T12:18:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": [ 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": [ 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.065 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 51, 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003124936, 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000061273 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": [ 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-14T12:18:14.066 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000121504, 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000030376 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": [ 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 5, 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 5, 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:14.067 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 12, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 10, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 10, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 7, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 3, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 10, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-14T12:18:14.068 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 10, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 10, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-14T12:18:14.069 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-14T12:18:14.070 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": [ 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.071 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-14T12:18:14.072 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics": [ 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.073 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "id": "6471" 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "num_clients": 1 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_client_metrics-cephfs": [ 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": { 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "client": "client.6136", 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "rank": "0" 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.074 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_hits": 41, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_miss": 0, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_read_latency": 0.000000000, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_write_latency": 0.000000000, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_metadata_latency": 0.000000000, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_hits": 0, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_lease_miss": 0, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_files": 0, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "opened_inodes": 1, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "pinned_icaps": 1, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "total_inodes": 1, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_ops": 0, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "total_read_size": 0, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_ops": 0, 2024-03-14T12:18:14.075 INFO:teuthology.orchestra.run.smithi082.stdout: "total_write_size": 0 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": [ 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 3, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 9, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-14T12:18:14.076 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 2, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 4197281, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 4196487, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009482915, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003160971 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 6 2024-03-14T12:18:14.077 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": [ 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 0, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 12, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 12, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 0, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 10, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 10, 2024-03-14T12:18:14.078 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 0, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 40736, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 231776 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": [ 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 5, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 5, 2024-03-14T12:18:14.079 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 1, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000542753, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000108550 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.081 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.087 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.088 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.089 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": [ 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 2, 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 4, 2024-03-14T12:18:14.090 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 4, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": [ 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-14T12:18:14.091 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-14T12:18:14.092 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-14T12:18:14.093 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 222926, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 70, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 5864, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 96, 2024-03-14T12:18:14.094 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 51672, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 186, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": [ 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.095 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 47, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 1076, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 0, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 47, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.055998963, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001191467 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-14T12:18:14.096 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 47, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 93 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 47, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 36, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 11, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 2, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 8, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 3, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 1, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-14T12:18:14.097 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 1, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 5, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 24, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-14T12:18:14.098 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-14T12:18:14.099 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 73, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 3, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 7, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 7, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 1, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 48, 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-03-14T12:18:14.100 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": [ 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": [ 2024-03-14T12:18:14.101 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": [ 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-14T12:18:14.102 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 111, 2024-03-14T12:18:14.103 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 135726, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 111, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 111, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 135726, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.104 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": [ 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 3251, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 3251, 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:14.105 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": [ 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:14.106 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 47, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 47, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 47, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 47, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.107 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": [ 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:14.108 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x560d75e68a00": [ 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "labels": {}, 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "counters": { 2024-03-14T12:18:14.109 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 538, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 538, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-14T12:18:14.110 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.111 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.111 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-14T12:18:14.111 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-14T12:18:14.111 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:14.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8806bbd0 msgr2=0x7fef8806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8806bbd0 0x7fef8806e090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fef8c035000 tx=0x7fef8c02e000 comp rx=0 tx=0).stop 2024-03-14T12:18:14.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c0b5090 msgr2=0x7fef9c15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c0b5090 0x7fef9c15af00 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c4d0 tx=0x7fef9000c990 comp rx=0 tx=0).stop 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 shutdown_connections 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fef8806bbd0 0x7fef8806e090 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef9c15bd10 0x7fef9c1f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c0b5090 0x7fef9c15af00 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 --2- 172.21.15.82:0/894313330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef9c06e9e0 0x7fef9c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 >> 172.21.15.82:0/894313330 conn(0x7fef9c09a9e0 msgr2=0x7fef9c156890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 shutdown_connections 2024-03-14T12:18:14.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.051+0000 7fefa1a0c640 1 -- 172.21.15.82:0/894313330 wait complete. 2024-03-14T12:18:14.114 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T12:18:14.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:18:14.159 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T12:18:14.208 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T12:18:14.209 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:18:14.209 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:14.224 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:18:14.224 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:14.279 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:18:14.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.420+0000 7f4891ec6640 1 Processor -- start 2024-03-14T12:18:14.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.420+0000 7f4891ec6640 1 -- start start 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.420+0000 7f4891ec6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f488c15e670 0x7f488c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.420+0000 7f4891ec6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c143f60 0x7f488c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.420+0000 7f4891ec6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c15ebe0 con 0x7f488c071740 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.420+0000 7f4891ec6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c15eec0 con 0x7f488c15e670 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.420+0000 7f4891ec6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c15f1a0 con 0x7f488c143f60 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f4882ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f488c15e670 0x7f488c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f4882ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f488c15e670 0x7f488c0b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55852/0 (socket says 172.21.15.82:55852) 2024-03-14T12:18:14.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f4882ffd640 1 -- 172.21.15.82:0/1232365926 learned_addr learned my addr 172.21.15.82:0/1232365926 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f488bfff640 1 --2- 172.21.15.82:0/1232365926 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c143f60 0x7f488c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f4882ffd640 1 -- 172.21.15.82:0/1232365926 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c143f60 msgr2=0x7f488c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f4882ffd640 1 --2- 172.21.15.82:0/1232365926 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c143f60 0x7f488c15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f4882ffd640 1 -- 172.21.15.82:0/1232365926 >> v1:172.21.15.164:6789/0 conn(0x7f488c071740 legacy=0x7f488c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.421+0000 7f4882ffd640 1 -- 172.21.15.82:0/1232365926 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c15f480 con 0x7f488c15e670 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4882ffd640 1 --2- 172.21.15.82:0/1232365926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c15e670 0x7f488c0b2940 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f48740098a0 tx=0x7f487402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b599c7142f5ab04 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f488affd640 1 -- 172.21.15.82:0/1232365926 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874034070 con 0x7f488c15e670 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f488affd640 1 -- 172.21.15.82:0/1232365926 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874030840 con 0x7f488c15e670 2024-03-14T12:18:14.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f488affd640 1 -- 172.21.15.82:0/1232365926 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874030c10 con 0x7f488c15e670 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 -- 172.21.15.82:0/1232365926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c15e670 msgr2=0x7f488c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 --2- 172.21.15.82:0/1232365926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c15e670 0x7f488c0b2940 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f48740098a0 tx=0x7f487402b980 comp rx=0 tx=0).stop 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 -- 172.21.15.82:0/1232365926 shutdown_connections 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 --2- 172.21.15.82:0/1232365926 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c143f60 0x7f488c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 --2- 172.21.15.82:0/1232365926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c15e670 0x7f488c0b2940 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 -- 172.21.15.82:0/1232365926 >> 172.21.15.82:0/1232365926 conn(0x7f488c0b1c70 msgr2=0x7f488c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 -- 172.21.15.82:0/1232365926 shutdown_connections 2024-03-14T12:18:14.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.422+0000 7f4891ec6640 1 -- 172.21.15.82:0/1232365926 wait complete. 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.423+0000 7f4891ec6640 1 Processor -- start 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4891ec6640 1 -- start start 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4891ec6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c071740 0x7f488c15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4891ec6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c143f60 0x7f488c1f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4891ec6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f488c15e670 0x7f488c139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4891ec6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c144670 con 0x7f488c143f60 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4891ec6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c15b080 con 0x7f488c15e670 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4891ec6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c1447f0 con 0x7f488c071740 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4882ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c143f60 0x7f488c1f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4882ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c143f60 0x7f488c1f3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55860/0 (socket says 172.21.15.82:55860) 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.424+0000 7f4882ffd640 1 -- 172.21.15.82:0/45864355 learned_addr learned my addr 172.21.15.82:0/45864355 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4882ffd640 1 -- 172.21.15.82:0/45864355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c071740 msgr2=0x7f488c15bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4882ffd640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c071740 0x7f488c15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4882ffd640 1 -- 172.21.15.82:0/45864355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f488c15e670 msgr2=0x7f488c139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f488bfff640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f488c15e670 0x7f488c139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4882ffd640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f488c15e670 0x7f488c139ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4882ffd640 1 -- 172.21.15.82:0/45864355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c13a530 con 0x7f488c143f60 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4882ffd640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c143f60 0x7f488c1f3ee0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f48740099c0 tx=0x7f48740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4888ff9640 1 -- 172.21.15.82:0/45864355 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874034040 con 0x7f488c143f60 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4888ff9640 1 -- 172.21.15.82:0/45864355 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874030640 con 0x7f488c143f60 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c13a740 con 0x7f488c143f60 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4888ff9640 1 -- 172.21.15.82:0/45864355 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740423f0 con 0x7f488c143f60 2024-03-14T12:18:14.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.425+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c13a9e0 con 0x7f488c143f60 2024-03-14T12:18:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.426+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c04f050 con 0x7f488c143f60 2024-03-14T12:18:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.427+0000 7f4888ff9640 1 -- 172.21.15.82:0/45864355 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4874037070 con 0x7f488c143f60 2024-03-14T12:18:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.427+0000 7f4888ff9640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f487006bc00 0x7f487006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.427+0000 7f4888ff9640 1 -- 172.21.15.82:0/45864355 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f48740ace60 con 0x7f488c143f60 2024-03-14T12:18:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.427+0000 7f488b7fe640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f487006bc00 0x7f487006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.427+0000 7f488b7fe640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f487006bc00 0x7f487006e0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f487c000f60 tx=0x7f487c0079d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.432+0000 7f4888ff9640 1 -- 172.21.15.82:0/45864355 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f487407b980 con 0x7f488c143f60 2024-03-14T12:18:14.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.606+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f488c0d07b0 con 0x7f488c143f60 2024-03-14T12:18:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.607+0000 7f4888ff9640 1 -- 172.21.15.82:0/45864355 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+17008 (secure 0 0 0) 0x7f487407b280 con 0x7f488c143f60 2024-03-14T12:18:14.608 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:14.608 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":73,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:18:05.349077+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6834/4033072140":"2024-03-15T12:17:15.041576+0000","192.168.0.1:0/895829753":"2024-03-14T13:18:05.110010+0000","172.21.15.164:0/1140720987":"2024-03-14T13:18:05.110143+0000","172.21.15.164:6832/4033072140":"2024-03-15T12:17:15.041576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:18:14.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.609+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f487006bc00 msgr2=0x7f487006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.609+0000 7f4891ec6640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f487006bc00 0x7f487006e0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f487c000f60 tx=0x7f487c0079d0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.609+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c143f60 msgr2=0x7f488c1f3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.609+0000 7f4891ec6640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c143f60 0x7f488c1f3ee0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f48740099c0 tx=0x7f48740046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 shutdown_connections 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f488c15e670 0x7f488c139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488c143f60 0x7f488c1f3ee0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f487006bc00 0x7f487006e0c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 --2- 172.21.15.82:0/45864355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f488c071740 0x7f488c15bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 >> 172.21.15.82:0/45864355 conn(0x7f488c0b1c70 msgr2=0x7f488c0b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 shutdown_connections 2024-03-14T12:18:14.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.610+0000 7f4891ec6640 1 -- 172.21.15.82:0/45864355 wait complete. 2024-03-14T12:18:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:57.716705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:58.458303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:18:14.622 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:18:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.725+0000 7f6f2f553640 1 Processor -- start 2024-03-14T12:18:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.725+0000 7f6f2f553640 1 -- start start 2024-03-14T12:18:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2f553640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f280d02d0 0x7f6f2815e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2f553640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f280b3450 con 0x7f6f280cf3f0 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2f553640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2815f4b0 con 0x7f6f280b41d0 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2f553640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f280cf7d0 con 0x7f6f280d02d0 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2caee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f280d02d0 0x7f6f2815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2caee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f280d02d0 0x7f6f2815e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55864/0 (socket says 172.21.15.82:55864) 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2d2ef640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6f280cf3f0 0x7f6f280cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44738/0 (socket says 172.21.15.82:44738) 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2caee640 1 -- 172.21.15.82:0/2512169353 learned_addr learned my addr 172.21.15.82:0/2512169353 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2caee640 1 -- 172.21.15.82:0/2512169353 >> v1:172.21.15.164:6789/0 conn(0x7f6f280b41d0 legacy=0x7f6f2815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2caee640 1 -- 172.21.15.82:0/2512169353 >> v1:172.21.15.82:6789/0 conn(0x7f6f280cf3f0 legacy=0x7f6f280cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f2caee640 1 -- 172.21.15.82:0/2512169353 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f280b36f0 con 0x7f6f280d02d0 2024-03-14T12:18:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.726+0000 7f6f1ffff640 1 -- 172.21.15.82:0/2512169353 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191157852 0 0) 0x7f6f280b3450 con 0x7f6f280cf3f0 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f2caee640 1 --2- 172.21.15.82:0/2512169353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f2815e780 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009870 tx=0x7f6f1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822fbdb88fe491ea server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f1ffff640 1 -- 172.21.15.82:0/2512169353 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18035020 con 0x7f6f280d02d0 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f1ffff640 1 -- 172.21.15.82:0/2512169353 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f18031870 con 0x7f6f280d02d0 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f1ffff640 1 -- 172.21.15.82:0/2512169353 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18031c40 con 0x7f6f280d02d0 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f2f553640 1 -- 172.21.15.82:0/2512169353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 msgr2=0x7f6f2815e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f2f553640 1 --2- 172.21.15.82:0/2512169353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f2815e780 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009870 tx=0x7f6f1802b910 comp rx=0 tx=0).stop 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f2f553640 1 -- 172.21.15.82:0/2512169353 shutdown_connections 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f2f553640 1 --2- 172.21.15.82:0/2512169353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f2815e780 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.727+0000 7f6f2f553640 1 -- 172.21.15.82:0/2512169353 >> 172.21.15.82:0/2512169353 conn(0x7f6f2806aef0 msgr2=0x7f6f2806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.728+0000 7f6f2f553640 1 -- 172.21.15.82:0/2512169353 shutdown_connections 2024-03-14T12:18:14.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.728+0000 7f6f2f553640 1 -- 172.21.15.82:0/2512169353 wait complete. 2024-03-14T12:18:14.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 Processor -- start 2024-03-14T12:18:14.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 -- start start 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f280b41d0 0x7f6f280b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f280cf3f0 0x7f6f28146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f28147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280707a0 con 0x7f6f280d02d0 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280b3cb0 con 0x7f6f280b41d0 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2f553640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28070920 con 0x7f6f280cf3f0 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2daf0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f28147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2daf0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f28147530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55870/0 (socket says 172.21.15.82:55870) 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.729+0000 7f6f2daf0640 1 -- 172.21.15.82:0/899912257 learned_addr learned my addr 172.21.15.82:0/899912257 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2d2ef640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f280b41d0 0x7f6f280b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2daf0640 1 -- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f280cf3f0 msgr2=0x7f6f28146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2daf0640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f280cf3f0 0x7f6f28146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2daf0640 1 -- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f280b41d0 msgr2=0x7f6f280b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2daf0640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f280b41d0 0x7f6f280b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2daf0640 1 -- 172.21.15.82:0/899912257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2815dce0 con 0x7f6f280d02d0 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2d2ef640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f280b41d0 0x7f6f280b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2daf0640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f28147530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000c780 tx=0x7f6f2000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f1dffb640 1 -- 172.21.15.82:0/899912257 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20004b60 con 0x7f6f280d02d0 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f1dffb640 1 -- 172.21.15.82:0/899912257 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f20004d00 con 0x7f6f280d02d0 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2815df50 con 0x7f6f280d02d0 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f1dffb640 1 -- 172.21.15.82:0/899912257 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000aa90 con 0x7f6f280d02d0 2024-03-14T12:18:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.730+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f280ce570 con 0x7f6f280d02d0 2024-03-14T12:18:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.731+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f280cdc10 con 0x7f6f280d02d0 2024-03-14T12:18:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.732+0000 7f6f1dffb640 1 -- 172.21.15.82:0/899912257 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6f2000ac30 con 0x7f6f280d02d0 2024-03-14T12:18:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.732+0000 7f6f1dffb640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0c06bbd0 0x7f6f0c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.732+0000 7f6f1dffb640 1 -- 172.21.15.82:0/899912257 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f6f2008ade0 con 0x7f6f280d02d0 2024-03-14T12:18:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.732+0000 7f6f2d2ef640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0c06bbd0 0x7f6f0c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.732+0000 7f6f2d2ef640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0c06bbd0 0x7f6f0c06e090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000f60 tx=0x7f6f24009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:14.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.736+0000 7f6f1dffb640 1 -- 172.21.15.82:0/899912257 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6f20020070 con 0x7f6f280d02d0 2024-03-14T12:18:14.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.908+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f2804f050 con 0x7f6f280d02d0 2024-03-14T12:18:14.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.908+0000 7f6f1dffb640 1 -- 172.21.15.82:0/899912257 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f200597e0 con 0x7f6f280d02d0 2024-03-14T12:18:14.909 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:18:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.910+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0c06bbd0 msgr2=0x7f6f0c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.910+0000 7f6f2f553640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0c06bbd0 0x7f6f0c06e090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000f60 tx=0x7f6f24009040 comp rx=0 tx=0).stop 2024-03-14T12:18:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.910+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 msgr2=0x7f6f28147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.910+0000 7f6f2f553640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f28147530 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000c780 tx=0x7f6f2000cc40 comp rx=0 tx=0).stop 2024-03-14T12:18:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 shutdown_connections 2024-03-14T12:18:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f280d02d0 0x7f6f28147530 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6f0c06bbd0 0x7f6f0c06e090 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f280cf3f0 0x7f6f28146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 --2- 172.21.15.82:0/899912257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f280b41d0 0x7f6f280b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 >> 172.21.15.82:0/899912257 conn(0x7f6f2806aef0 msgr2=0x7f6f280b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 shutdown_connections 2024-03-14T12:18:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:14.911+0000 7f6f2f553640 1 -- 172.21.15.82:0/899912257 wait complete. 2024-03-14T12:18:14.922 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:18:15.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.025+0000 7fe748ec0640 1 Processor -- start 2024-03-14T12:18:15.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.025+0000 7fe748ec0640 1 -- start start 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.025+0000 7fe748ec0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe7440cfc30 0x7fe74415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe748ec0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe74415ff80 0x7fe7440af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe748ec0640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7441608b0 con 0x7fe74415f3a0 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe748ec0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe744160390 con 0x7fe7440cfc30 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe748ec0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7440d01a0 con 0x7fe74415ff80 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe739d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe7440cfc30 0x7fe74415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe739d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe7440cfc30 0x7fe74415fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55886/0 (socket says 172.21.15.82:55886) 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe739d9b640 1 -- 172.21.15.82:0/2418478655 learned_addr learned my addr 172.21.15.82:0/2418478655 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe739d9b640 1 -- 172.21.15.82:0/2418478655 >> v2:172.21.15.164:3300/0 conn(0x7fe74415ff80 msgr2=0x7fe7440af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe739d9b640 1 --2- 172.21.15.82:0/2418478655 >> v2:172.21.15.164:3300/0 conn(0x7fe74415ff80 0x7fe7440af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe739d9b640 1 -- 172.21.15.82:0/2418478655 >> v1:172.21.15.164:6789/0 conn(0x7fe74415f3a0 legacy=0x7fe744070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.026+0000 7fe739d9b640 1 -- 172.21.15.82:0/2418478655 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440d0480 con 0x7fe7440cfc30 2024-03-14T12:18:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe739d9b640 1 --2- 172.21.15.82:0/2418478655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe74415fa40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe74406b770 tx=0x7fe72c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5499790afd50858 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe741d9b640 1 -- 172.21.15.82:0/2418478655 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c035020 con 0x7fe7440cfc30 2024-03-14T12:18:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe741d9b640 1 -- 172.21.15.82:0/2418478655 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c031870 con 0x7fe7440cfc30 2024-03-14T12:18:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe741d9b640 1 -- 172.21.15.82:0/2418478655 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c031c40 con 0x7fe7440cfc30 2024-03-14T12:18:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 -- 172.21.15.82:0/2418478655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 msgr2=0x7fe74415fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 --2- 172.21.15.82:0/2418478655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe74415fa40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe74406b770 tx=0x7fe72c02b910 comp rx=0 tx=0).stop 2024-03-14T12:18:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 -- 172.21.15.82:0/2418478655 shutdown_connections 2024-03-14T12:18:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 --2- 172.21.15.82:0/2418478655 >> v2:172.21.15.164:3300/0 conn(0x7fe74415ff80 0x7fe7440af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 --2- 172.21.15.82:0/2418478655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe74415fa40 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 -- 172.21.15.82:0/2418478655 >> 172.21.15.82:0/2418478655 conn(0x7fe7440b3680 msgr2=0x7fe7440b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 -- 172.21.15.82:0/2418478655 shutdown_connections 2024-03-14T12:18:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.027+0000 7fe748ec0640 1 -- 172.21.15.82:0/2418478655 wait complete. 2024-03-14T12:18:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 Processor -- start 2024-03-14T12:18:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 -- start start 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe744151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe74415f3a0 0x7fe7441f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe74415ff80 0x7fe7441f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74415f780 con 0x7fe7440cfc30 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7440aeb70 con 0x7fe74415f3a0 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.029+0000 7fe748ec0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7440af5d0 con 0x7fe74415ff80 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe744151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe744151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55902/0 (socket says 172.21.15.82:55902) 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 -- 172.21.15.82:0/1764963045 learned_addr learned my addr 172.21.15.82:0/1764963045 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 -- 172.21.15.82:0/1764963045 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe74415ff80 msgr2=0x7fe7441f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe74415ff80 0x7fe7441f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 -- 172.21.15.82:0/1764963045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe74415f3a0 msgr2=0x7fe7441f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe739d9b640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe74415f3a0 0x7fe7441f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe74415f3a0 0x7fe7441f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 -- 172.21.15.82:0/1764963045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440c53c0 con 0x7fe7440cfc30 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe74259c640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe744151fd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe73400c410 tx=0x7fe73400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe73b7fe640 1 -- 172.21.15.82:0/1764963045 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734017070 con 0x7fe7440cfc30 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe73b7fe640 1 -- 172.21.15.82:0/1764963045 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7340035f0 con 0x7fe7440cfc30 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74415dd50 con 0x7fe7440cfc30 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe73b7fe640 1 -- 172.21.15.82:0/1764963045 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340039e0 con 0x7fe7440cfc30 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.030+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74406d6b0 con 0x7fe7440cfc30 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.031+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74404f050 con 0x7fe7440cfc30 2024-03-14T12:18:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.032+0000 7fe73b7fe640 1 -- 172.21.15.82:0/1764963045 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe73401a070 con 0x7fe7440cfc30 2024-03-14T12:18:15.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.032+0000 7fe73b7fe640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe72406bbd0 0x7fe72406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.032+0000 7fe73b7fe640 1 -- 172.21.15.82:0/1764963045 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe73408a000 con 0x7fe7440cfc30 2024-03-14T12:18:15.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.032+0000 7fe739d9b640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe72406bbd0 0x7fe72406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.033+0000 7fe739d9b640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe72406bbd0 0x7fe72406e090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe72c036000 tx=0x7fe72c03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.036+0000 7fe73b7fe640 1 -- 172.21.15.82:0/1764963045 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe734058a00 con 0x7fe7440cfc30 2024-03-14T12:18:15.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.209+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7440b7ab0 con 0x7fe7440cfc30 2024-03-14T12:18:15.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.209+0000 7fe73b7fe640 1 -- 172.21.15.82:0/1764963045 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7340010c0 con 0x7fe7440cfc30 2024-03-14T12:18:15.210 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:18:15.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe72406bbd0 msgr2=0x7fe72406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe72406bbd0 0x7fe72406e090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe72c036000 tx=0x7fe72c03c040 comp rx=0 tx=0).stop 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 msgr2=0x7fe744151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe744151fd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe73400c410 tx=0x7fe73400c8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 shutdown_connections 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fe72406bbd0 0x7fe72406e090 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe74415ff80 0x7fe7441f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe74415f3a0 0x7fe7441f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 --2- 172.21.15.82:0/1764963045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7440cfc30 0x7fe744151fd0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 >> 172.21.15.82:0/1764963045 conn(0x7fe7440b3680 msgr2=0x7fe7440b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 shutdown_connections 2024-03-14T12:18:15.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.212+0000 7fe748ec0640 1 -- 172.21.15.82:0/1764963045 wait complete. 2024-03-14T12:18:15.223 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:18:15.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.326+0000 7fdec52c6640 1 Processor -- start 2024-03-14T12:18:15.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.326+0000 7fdec52c6640 1 -- start start 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.326+0000 7fdec52c6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdec00cf3f0 0x7fdec00cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.326+0000 7fdec52c6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdec015ee90 0x7fdec00d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.326+0000 7fdec52c6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec00b3450 con 0x7fdec00b41d0 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdec52c6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec00cf7d0 con 0x7fdec00cf3f0 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdec52c6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec015f4b0 con 0x7fdec015ee90 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdebeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdec00cf3f0 0x7fdec00cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdebeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdec00cf3f0 0x7fdec00cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55904/0 (socket says 172.21.15.82:55904) 2024-03-14T12:18:15.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdebeffd640 1 -- 172.21.15.82:0/3576964080 learned_addr learned my addr 172.21.15.82:0/3576964080 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdebeffd640 1 -- 172.21.15.82:0/3576964080 >> v2:172.21.15.164:3300/0 conn(0x7fdec015ee90 msgr2=0x7fdec00d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdebeffd640 1 --2- 172.21.15.82:0/3576964080 >> v2:172.21.15.164:3300/0 conn(0x7fdec015ee90 0x7fdec00d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdebeffd640 1 -- 172.21.15.82:0/3576964080 >> v1:172.21.15.164:6789/0 conn(0x7fdec00b41d0 legacy=0x7fdec015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.327+0000 7fdebeffd640 1 -- 172.21.15.82:0/3576964080 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00b36f0 con 0x7fdec00cf3f0 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdebeffd640 1 --2- 172.21.15.82:0/3576964080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00cf3f0 0x7fdec00cfbc0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdec006bcb0 tx=0x7fdeb002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b495e2b7665a35f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdebe7fc640 1 -- 172.21.15.82:0/3576964080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb003b070 con 0x7fdec00cf3f0 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdebe7fc640 1 -- 172.21.15.82:0/3576964080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb002ee00 con 0x7fdec00cf3f0 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdebe7fc640 1 -- 172.21.15.82:0/3576964080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0036660 con 0x7fdec00cf3f0 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 -- 172.21.15.82:0/3576964080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00cf3f0 msgr2=0x7fdec00cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 --2- 172.21.15.82:0/3576964080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00cf3f0 0x7fdec00cfbc0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdec006bcb0 tx=0x7fdeb002e050 comp rx=0 tx=0).stop 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 -- 172.21.15.82:0/3576964080 shutdown_connections 2024-03-14T12:18:15.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 --2- 172.21.15.82:0/3576964080 >> v2:172.21.15.164:3300/0 conn(0x7fdec015ee90 0x7fdec00d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 --2- 172.21.15.82:0/3576964080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00cf3f0 0x7fdec00cfbc0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 -- 172.21.15.82:0/3576964080 >> 172.21.15.82:0/3576964080 conn(0x7fdec006aef0 msgr2=0x7fdec006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:15.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 -- 172.21.15.82:0/3576964080 shutdown_connections 2024-03-14T12:18:15.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.328+0000 7fdec52c6640 1 -- 172.21.15.82:0/3576964080 wait complete. 2024-03-14T12:18:15.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 Processor -- start 2024-03-14T12:18:15.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 -- start start 2024-03-14T12:18:15.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b41d0 0x7fdec0151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00cf3f0 0x7fdec0153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec015ee90 0x7fdec015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec006d890 con 0x7fdec00b41d0 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00b3cb0 con 0x7fdec015ee90 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdec52c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec006da10 con 0x7fdec00cf3f0 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdebeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b41d0 0x7fdec0151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdebeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b41d0 0x7fdec0151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55910/0 (socket says 172.21.15.82:55910) 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.330+0000 7fdebeffd640 1 -- 172.21.15.82:0/499618298 learned_addr learned my addr 172.21.15.82:0/499618298 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdebf7fe640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec015ee90 0x7fdec015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdeb7fff640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00cf3f0 0x7fdec0153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdebeffd640 1 -- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00cf3f0 msgr2=0x7fdec0153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdebeffd640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00cf3f0 0x7fdec0153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdebeffd640 1 -- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec015ee90 msgr2=0x7fdec015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdebeffd640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec015ee90 0x7fdec015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdebeffd640 1 -- 172.21.15.82:0/499618298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec0153f70 con 0x7fdec00b41d0 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdeb7fff640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00cf3f0 0x7fdec0153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:15.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdebeffd640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b41d0 0x7fdec0151100 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002e090 tx=0x7fdeb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdeb77fe640 1 -- 172.21.15.82:0/499618298 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0046070 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdeb77fe640 1 -- 172.21.15.82:0/499618298 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb0042670 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec0154250 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdeb77fe640 1 -- 172.21.15.82:0/499618298 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb003b040 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.331+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec0154810 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.332+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec00b46f0 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.332+0000 7fdeb77fe640 1 -- 172.21.15.82:0/499618298 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdeb003e040 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.333+0000 7fdeb77fe640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdea006bbd0 0x7fdea006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.333+0000 7fdeb7fff640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdea006bbd0 0x7fdea006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.333+0000 7fdeb77fe640 1 -- 172.21.15.82:0/499618298 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fdeb00aef10 con 0x7fdec00b41d0 2024-03-14T12:18:15.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.333+0000 7fdeb7fff640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdea006bbd0 0x7fdea006e090 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fd0 tx=0x7fdea8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.337+0000 7fdeb77fe640 1 -- 172.21.15.82:0/499618298 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdec00b46f0 con 0x7fdec00b41d0 2024-03-14T12:18:15.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.509+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdec015b6a0 con 0x7fdec00b41d0 2024-03-14T12:18:15.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.509+0000 7fdeb77fe640 1 -- 172.21.15.82:0/499618298 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdec015b6a0 con 0x7fdec00b41d0 2024-03-14T12:18:15.510 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:18:15.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdea006bbd0 msgr2=0x7fdea006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdea006bbd0 0x7fdea006e090 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fd0 tx=0x7fdea8009040 comp rx=0 tx=0).stop 2024-03-14T12:18:15.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b41d0 msgr2=0x7fdec0151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b41d0 0x7fdec0151100 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002e090 tx=0x7fdeb00046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 shutdown_connections 2024-03-14T12:18:15.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec015ee90 0x7fdec015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fdea006bbd0 0x7fdea006e090 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdec00cf3f0 0x7fdec0153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 --2- 172.21.15.82:0/499618298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec00b41d0 0x7fdec0151100 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 >> 172.21.15.82:0/499618298 conn(0x7fdec006aef0 msgr2=0x7fdec00b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:15.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 shutdown_connections 2024-03-14T12:18:15.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.511+0000 7fdec52c6640 1 -- 172.21.15.82:0/499618298 wait complete. 2024-03-14T12:18:15.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T12:18:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.623+0000 7fd61b7f1640 1 Processor -- start 2024-03-14T12:18:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.624+0000 7fd61b7f1640 1 -- start start 2024-03-14T12:18:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.624+0000 7fd61b7f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.624+0000 7fd61b7f1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd61406c7e0 0x7fd61406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.624+0000 7fd61b7f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61406cca0 con 0x7fd6140b4350 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.624+0000 7fd61b7f1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61406cf80 con 0x7fd61406c7e0 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.624+0000 7fd61b7f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61406f1d0 con 0x7fd61415e6b0 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd618d8c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd61406c7e0 0x7fd61406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd61958d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd618d8c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd61406c7e0 0x7fd61406f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:38058/0 (socket says 172.21.15.82:38058) 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd618d8c640 1 -- 172.21.15.82:0/3637882484 learned_addr learned my addr 172.21.15.82:0/3637882484 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd618d8c640 1 -- 172.21.15.82:0/3637882484 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 msgr2=0x7fd6140cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd618d8c640 1 --2- 172.21.15.82:0/3637882484 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd618d8c640 1 -- 172.21.15.82:0/3637882484 >> v1:172.21.15.164:6789/0 conn(0x7fd6140b4350 legacy=0x7fd61415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.625+0000 7fd618d8c640 1 -- 172.21.15.82:0/3637882484 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61406f4b0 con 0x7fd61406c7e0 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.626+0000 7fd618d8c640 1 --2- 172.21.15.82:0/3637882484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61406c7e0 0x7fd61406f9d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd604009870 tx=0x7fd60402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=742b5fb0c682fb92 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.626+0000 7fd60bfff640 1 -- 172.21.15.82:0/3637882484 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604034070 con 0x7fd61406c7e0 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.626+0000 7fd60bfff640 1 -- 172.21.15.82:0/3637882484 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd604030840 con 0x7fd61406c7e0 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.626+0000 7fd60bfff640 1 -- 172.21.15.82:0/3637882484 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604030c10 con 0x7fd61406c7e0 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 -- 172.21.15.82:0/3637882484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61406c7e0 msgr2=0x7fd61406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/3637882484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61406c7e0 0x7fd61406f9d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd604009870 tx=0x7fd60402b980 comp rx=0 tx=0).stop 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 -- 172.21.15.82:0/3637882484 shutdown_connections 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/3637882484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61406c7e0 0x7fd61406f9d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/3637882484 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 -- 172.21.15.82:0/3637882484 >> 172.21.15.82:0/3637882484 conn(0x7fd6140b3670 msgr2=0x7fd6140b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:15.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 -- 172.21.15.82:0/3637882484 shutdown_connections 2024-03-14T12:18:15.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.627+0000 7fd61b7f1640 1 -- 172.21.15.82:0/3637882484 wait complete. 2024-03-14T12:18:15.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.628+0000 7fd61b7f1640 1 Processor -- start 2024-03-14T12:18:15.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.628+0000 7fd61b7f1640 1 -- start start 2024-03-14T12:18:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61b7f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd61406c7e0 0x7fd6141f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61b7f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6140b4350 0x7fd614153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61b7f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61b7f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140cffa0 con 0x7fd61406c7e0 2024-03-14T12:18:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61b7f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61406ee80 con 0x7fd6140b4350 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61b7f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140d0120 con 0x7fd61415e6b0 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd61406c7e0 0x7fd6141f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd61406c7e0 0x7fd6141f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55922/0 (socket says 172.21.15.82:55922) 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 -- 172.21.15.82:0/365845934 learned_addr learned my addr 172.21.15.82:0/365845934 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd618d8c640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6140b4350 0x7fd614153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd619d8e640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 -- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 msgr2=0x7fd6140b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 -- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6140b4350 msgr2=0x7fd614153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6140b4350 0x7fd614153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 -- 172.21.15.82:0/365845934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6141f4720 con 0x7fd61406c7e0 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd618d8c640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6140b4350 0x7fd614153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61958d640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd61406c7e0 0x7fd6141f41e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd61000e9b0 tx=0x7fd61000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd609ffb640 1 -- 172.21.15.82:0/365845934 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610017070 con 0x7fd61406c7e0 2024-03-14T12:18:15.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd609ffb640 1 -- 172.21.15.82:0/365845934 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61000c9a0 con 0x7fd61406c7e0 2024-03-14T12:18:15.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6140b49d0 con 0x7fd61406c7e0 2024-03-14T12:18:15.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.629+0000 7fd609ffb640 1 -- 172.21.15.82:0/365845934 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000cd90 con 0x7fd61406c7e0 2024-03-14T12:18:15.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.630+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6140b4f90 con 0x7fd61406c7e0 2024-03-14T12:18:15.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.631+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6140d0b70 con 0x7fd61406c7e0 2024-03-14T12:18:15.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.631+0000 7fd609ffb640 1 -- 172.21.15.82:0/365845934 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd61001a070 con 0x7fd61406c7e0 2024-03-14T12:18:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.631+0000 7fd609ffb640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5f806bbd0 0x7fd5f806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.632+0000 7fd609ffb640 1 -- 172.21.15.82:0/365845934 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd61008ab40 con 0x7fd61406c7e0 2024-03-14T12:18:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.632+0000 7fd618d8c640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5f806bbd0 0x7fd5f806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.632+0000 7fd618d8c640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5f806bbd0 0x7fd5f806e090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd604036000 tx=0x7fd60403b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.636+0000 7fd609ffb640 1 -- 172.21.15.82:0/365845934 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd6140d0b70 con 0x7fd61406c7e0 2024-03-14T12:18:15.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.820+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd61404f050 con 0x7fd61406c7e0 2024-03-14T12:18:15.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.820+0000 7fd609ffb640 1 -- 172.21.15.82:0/365845934 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v73) v1 ==== 80+0+239 (secure 0 0 0) 0x7fd6100595d0 con 0x7fd61406c7e0 2024-03-14T12:18:15.821 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.164:6834/4033072140 2024-03-15T12:17:15.041576+0000 2024-03-14T12:18:15.821 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.1:0/895829753 2024-03-14T13:18:05.110010+0000 2024-03-14T12:18:15.821 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.164:0/1140720987 2024-03-14T13:18:05.110143+0000 2024-03-14T12:18:15.821 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.164:6832/4033072140 2024-03-15T12:17:15.041576+0000 2024-03-14T12:18:15.821 INFO:teuthology.orchestra.run.smithi082.stderr:listed 4 entries 2024-03-14T12:18:15.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5f806bbd0 msgr2=0x7fd5f806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5f806bbd0 0x7fd5f806e090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd604036000 tx=0x7fd60403b040 comp rx=0 tx=0).stop 2024-03-14T12:18:15.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd61406c7e0 msgr2=0x7fd6141f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:15.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd61406c7e0 0x7fd6141f41e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd61000e9b0 tx=0x7fd61000ee70 comp rx=0 tx=0).stop 2024-03-14T12:18:15.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 shutdown_connections 2024-03-14T12:18:15.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fd5f806bbd0 0x7fd5f806e090 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd61415e6b0 0x7fd6140b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6140b4350 0x7fd614153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 --2- 172.21.15.82:0/365845934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd61406c7e0 0x7fd6141f41e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:15.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.822+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 >> 172.21.15.82:0/365845934 conn(0x7fd6140b3670 msgr2=0x7fd6140d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:15.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.823+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 shutdown_connections 2024-03-14T12:18:15.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:15.823+0000 7fd61b7f1640 1 -- 172.21.15.82:0/365845934 wait complete. 2024-03-14T12:18:15.834 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T12:18:15.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:15.834 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T12:18:15.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:18:15.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:15.864 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T12:18:15.919 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:18:15.919 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:15.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:18:15.938 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:18:15.938 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:15.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:18:15.974 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:18:15.974 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:15.996 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:18:15.997 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:16.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.113+0000 7fa348020640 1 Processor -- start 2024-03-14T12:18:16.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa348020640 1 -- start start 2024-03-14T12:18:16.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa348020640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3400cf370 0x7fa3400d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa348020640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3400b7720 0x7fa3400b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa348020640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400cf8c0 con 0x7fa3400d0760 2024-03-14T12:18:16.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa348020640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3400cfba0 con 0x7fa3400cf370 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa348020640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3400b6c40 con 0x7fa3400b7720 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa3455bb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3400cf370 0x7fa3400d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa3455bb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3400cf370 0x7fa3400d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55924/0 (socket says 172.21.15.82:55924) 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.114+0000 7fa3455bb640 1 -- 172.21.15.82:0/2045947023 learned_addr learned my addr 172.21.15.82:0/2045947023 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.115+0000 7fa3465bd640 1 --2- 172.21.15.82:0/2045947023 >> v2:172.21.15.164:3300/0 conn(0x7fa3400b7720 0x7fa3400b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.115+0000 7fa3455bb640 1 -- 172.21.15.82:0/2045947023 >> v2:172.21.15.164:3300/0 conn(0x7fa3400b7720 msgr2=0x7fa3400b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.115+0000 7fa3455bb640 1 --2- 172.21.15.82:0/2045947023 >> v2:172.21.15.164:3300/0 conn(0x7fa3400b7720 0x7fa3400b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.115+0000 7fa3455bb640 1 -- 172.21.15.82:0/2045947023 >> v1:172.21.15.164:6789/0 conn(0x7fa3400d0760 legacy=0x7fa3400d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.115+0000 7fa3455bb640 1 -- 172.21.15.82:0/2045947023 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400b6f20 con 0x7fa3400cf370 2024-03-14T12:18:16.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.115+0000 7fa3455bb640 1 --2- 172.21.15.82:0/2045947023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400cf370 0x7fa3400d4660 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa330009870 tx=0x7fa33002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ebe4e16450e850 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa344dba640 1 -- 172.21.15.82:0/2045947023 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33002f840 con 0x7fa3400cf370 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa344dba640 1 -- 172.21.15.82:0/2045947023 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33002f9e0 con 0x7fa3400cf370 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa344dba640 1 -- 172.21.15.82:0/2045947023 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33002fdb0 con 0x7fa3400cf370 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 -- 172.21.15.82:0/2045947023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400cf370 msgr2=0x7fa3400d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 --2- 172.21.15.82:0/2045947023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400cf370 0x7fa3400d4660 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa330009870 tx=0x7fa33002ba20 comp rx=0 tx=0).stop 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 -- 172.21.15.82:0/2045947023 shutdown_connections 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 --2- 172.21.15.82:0/2045947023 >> v2:172.21.15.164:3300/0 conn(0x7fa3400b7720 0x7fa3400b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 --2- 172.21.15.82:0/2045947023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400cf370 0x7fa3400d4660 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 -- 172.21.15.82:0/2045947023 >> 172.21.15.82:0/2045947023 conn(0x7fa34009d030 msgr2=0x7fa3400d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 -- 172.21.15.82:0/2045947023 shutdown_connections 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.116+0000 7fa348020640 1 -- 172.21.15.82:0/2045947023 wait complete. 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.117+0000 7fa348020640 1 Processor -- start 2024-03-14T12:18:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.117+0000 7fa348020640 1 -- start start 2024-03-14T12:18:16.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa348020640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400b7720 0x7fa340152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa348020640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400cf370 0x7fa3400cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa348020640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d0760 0x7fa34015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa348020640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3401617a0 con 0x7fa3400d0760 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa348020640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400d4ba0 con 0x7fa3400b7720 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa348020640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa340161920 con 0x7fa3400cf370 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa3465bd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d0760 0x7fa34015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa3465bd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d0760 0x7fa34015cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55930/0 (socket says 172.21.15.82:55930) 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.118+0000 7fa3465bd640 1 -- 172.21.15.82:0/2665173431 learned_addr learned my addr 172.21.15.82:0/2665173431 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa345dbc640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400b7720 0x7fa340152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3455bb640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400cf370 0x7fa3400cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3465bd640 1 -- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400cf370 msgr2=0x7fa3400cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3465bd640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400cf370 0x7fa3400cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3465bd640 1 -- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400b7720 msgr2=0x7fa340152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3465bd640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400b7720 0x7fa340152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3465bd640 1 -- 172.21.15.82:0/2665173431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400cbf40 con 0x7fa3400d0760 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa345dbc640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400b7720 0x7fa340152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:16.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3455bb640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400cf370 0x7fa3400cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3465bd640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d0760 0x7fa34015cf00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa33c007e40 tx=0x7fa33c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3367fc640 1 -- 172.21.15.82:0/2665173431 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c014620 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3367fc640 1 -- 172.21.15.82:0/2665173431 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c019070 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa3367fc640 1 -- 172.21.15.82:0/2665173431 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c014a20 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa340158ab0 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.119+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa340159070 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.120+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34004f050 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.120+0000 7fa3367fc640 1 -- 172.21.15.82:0/2665173431 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa33c014bc0 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.121+0000 7fa3367fc640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32406bbd0 0x7fa32406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.121+0000 7fa3367fc640 1 -- 172.21.15.82:0/2665173431 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa33c08b470 con 0x7fa3400d0760 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.121+0000 7fa345dbc640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32406bbd0 0x7fa32406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.121+0000 7fa345dbc640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32406bbd0 0x7fa32406e090 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa3380015e0 tx=0x7fa338009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.125+0000 7fa3367fc640 1 -- 172.21.15.82:0/2665173431 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa33c059e70 con 0x7fa3400d0760 2024-03-14T12:18:16.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.338+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3400d0540 con 0x7fa3400d0760 2024-03-14T12:18:16.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.339+0000 7fa3367fc640 1 -- 172.21.15.82:0/2665173431 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fa33c059770 con 0x7fa3400d0760 2024-03-14T12:18:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:16.340 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:18:06.362029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":21,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:16.340 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 40 2024-03-14T12:18:16.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32406bbd0 msgr2=0x7fa32406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32406bbd0 0x7fa32406e090 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa3380015e0 tx=0x7fa338009040 comp rx=0 tx=0).stop 2024-03-14T12:18:16.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d0760 msgr2=0x7fa34015cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d0760 0x7fa34015cf00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa33c007e40 tx=0x7fa33c00e4b0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 shutdown_connections 2024-03-14T12:18:16.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400d0760 0x7fa34015cf00 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32406bbd0 0x7fa32406e090 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa3400cf370 0x7fa3400cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 --2- 172.21.15.82:0/2665173431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3400b7720 0x7fa340152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 >> 172.21.15.82:0/2665173431 conn(0x7fa34009d030 msgr2=0x7fa3400d23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 shutdown_connections 2024-03-14T12:18:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.341+0000 7fa348020640 1 -- 172.21.15.82:0/2665173431 wait complete. 2024-03-14T12:18:16.353 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:18:16.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.453+0000 7f6b6bf22640 1 Processor -- start 2024-03-14T12:18:16.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.453+0000 7f6b6bf22640 1 -- start start 2024-03-14T12:18:16.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.454+0000 7f6b6bf22640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6b640d0760 0x7f6b640d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.454+0000 7f6b6bf22640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 0x7f6b640b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.454+0000 7f6b6bf22640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640d1080 con 0x7f6b640cf370 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.454+0000 7f6b6bf22640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b640cf8c0 con 0x7f6b640d0760 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.454+0000 7f6b6bf22640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cfba0 con 0x7f6b640b7720 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.454+0000 7f6b694bd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6b640cf370 0x7f6b640d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44740/0 (socket says 172.21.15.82:44740) 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.454+0000 7f6b694bd640 1 -- 172.21.15.82:0/3601432844 learned_addr learned my addr 172.21.15.82:0/3601432844 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b69cbe640 1 --2- 172.21.15.82:0/3601432844 >> v2:172.21.15.164:3300/0 conn(0x7f6b640d0760 0x7f6b640d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 0x7f6b640b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186836011 0 0) 0x7f6b640d1080 con 0x7f6b640cf370 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b50003520 con 0x7f6b640cf370 2024-03-14T12:18:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1755973689 0 0) 0x7f6b50003520 con 0x7f6b640cf370 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 msgr2=0x7f6b640b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 0x7f6b640b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 >> v2:172.21.15.164:3300/0 conn(0x7f6b640d0760 msgr2=0x7f6b640d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 --2- 172.21.15.82:0/3601432844 >> v2:172.21.15.164:3300/0 conn(0x7f6b640d0760 0x7f6b640d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b640b6c40 con 0x7f6b640cf370 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6b54001670 con 0x7f6b640cf370 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 >> v1:172.21.15.82:6789/0 conn(0x7f6b640cf370 legacy=0x7f6b640d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b500039b0 0x7f6b50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 0x7f6b50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50004d40 0x7f6b500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cf8c0 con 0x7f6b50004d40 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cfba0 con 0x7f6b500039b0 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5000b9c0 con 0x7f6b640b7720 2024-03-14T12:18:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50004d40 0x7f6b500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b69cbe640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b500039b0 0x7f6b50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b694bd640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 0x7f6b50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 msgr2=0x7f6b50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b640b7720 0x7f6b50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b500039b0 msgr2=0x7f6b50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b500039b0 0x7f6b50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.455+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/3601432844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50003520 con 0x7f6b50004d40 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.456+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50004d40 0x7f6b500051c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009ac0 tx=0x7f6b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b8df740fca9efd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.456+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c039070 con 0x7f6b50004d40 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.456+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c02c6b0 con 0x7f6b50004d40 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.456+0000 7f6b68cbc640 1 -- 172.21.15.82:0/3601432844 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c009690 con 0x7f6b50004d40 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.457+0000 7f6b6bf22640 1 -- 172.21.15.82:0/3601432844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50004d40 msgr2=0x7f6b500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.457+0000 7f6b6bf22640 1 --2- 172.21.15.82:0/3601432844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50004d40 0x7f6b500051c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009ac0 tx=0x7f6b50005830 comp rx=0 tx=0).stop 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.457+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/3601432844 reap_dead start 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.457+0000 7f6b6bf22640 1 -- 172.21.15.82:0/3601432844 shutdown_connections 2024-03-14T12:18:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.457+0000 7f6b6bf22640 1 -- 172.21.15.82:0/3601432844 >> 172.21.15.82:0/3601432844 conn(0x7f6b6409d030 msgr2=0x7f6b640d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:16.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.457+0000 7f6b6bf22640 1 -- 172.21.15.82:0/3601432844 shutdown_connections 2024-03-14T12:18:16.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.457+0000 7f6b6bf22640 1 -- 172.21.15.82:0/3601432844 wait complete. 2024-03-14T12:18:16.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.458+0000 7f6b6bf22640 1 Processor -- start 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.458+0000 7f6b6bf22640 1 -- start start 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b50004d40 0x7f6b64158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b6414b1f0 0x7f6b64159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6415a1a0 0x7f6b6406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b641617a0 con 0x7f6b6415a1a0 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640b7200 con 0x7f6b6414b1f0 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64161920 con 0x7f6b50004d40 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6415a1a0 0x7f6b6406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6415a1a0 0x7f6b6406bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55950/0 (socket says 172.21.15.82:55950) 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/306523863 learned_addr learned my addr 172.21.15.82:0/306523863 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/306523863 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b50004d40 msgr2=0x7f6b64158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b50004d40 0x7f6b64158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/306523863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b6414b1f0 msgr2=0x7f6b64159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b6414b1f0 0x7f6b64159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 -- 172.21.15.82:0/306523863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6414b670 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6a4bf640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6415a1a0 0x7f6b6406bc10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009ac0 tx=0x7f6b5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b5a7fc640 1 -- 172.21.15.82:0/306523863 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c039070 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b5a7fc640 1 -- 172.21.15.82:0/306523863 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c043070 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b64158bc0 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b5a7fc640 1 -- 172.21.15.82:0/306523863 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c009690 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.459+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b64159180 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.460+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6404f050 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.461+0000 7f6b5a7fc640 1 -- 172.21.15.82:0/306523863 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6b5c003850 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.461+0000 7f6b5a7fc640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b4806bc00 0x7f6b4806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.461+0000 7f6b5a7fc640 1 -- 172.21.15.82:0/306523863 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f6b5c0ac040 con 0x7f6b6415a1a0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.461+0000 7f6b69cbe640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b4806bc00 0x7f6b4806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.462+0000 7f6b69cbe640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b4806bc00 0x7f6b4806e0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009f40 tx=0x7f6b60009750 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.465+0000 7f6b5a7fc640 1 -- 172.21.15.82:0/306523863 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6b5c04d020 con 0x7f6b6415a1a0 2024-03-14T12:18:16.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.636+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b640d0540 con 0x7f6b6415a1a0 2024-03-14T12:18:16.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.637+0000 7f6b5a7fc640 1 -- 172.21.15.82:0/306523863 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+17008 (secure 0 0 0) 0x7f6b5c07a9b0 con 0x7f6b6415a1a0 2024-03-14T12:18:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:16.639 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":73,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:18:05.349077+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6834/4033072140":"2024-03-15T12:17:15.041576+0000","192.168.0.1:0/895829753":"2024-03-14T13:18:05.110010+0000","172.21.15.164:0/1140720987":"2024-03-14T13:18:05.110143+0000","172.21.15.164:6832/4033072140":"2024-03-15T12:17:15.041576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:18:16.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b4806bc00 msgr2=0x7f6b4806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b4806bc00 0x7f6b4806e0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009f40 tx=0x7f6b60009750 comp rx=0 tx=0).stop 2024-03-14T12:18:16.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6415a1a0 msgr2=0x7f6b6406bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6415a1a0 0x7f6b6406bc10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009ac0 tx=0x7f6b5c004730 comp rx=0 tx=0).stop 2024-03-14T12:18:16.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 shutdown_connections 2024-03-14T12:18:16.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6415a1a0 0x7f6b6406bc10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b6414b1f0 0x7f6b64159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6b4806bc00 0x7f6b4806e0c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 --2- 172.21.15.82:0/306523863 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b50004d40 0x7f6b64158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.639+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 >> 172.21.15.82:0/306523863 conn(0x7f6b6409d030 msgr2=0x7f6b6406c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:16.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.640+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 shutdown_connections 2024-03-14T12:18:16.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.640+0000 7f6b6bf22640 1 -- 172.21.15.82:0/306523863 wait complete. 2024-03-14T12:18:16.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T12:13:39.541273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T12:16:57.716705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T12:16:58.458303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T12:18:16.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T12:18:16.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.754+0000 7f3c20ebf640 1 Processor -- start 2024-03-14T12:18:16.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.754+0000 7f3c20ebf640 1 -- start start 2024-03-14T12:18:16.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c20ebf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c1608c0 0x7f3c1c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c20ebf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c1c15fd00 0x7f3c1c1601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c20ebf640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1c0cf8a0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c20ebf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c0d0ba0 con 0x7f3c1c15fd00 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c20ebf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c0d01d0 con 0x7f3c1c1608c0 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c1ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c1c15fd00 0x7f3c1c1601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c1ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c1c15fd00 0x7f3c1c1601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55966/0 (socket says 172.21.15.82:55966) 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.755+0000 7f3c1ad9d640 1 -- 172.21.15.82:0/3996654467 learned_addr learned my addr 172.21.15.82:0/3996654467 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c13fff640 1 --2- 172.21.15.82:0/3996654467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c1608c0 0x7f3c1c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c1ad9d640 1 -- 172.21.15.82:0/3996654467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c1608c0 msgr2=0x7f3c1c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c1ad9d640 1 --2- 172.21.15.82:0/3996654467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c1608c0 0x7f3c1c0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c1ad9d640 1 -- 172.21.15.82:0/3996654467 >> v1:172.21.15.82:6789/0 conn(0x7f3c1c15c0e0 legacy=0x7f3c1c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c1ad9d640 1 -- 172.21.15.82:0/3996654467 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0cfb80 con 0x7f3c1c15fd00 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c19d9b640 1 -- 172.21.15.82:0/3996654467 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205057531 0 0) 0x7f3c1c0cf8a0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c13fff640 1 --2- 172.21.15.82:0/3996654467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c1608c0 0x7f3c1c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c1ad9d640 1 --2- 172.21.15.82:0/3996654467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15fd00 0x7f3c1c1601b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009e50 tx=0x7f3c1402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e719faac18cc38e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c19d9b640 1 -- 172.21.15.82:0/3996654467 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1402cb00 con 0x7f3c1c15fd00 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c19d9b640 1 -- 172.21.15.82:0/3996654467 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1402cca0 con 0x7f3c1c15fd00 2024-03-14T12:18:16.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.756+0000 7f3c19d9b640 1 -- 172.21.15.82:0/3996654467 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14034610 con 0x7f3c1c15fd00 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 -- 172.21.15.82:0/3996654467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15fd00 msgr2=0x7f3c1c1601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/3996654467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15fd00 0x7f3c1c1601b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009e50 tx=0x7f3c1402bf60 comp rx=0 tx=0).stop 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 -- 172.21.15.82:0/3996654467 shutdown_connections 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/3996654467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15fd00 0x7f3c1c1601b0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/3996654467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c1608c0 0x7f3c1c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 -- 172.21.15.82:0/3996654467 >> 172.21.15.82:0/3996654467 conn(0x7f3c1c0b3680 msgr2=0x7f3c1c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 -- 172.21.15.82:0/3996654467 shutdown_connections 2024-03-14T12:18:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.757+0000 7f3c20ebf640 1 -- 172.21.15.82:0/3996654467 wait complete. 2024-03-14T12:18:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.758+0000 7f3c20ebf640 1 Processor -- start 2024-03-14T12:18:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.758+0000 7f3c20ebf640 1 -- start start 2024-03-14T12:18:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c20ebf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15c0e0 0x7f3c1c0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c20ebf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c15fd00 0x7f3c1c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c20ebf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c1c1608c0 0x7f3c1c0c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c20ebf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c15c4c0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c20ebf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c0cd960 con 0x7f3c1c1608c0 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c20ebf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c15c640 con 0x7f3c1c15fd00 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15c0e0 0x7f3c1c0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15c0e0 0x7f3c1c0c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55974/0 (socket says 172.21.15.82:55974) 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 -- 172.21.15.82:0/2517902958 learned_addr learned my addr 172.21.15.82:0/2517902958 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1ad9d640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c1c1608c0 0x7f3c1c0c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c13fff640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c15fd00 0x7f3c1c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 -- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c15fd00 msgr2=0x7f3c1c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c15fd00 0x7f3c1c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 -- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c1c1608c0 msgr2=0x7f3c1c0c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c1c1608c0 0x7f3c1c0c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1a59c640 1 -- 172.21.15.82:0/2517902958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0caf80 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.759+0000 7f3c1ad9d640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c1c1608c0 0x7f3c1c0c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.760+0000 7f3c1a59c640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15c0e0 0x7f3c1c0c9ba0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00c410 tx=0x7f3c0c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.760+0000 7f3c12ffd640 1 -- 172.21.15.82:0/2517902958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c017070 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.760+0000 7f3c12ffd640 1 -- 172.21.15.82:0/2517902958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0035f0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.760+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1c0cb260 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.760+0000 7f3c12ffd640 1 -- 172.21.15.82:0/2517902958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0039e0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.760+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c0cb820 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.761+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1c04f050 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.761+0000 7f3c12ffd640 1 -- 172.21.15.82:0/2517902958 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3c0c01a070 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.762+0000 7f3c12ffd640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:16.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.762+0000 7f3c12ffd640 1 -- 172.21.15.82:0/2517902958 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3c0c089fc0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.762+0000 7f3c13fff640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:16.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.762+0000 7f3c13fff640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c1f6ff0 tx=0x7f3c04008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:16.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.766+0000 7f3c12ffd640 1 -- 172.21.15.82:0/2517902958 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3c0c0589c0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.939+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c1c0b7ab0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.940+0000 7f3c12ffd640 1 -- 172.21.15.82:0/2517902958 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c0c0582c0 con 0x7f3c1c15c0e0 2024-03-14T12:18:16.940 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-14T12:18:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.941+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 msgr2=0x7f3bfc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.941+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c1f6ff0 tx=0x7f3c04008040 comp rx=0 tx=0).stop 2024-03-14T12:18:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.941+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15c0e0 msgr2=0x7f3c1c0c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.941+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15c0e0 0x7f3c1c0c9ba0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00c410 tx=0x7f3c0c00c8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 shutdown_connections 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c1c1608c0 0x7f3c1c0c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f3bfc06bbd0 0x7f3bfc06e090 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3c1c15fd00 0x7f3c1c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 --2- 172.21.15.82:0/2517902958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c1c15c0e0 0x7f3c1c0c9ba0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 >> 172.21.15.82:0/2517902958 conn(0x7f3c1c0b3680 msgr2=0x7f3c1c0b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 shutdown_connections 2024-03-14T12:18:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:16.942+0000 7f3c20ebf640 1 -- 172.21.15.82:0/2517902958 wait complete. 2024-03-14T12:18:16.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T12:18:17.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.056+0000 7fa23e345640 1 Processor -- start 2024-03-14T12:18:17.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.056+0000 7fa23e345640 1 -- start start 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.057+0000 7fa23e345640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 0x7fa23815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.057+0000 7fa23e345640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2380b5a30 con 0x7fa2380cd960 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.057+0000 7fa23e345640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2380ce380 con 0x7fa23815bde0 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.057+0000 7fa23e345640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380cef90 con 0x7fa2380d08c0 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.057+0000 7fa2377fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa2380cd960 0x7fa2380d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44754/0 (socket says 172.21.15.82:44754) 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.057+0000 7fa2377fe640 1 -- 172.21.15.82:0/3989807282 learned_addr learned my addr 172.21.15.82:0/3989807282 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377109111 0 0) 0x7fa22c001110 con 0x7fa2380cd960 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa220003520 con 0x7fa2380cd960 2024-03-14T12:18:17.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa23c8e2640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 0x7fa23815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2654755319 0 0) 0x7fa220003520 con 0x7fa2380cd960 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 msgr2=0x7fa23815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 0x7fa23815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 >> v1:172.21.15.164:6789/0 conn(0x7fa23815bde0 legacy=0x7fa2380cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380ce660 con 0x7fa2380cd960 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fa22c001840 con 0x7fa2380cd960 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 >> v1:172.21.15.82:6789/0 conn(0x7fa2380cd960 legacy=0x7fa2380d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa220003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 0x7fa220004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220004d40 0x7fa2200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380cef90 con 0x7fa2200039e0 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22000bb30 con 0x7fa220004d40 2024-03-14T12:18:17.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22000bd30 con 0x7fa2380d08c0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa237fff640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa220003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa2377fe640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 0x7fa220004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa23c8e2640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220004d40 0x7fa2200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa237fff640 1 -- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 msgr2=0x7fa220004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa237fff640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2380d08c0 0x7fa220004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa237fff640 1 -- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220004d40 msgr2=0x7fa2200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa237fff640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220004d40 0x7fa2200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa237fff640 1 -- 172.21.15.82:0/3989807282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220003520 con 0x7fa2200039e0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.058+0000 7fa23c8e2640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa220004d40 0x7fa2200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.059+0000 7fa237fff640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa220003e00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa228001720 tx=0x7fa220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aebb7e7a774747 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.059+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228001370 con 0x7fa2200039e0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.059+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2280052d0 con 0x7fa2200039e0 2024-03-14T12:18:17.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.059+0000 7fa236ffd640 1 -- 172.21.15.82:0/3989807282 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228003830 con 0x7fa2200039e0 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.060+0000 7fa23e345640 1 -- 172.21.15.82:0/3989807282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 msgr2=0x7fa220003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.060+0000 7fa23e345640 1 --2- 172.21.15.82:0/3989807282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa220003e00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa228001720 tx=0x7fa220005830 comp rx=0 tx=0).stop 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.060+0000 7fa23c8e2640 1 -- 172.21.15.82:0/3989807282 reap_dead start 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.060+0000 7fa23e345640 1 -- 172.21.15.82:0/3989807282 shutdown_connections 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.060+0000 7fa23e345640 1 -- 172.21.15.82:0/3989807282 >> 172.21.15.82:0/3989807282 conn(0x7fa2380d1bd0 msgr2=0x7fa2380d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.060+0000 7fa23e345640 1 -- 172.21.15.82:0/3989807282 shutdown_connections 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.060+0000 7fa23e345640 1 -- 172.21.15.82:0/3989807282 wait complete. 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.061+0000 7fa23e345640 1 Processor -- start 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.061+0000 7fa23e345640 1 -- start start 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23e345640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa23815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23e345640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2380c9f40 0x7fa2380d2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23e345640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2381f0e00 0x7fa2381f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23e345640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380cf420 con 0x7fa2200039e0 2024-03-14T12:18:17.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23e345640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380cec20 con 0x7fa2380c9f40 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23e345640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23815c1c0 con 0x7fa2381f0e00 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa23815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa23815c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55996/0 (socket says 172.21.15.82:55996) 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 -- 172.21.15.82:0/3269463412 learned_addr learned my addr 172.21.15.82:0/3269463412 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa2377fe640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2380c9f40 0x7fa2380d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 -- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2381f0e00 msgr2=0x7fa2381f1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23c8e2640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2381f0e00 0x7fa2381f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2381f0e00 0x7fa2381f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 -- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2380c9f40 msgr2=0x7fa2380d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2380c9f40 0x7fa2380d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 -- 172.21.15.82:0/3269463412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2381f5f90 con 0x7fa2200039e0 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa2377fe640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2380c9f40 0x7fa2380d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:17.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa237fff640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa23815c600 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa228008fd0 tx=0x7fa228001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa234ff9640 1 -- 172.21.15.82:0/3269463412 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228041070 con 0x7fa2200039e0 2024-03-14T12:18:17.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa234ff9640 1 -- 172.21.15.82:0/3269463412 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228005480 con 0x7fa2200039e0 2024-03-14T12:18:17.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa234ff9640 1 -- 172.21.15.82:0/3269463412 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228007a30 con 0x7fa2200039e0 2024-03-14T12:18:17.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.062+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2381f5410 con 0x7fa2200039e0 2024-03-14T12:18:17.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.063+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2381f59d0 con 0x7fa2200039e0 2024-03-14T12:18:17.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.063+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23804f050 con 0x7fa2200039e0 2024-03-14T12:18:17.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.064+0000 7fa234ff9640 1 -- 172.21.15.82:0/3269463412 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa228044070 con 0x7fa2200039e0 2024-03-14T12:18:17.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.064+0000 7fa234ff9640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa22406bbd0 0x7fa22406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.064+0000 7fa234ff9640 1 -- 172.21.15.82:0/3269463412 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa2280ac6f0 con 0x7fa2200039e0 2024-03-14T12:18:17.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.064+0000 7fa2377fe640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa22406bbd0 0x7fa22406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.065+0000 7fa2377fe640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa22406bbd0 0x7fa22406e090 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa22c007fd0 tx=0x7fa22c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.069+0000 7fa234ff9640 1 -- 172.21.15.82:0/3269463412 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa22807b0f0 con 0x7fa2200039e0 2024-03-14T12:18:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.241+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2381f5d70 con 0x7fa2200039e0 2024-03-14T12:18:17.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.241+0000 7fa234ff9640 1 -- 172.21.15.82:0/3269463412 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa22807a9f0 con 0x7fa2200039e0 2024-03-14T12:18:17.242 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa22406bbd0 msgr2=0x7fa22406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa22406bbd0 0x7fa22406e090 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa22c007fd0 tx=0x7fa22c009040 comp rx=0 tx=0).stop 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 msgr2=0x7fa23815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa23815c600 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa228008fd0 tx=0x7fa228001120 comp rx=0 tx=0).stop 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 shutdown_connections 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa22406bbd0 0x7fa22406e090 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2381f0e00 0x7fa2381f1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2380c9f40 0x7fa2380d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 --2- 172.21.15.82:0/3269463412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2200039e0 0x7fa23815c600 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 >> 172.21.15.82:0/3269463412 conn(0x7fa2380d1bd0 msgr2=0x7fa2380b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 shutdown_connections 2024-03-14T12:18:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.243+0000 7fa23e345640 1 -- 172.21.15.82:0/3269463412 wait complete. 2024-03-14T12:18:17.255 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T12:18:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.357+0000 7f37439f3640 1 Processor -- start 2024-03-14T12:18:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.357+0000 7f37439f3640 1 -- start start 2024-03-14T12:18:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f37439f3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f373c1446f0 0x7f373c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f37439f3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f373c0d0210 0x7f373c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f37439f3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c0b34e0 con 0x7f373c0cfe30 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f37439f3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c15f1f0 con 0x7f373c1446f0 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f37439f3640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c15f4a0 con 0x7f373c0d0210 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f3740f8e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f373c1446f0 0x7f373c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f374178f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f373c0cfe30 0x7f373c143fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44764/0 (socket says 172.21.15.82:44764) 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f374178f640 1 -- 172.21.15.82:0/3788942870 learned_addr learned my addr 172.21.15.82:0/3788942870 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f3740f8e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f373c1446f0 0x7f373c15e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56000/0 (socket says 172.21.15.82:56000) 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f3740f8e640 1 -- 172.21.15.82:0/3788942870 >> v2:172.21.15.164:3300/0 conn(0x7f373c0d0210 msgr2=0x7f373c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f3740f8e640 1 --2- 172.21.15.82:0/3788942870 >> v2:172.21.15.164:3300/0 conn(0x7f373c0d0210 0x7f373c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f3740f8e640 1 -- 172.21.15.82:0/3788942870 >> v1:172.21.15.82:6789/0 conn(0x7f373c0cfe30 legacy=0x7f373c143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f3733fff640 1 -- 172.21.15.82:0/3788942870 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178673762 0 0) 0x7f373c0b34e0 con 0x7f373c0cfe30 2024-03-14T12:18:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.358+0000 7f3740f8e640 1 -- 172.21.15.82:0/3788942870 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c0b3780 con 0x7f373c1446f0 2024-03-14T12:18:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.359+0000 7f3740f8e640 1 --2- 172.21.15.82:0/3788942870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c15e770 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f373c15b110 tx=0x7f372c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a13925a0d8cb86a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.359+0000 7f3733fff640 1 -- 172.21.15.82:0/3788942870 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c035020 con 0x7f373c1446f0 2024-03-14T12:18:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.359+0000 7f3733fff640 1 -- 172.21.15.82:0/3788942870 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c031830 con 0x7f373c1446f0 2024-03-14T12:18:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.359+0000 7f3733fff640 1 -- 172.21.15.82:0/3788942870 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c031c00 con 0x7f373c1446f0 2024-03-14T12:18:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 -- 172.21.15.82:0/3788942870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 msgr2=0x7f373c15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 --2- 172.21.15.82:0/3788942870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c15e770 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f373c15b110 tx=0x7f372c02b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 -- 172.21.15.82:0/3788942870 shutdown_connections 2024-03-14T12:18:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 --2- 172.21.15.82:0/3788942870 >> v2:172.21.15.164:3300/0 conn(0x7f373c0d0210 0x7f373c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 --2- 172.21.15.82:0/3788942870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c15e770 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 -- 172.21.15.82:0/3788942870 >> 172.21.15.82:0/3788942870 conn(0x7f373c06aef0 msgr2=0x7f373c06b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 -- 172.21.15.82:0/3788942870 shutdown_connections 2024-03-14T12:18:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.360+0000 7f37439f3640 1 -- 172.21.15.82:0/3788942870 wait complete. 2024-03-14T12:18:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 Processor -- start 2024-03-14T12:18:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 -- start start 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0cfe30 0x7f373c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0d0210 0x7f373c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b4260 con 0x7f373c1446f0 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b3d40 con 0x7f373c0d0210 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.361+0000 7f37439f3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b43e0 con 0x7f373c0cfe30 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c1f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56008/0 (socket says 172.21.15.82:56008) 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 -- 172.21.15.82:0/2142458612 learned_addr learned my addr 172.21.15.82:0/2142458612 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3740f8e640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0d0210 0x7f373c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f374178f640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0cfe30 0x7f373c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 -- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0cfe30 msgr2=0x7f373c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0cfe30 0x7f373c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 -- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0d0210 msgr2=0x7f373c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0d0210 0x7f373c070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 -- 172.21.15.82:0/2142458612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c1f24c0 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3741f90640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c1f1800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f373400b9e0 tx=0x7f373400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3731ffb640 1 -- 172.21.15.82:0/2142458612 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734019070 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3731ffb640 1 -- 172.21.15.82:0/2142458612 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373400cb20 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c1562d0 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.362+0000 7f3731ffb640 1 -- 172.21.15.82:0/2142458612 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734014440 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.363+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c0aef10 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.363+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c04f050 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.364+0000 7f3731ffb640 1 -- 172.21.15.82:0/2142458612 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f373401c070 con 0x7f373c1446f0 2024-03-14T12:18:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.364+0000 7f3731ffb640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f372006bbd0 0x7f372006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.364+0000 7f3731ffb640 1 -- 172.21.15.82:0/2142458612 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f373408a860 con 0x7f373c1446f0 2024-03-14T12:18:17.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.364+0000 7f374178f640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f372006bbd0 0x7f372006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.365+0000 7f374178f640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f372006bbd0 0x7f372006e090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3738000f60 tx=0x7f3738009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.369+0000 7f3731ffb640 1 -- 172.21.15.82:0/2142458612 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3734059260 con 0x7f373c1446f0 2024-03-14T12:18:17.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.541+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f373c15a910 con 0x7f373c1446f0 2024-03-14T12:18:17.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.541+0000 7f3731ffb640 1 -- 172.21.15.82:0/2142458612 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3734058b60 con 0x7f373c1446f0 2024-03-14T12:18:17.542 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f372006bbd0 msgr2=0x7f372006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f372006bbd0 0x7f372006e090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3738000f60 tx=0x7f3738009040 comp rx=0 tx=0).stop 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 msgr2=0x7f373c1f1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c1f1800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f373400b9e0 tx=0x7f373400bea0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 shutdown_connections 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373c1446f0 0x7f373c1f1800 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0d0210 0x7f373c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f372006bbd0 0x7f372006e090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 --2- 172.21.15.82:0/2142458612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0cfe30 0x7f373c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 >> 172.21.15.82:0/2142458612 conn(0x7f373c06aef0 msgr2=0x7f373c0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 shutdown_connections 2024-03-14T12:18:17.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.543+0000 7f37439f3640 1 -- 172.21.15.82:0/2142458612 wait complete. 2024-03-14T12:18:17.554 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:17.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.654+0000 7f7fd51cd640 1 Processor -- start 2024-03-14T12:18:17.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.654+0000 7f7fd51cd640 1 -- start start 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fd51cd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7fd015df30 0x7f7fd006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fd51cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd006f7f0 0x7f7fd006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fd51cd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd00d1590 con 0x7f7fd00b39c0 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fd51cd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd0161b70 con 0x7f7fd015df30 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fd51cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd0161e50 con 0x7f7fd006f7f0 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fced9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7fd015df30 0x7f7fd006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fcf59e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7fd00b39c0 0x7f7fd0071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44776/0 (socket says 172.21.15.82:44776) 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fced9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7fd015df30 0x7f7fd006ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56014/0 (socket says 172.21.15.82:56014) 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fcf59e640 1 -- 172.21.15.82:0/1843008556 learned_addr learned my addr 172.21.15.82:0/1843008556 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fc7fff640 1 --2- 172.21.15.82:0/1843008556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd006f7f0 0x7f7fd006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fced9d640 1 -- 172.21.15.82:0/1843008556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd006f7f0 msgr2=0x7f7fd006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:17.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fced9d640 1 --2- 172.21.15.82:0/1843008556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd006f7f0 0x7f7fd006b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fced9d640 1 -- 172.21.15.82:0/1843008556 >> v1:172.21.15.82:6789/0 conn(0x7f7fd00b39c0 legacy=0x7f7fd0071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fced9d640 1 -- 172.21.15.82:0/1843008556 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0162130 con 0x7f7fd015df30 2024-03-14T12:18:17.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.655+0000 7f7fce59c640 1 -- 172.21.15.82:0/1843008556 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290140223 0 0) 0x7f7fd00d1590 con 0x7f7fd00b39c0 2024-03-14T12:18:17.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.656+0000 7f7fced9d640 1 --2- 172.21.15.82:0/1843008556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd015df30 0x7f7fd006ad10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00097a0 tx=0x7f7fc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfcf99d9edc6216 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.656+0000 7f7fce59c640 1 -- 172.21.15.82:0/1843008556 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0035020 con 0x7f7fd015df30 2024-03-14T12:18:17.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.656+0000 7f7fce59c640 1 -- 172.21.15.82:0/1843008556 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc0031830 con 0x7f7fd015df30 2024-03-14T12:18:17.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.656+0000 7f7fce59c640 1 -- 172.21.15.82:0/1843008556 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0031c00 con 0x7f7fd015df30 2024-03-14T12:18:17.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1843008556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd015df30 msgr2=0x7f7fd006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1843008556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd015df30 0x7f7fd006ad10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00097a0 tx=0x7f7fc002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1843008556 shutdown_connections 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1843008556 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd006f7f0 0x7f7fd006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1843008556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd015df30 0x7f7fd006ad10 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1843008556 >> 172.21.15.82:0/1843008556 conn(0x7f7fd00d09d0 msgr2=0x7f7fd00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1843008556 shutdown_connections 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.657+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1843008556 wait complete. 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.658+0000 7f7fd51cd640 1 Processor -- start 2024-03-14T12:18:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.658+0000 7f7fd51cd640 1 -- start start 2024-03-14T12:18:17.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd006f7f0 0x7f7fd0149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd00b39c0 0x7f7fd00c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fd015df30 0x7f7fd00c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd006fe40 con 0x7f7fd006f7f0 2024-03-14T12:18:17.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd01628e0 con 0x7f7fd015df30 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd006ffc0 con 0x7f7fd00b39c0 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd006f7f0 0x7f7fd0149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd006f7f0 0x7f7fd0149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56016/0 (socket says 172.21.15.82:56016) 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 -- 172.21.15.82:0/1068946445 learned_addr learned my addr 172.21.15.82:0/1068946445 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 -- 172.21.15.82:0/1068946445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd00b39c0 msgr2=0x7f7fd00c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd00b39c0 0x7f7fd00c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 -- 172.21.15.82:0/1068946445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fd015df30 msgr2=0x7f7fd00c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fd015df30 0x7f7fd00c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 -- 172.21.15.82:0/1068946445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd014a550 con 0x7f7fd006f7f0 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fced9d640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd006f7f0 0x7f7fd0149e10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00097a0 tx=0x7f7fc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fc77fe640 1 -- 172.21.15.82:0/1068946445 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0035070 con 0x7f7fd006f7f0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fc77fe640 1 -- 172.21.15.82:0/1068946445 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc0004230 con 0x7f7fd006f7f0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd015c540 con 0x7f7fd006f7f0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fc77fe640 1 -- 172.21.15.82:0/1068946445 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc003a6f0 con 0x7f7fd006f7f0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.659+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd015c820 con 0x7f7fd006f7f0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.660+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd004f050 con 0x7f7fd006f7f0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.661+0000 7f7fc77fe640 1 -- 172.21.15.82:0/1068946445 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7fc0038020 con 0x7f7fd006f7f0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.661+0000 7f7fc77fe640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7fb006bbd0 0x7f7fb006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.661+0000 7f7fc7fff640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7fb006bbd0 0x7f7fb006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.661+0000 7f7fc77fe640 1 -- 172.21.15.82:0/1068946445 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7fc00ac800 con 0x7f7fd006f7f0 2024-03-14T12:18:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.662+0000 7f7fc7fff640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7fb006bbd0 0x7f7fb006e090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8006fd0 tx=0x7f7fb8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.665+0000 7f7fc77fe640 1 -- 172.21.15.82:0/1068946445 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f7fc0040070 con 0x7f7fd006f7f0 2024-03-14T12:18:17.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.871+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fd00747b0 con 0x7f7fd006f7f0 2024-03-14T12:18:17.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.872+0000 7f7fc77fe640 1 -- 172.21.15.82:0/1068946445 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f7fc0031800 con 0x7f7fd006f7f0 2024-03-14T12:18:17.872 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:17.873 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:18:06.362029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":21,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:17.873 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 40 2024-03-14T12:18:17.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7fb006bbd0 msgr2=0x7f7fb006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7fb006bbd0 0x7f7fb006e090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8006fd0 tx=0x7f7fb8008040 comp rx=0 tx=0).stop 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd006f7f0 msgr2=0x7f7fd0149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd006f7f0 0x7f7fd0149e10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00097a0 tx=0x7f7fc00046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 shutdown_connections 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fd015df30 0x7f7fd00c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f7fb006bbd0 0x7f7fb006e090 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7fd00b39c0 0x7f7fd00c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 --2- 172.21.15.82:0/1068946445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fd006f7f0 0x7f7fd0149e10 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 >> 172.21.15.82:0/1068946445 conn(0x7f7fd00d09d0 msgr2=0x7f7fd00b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 shutdown_connections 2024-03-14T12:18:17.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.874+0000 7f7fd51cd640 1 -- 172.21.15.82:0/1068946445 wait complete. 2024-03-14T12:18:17.886 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:18:17.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.985+0000 7f77dc3a9640 1 Processor -- start 2024-03-14T12:18:17.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.985+0000 7f77dc3a9640 1 -- start start 2024-03-14T12:18:17.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77dc3a9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f77d40d0760 0x7f77d40d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77dc3a9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 0x7f77d40d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77dc3a9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d40d1080 con 0x7f77d40d1da0 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77dc3a9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d40cf8c0 con 0x7f77d40d0760 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77dc3a9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40cfba0 con 0x7f77d40cf370 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77da145640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f77d40d0760 0x7f77d40d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77da946640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f77d40d1da0 0x7f77d40d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44790/0 (socket says 172.21.15.82:44790) 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.986+0000 7f77da946640 1 -- 172.21.15.82:0/619185300 learned_addr learned my addr 172.21.15.82:0/619185300 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.987+0000 7f77da145640 1 -- 172.21.15.82:0/619185300 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 msgr2=0x7f77d40d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.987+0000 7f77da145640 1 --2- 172.21.15.82:0/619185300 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 0x7f77d40d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.987+0000 7f77da145640 1 -- 172.21.15.82:0/619185300 >> v1:172.21.15.82:6789/0 conn(0x7f77d40d1da0 legacy=0x7f77d40d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.987+0000 7f77da145640 1 -- 172.21.15.82:0/619185300 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40b6c40 con 0x7f77d40d0760 2024-03-14T12:18:17.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.987+0000 7f77da145640 1 --2- 172.21.15.82:0/619185300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d0760 0x7f77d40d0b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097a0 tx=0x7f77c402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4865419abe0c65b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77d9143640 1 -- 172.21.15.82:0/619185300 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4034070 con 0x7f77d40d0760 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77d9143640 1 -- 172.21.15.82:0/619185300 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4030610 con 0x7f77d40d0760 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77d9143640 1 -- 172.21.15.82:0/619185300 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40309e0 con 0x7f77d40d0760 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 -- 172.21.15.82:0/619185300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d0760 msgr2=0x7f77d40d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/619185300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d0760 0x7f77d40d0b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097a0 tx=0x7f77c402b960 comp rx=0 tx=0).stop 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 -- 172.21.15.82:0/619185300 shutdown_connections 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/619185300 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 0x7f77d40d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/619185300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d0760 0x7f77d40d0b40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 -- 172.21.15.82:0/619185300 >> 172.21.15.82:0/619185300 conn(0x7f77d409d030 msgr2=0x7f77d40d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 -- 172.21.15.82:0/619185300 shutdown_connections 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.988+0000 7f77dc3a9640 1 -- 172.21.15.82:0/619185300 wait complete. 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.989+0000 7f77dc3a9640 1 Processor -- start 2024-03-14T12:18:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.989+0000 7f77dc3a9640 1 -- start start 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77dc3a9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 0x7f77d40c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77dc3a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77d40d0760 0x7f77d4158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77dc3a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d1da0 0x7f77d4153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77dc3a9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40d4ba0 con 0x7f77d40d1da0 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77dc3a9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40b7200 con 0x7f77d40d0760 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77dc3a9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d41617a0 con 0x7f77d40cf370 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77da946640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d1da0 0x7f77d4153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77da946640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d1da0 0x7f77d4153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56042/0 (socket says 172.21.15.82:56042) 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77da946640 1 -- 172.21.15.82:0/3374333283 learned_addr learned my addr 172.21.15.82:0/3374333283 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77da145640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 0x7f77d40c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.990+0000 7f77d9944640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77d40d0760 0x7f77d4158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77da946640 1 -- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 msgr2=0x7f77d40c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77da946640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 0x7f77d40c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77da946640 1 -- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77d40d0760 msgr2=0x7f77d4158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77da946640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77d40d0760 0x7f77d4158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77da946640 1 -- 172.21.15.82:0/3374333283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d4158770 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77d9944640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77d40d0760 0x7f77d4158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77da946640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d1da0 0x7f77d4153b70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cbc0 tx=0x7f77cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77caffd640 1 -- 172.21.15.82:0/3374333283 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc017070 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77caffd640 1 -- 172.21.15.82:0/3374333283 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc00ad40 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d40c7a60 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77caffd640 1 -- 172.21.15.82:0/3374333283 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc012600 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.991+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d40c8020 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.992+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d404f050 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.992+0000 7f77caffd640 1 -- 172.21.15.82:0/3374333283 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f77cc01a070 con 0x7f77d40d1da0 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.993+0000 7f77caffd640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f77b806bbd0 0x7f77b806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.993+0000 7f77caffd640 1 -- 172.21.15.82:0/3374333283 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f77cc08a800 con 0x7f77d40d1da0 2024-03-14T12:18:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.993+0000 7f77da145640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f77b806bbd0 0x7f77b806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.993+0000 7f77da145640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f77b806bbd0 0x7f77b806e090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77c4036000 tx=0x7f77c403b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:17.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:17.997+0000 7f77caffd640 1 -- 172.21.15.82:0/3374333283 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f77cc058bb0 con 0x7f77d40d1da0 2024-03-14T12:18:18.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.172+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77d40d0540 con 0x7f77d40d1da0 2024-03-14T12:18:18.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.173+0000 7f77caffd640 1 -- 172.21.15.82:0/3374333283 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+17008 (secure 0 0 0) 0x7f77cc05cc50 con 0x7f77d40d1da0 2024-03-14T12:18:18.174 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:18.174 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":73,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:18:05.349077+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6834/4033072140":"2024-03-15T12:17:15.041576+0000","192.168.0.1:0/895829753":"2024-03-14T13:18:05.110010+0000","172.21.15.164:0/1140720987":"2024-03-14T13:18:05.110143+0000","172.21.15.164:6832/4033072140":"2024-03-15T12:17:15.041576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f77b806bbd0 msgr2=0x7f77b806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f77b806bbd0 0x7f77b806e090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77c4036000 tx=0x7f77c403b040 comp rx=0 tx=0).stop 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d1da0 msgr2=0x7f77d4153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d1da0 0x7f77d4153b70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cbc0 tx=0x7f77cc00a430 comp rx=0 tx=0).stop 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 shutdown_connections 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d40d1da0 0x7f77d4153b70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77d40d0760 0x7f77d4158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f77b806bbd0 0x7f77b806e090 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 --2- 172.21.15.82:0/3374333283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77d40cf370 0x7f77d40c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 >> 172.21.15.82:0/3374333283 conn(0x7f77d409d030 msgr2=0x7f77d415edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 shutdown_connections 2024-03-14T12:18:18.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.175+0000 7f77dc3a9640 1 -- 172.21.15.82:0/3374333283 wait complete. 2024-03-14T12:18:18.187 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T12:18:18.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:18.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.285+0000 7fc352b37640 1 Processor -- start 2024-03-14T12:18:18.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.285+0000 7fc352b37640 1 -- start start 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc352b37640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c06ed90 0x7fc34c160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc352b37640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc34c0cc770 0x7fc34c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc352b37640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c06fc70 con 0x7fc34c0b4190 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc352b37640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34c15d340 con 0x7fc34c0cc770 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc352b37640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0ce610 con 0x7fc34c06ed90 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc3510d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc34c0cc770 0x7fc34c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc34bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc34c0b4190 0x7fc34c06f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44794/0 (socket says 172.21.15.82:44794) 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc3510d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc34c0cc770 0x7fc34c0ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56052/0 (socket says 172.21.15.82:56052) 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.286+0000 7fc34bfff640 1 -- 172.21.15.82:0/646048860 learned_addr learned my addr 172.21.15.82:0/646048860 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.287+0000 7fc3508d3640 1 --2- 172.21.15.82:0/646048860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c06ed90 0x7fc34c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.287+0000 7fc3510d4640 1 -- 172.21.15.82:0/646048860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c06ed90 msgr2=0x7fc34c160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.287+0000 7fc3510d4640 1 --2- 172.21.15.82:0/646048860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c06ed90 0x7fc34c160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.287+0000 7fc34b7fe640 1 -- 172.21.15.82:0/646048860 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857071623 0 0) 0x7fc340001110 con 0x7fc34c0b4190 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.287+0000 7fc3510d4640 1 -- 172.21.15.82:0/646048860 >> v1:172.21.15.82:6789/0 conn(0x7fc34c0b4190 legacy=0x7fc34c06f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.287+0000 7fc3510d4640 1 -- 172.21.15.82:0/646048860 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0ce8f0 con 0x7fc34c0cc770 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.287+0000 7fc3510d4640 1 --2- 172.21.15.82:0/646048860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c0ccc00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc344009d90 tx=0x7fc34402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a999611483449a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc34b7fe640 1 -- 172.21.15.82:0/646048860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344039070 con 0x7fc34c0cc770 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc34b7fe640 1 -- 172.21.15.82:0/646048860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34402cc50 con 0x7fc34c0cc770 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc34b7fe640 1 -- 172.21.15.82:0/646048860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344034440 con 0x7fc34c0cc770 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 -- 172.21.15.82:0/646048860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 msgr2=0x7fc34c0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 --2- 172.21.15.82:0/646048860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c0ccc00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc344009d90 tx=0x7fc34402bea0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 -- 172.21.15.82:0/646048860 shutdown_connections 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 --2- 172.21.15.82:0/646048860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c0ccc00 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 --2- 172.21.15.82:0/646048860 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c06ed90 0x7fc34c160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 -- 172.21.15.82:0/646048860 >> 172.21.15.82:0/646048860 conn(0x7fc34c0b3670 msgr2=0x7fc34c0b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 -- 172.21.15.82:0/646048860 shutdown_connections 2024-03-14T12:18:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.288+0000 7fc352b37640 1 -- 172.21.15.82:0/646048860 wait complete. 2024-03-14T12:18:18.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.289+0000 7fc352b37640 1 Processor -- start 2024-03-14T12:18:18.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.289+0000 7fc352b37640 1 -- start start 2024-03-14T12:18:18.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc352b37640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c06ed90 0x7fc34c1f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc352b37640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c0b4190 0x7fc34c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc352b37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc352b37640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0cf190 con 0x7fc34c0cc770 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc352b37640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0ceeb0 con 0x7fc34c06ed90 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc352b37640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0cd140 con 0x7fc34c0b4190 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc3510d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc3510d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c14e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56068/0 (socket says 172.21.15.82:56068) 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.290+0000 7fc3510d4640 1 -- 172.21.15.82:0/1079804752 learned_addr learned my addr 172.21.15.82:0/1079804752 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc34bfff640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c0b4190 0x7fc34c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3510d4640 1 -- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c0b4190 msgr2=0x7fc34c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3508d3640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c06ed90 0x7fc34c1f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3510d4640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c0b4190 0x7fc34c158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3510d4640 1 -- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c06ed90 msgr2=0x7fc34c1f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3510d4640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c06ed90 0x7fc34c1f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3510d4640 1 -- 172.21.15.82:0/1079804752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0c70a0 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3510d4640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c14e8a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc344009d90 tx=0x7fc344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3497fa640 1 -- 172.21.15.82:0/1079804752 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344044020 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3497fa640 1 -- 172.21.15.82:0/1079804752 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34402ca50 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c158cc0 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc3497fa640 1 -- 172.21.15.82:0/1079804752 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344039040 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.291+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c15b9b0 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.292+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34c04f050 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.292+0000 7fc3497fa640 1 -- 172.21.15.82:0/1079804752 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc344047020 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.293+0000 7fc3497fa640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc33806bbd0 0x7fc33806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.293+0000 7fc3508d3640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc33806bbd0 0x7fc33806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.293+0000 7fc3497fa640 1 -- 172.21.15.82:0/1079804752 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fc3440ad440 con 0x7fc34c0cc770 2024-03-14T12:18:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.293+0000 7fc3508d3640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc33806bbd0 0x7fc33806e090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc33c006fd0 tx=0x7fc33c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.297+0000 7fc3497fa640 1 -- 172.21.15.82:0/1079804752 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc34404e030 con 0x7fc34c0cc770 2024-03-14T12:18:18.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.505+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc34c070480 con 0x7fc34c0cc770 2024-03-14T12:18:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.506+0000 7fc3497fa640 1 -- 172.21.15.82:0/1079804752 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fc34407bed0 con 0x7fc34c0cc770 2024-03-14T12:18:18.506 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:18.506 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:18:06.362029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":21,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:18.506 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 40 2024-03-14T12:18:18.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.507+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc33806bbd0 msgr2=0x7fc33806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.507+0000 7fc352b37640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc33806bbd0 0x7fc33806e090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc33c006fd0 tx=0x7fc33c008040 comp rx=0 tx=0).stop 2024-03-14T12:18:18.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.507+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 msgr2=0x7fc34c14e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.507+0000 7fc352b37640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c14e8a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc344009d90 tx=0x7fc344004a40 comp rx=0 tx=0).stop 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 shutdown_connections 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34c0cc770 0x7fc34c14e8a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc33806bbd0 0x7fc33806e090 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc34c0b4190 0x7fc34c158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 --2- 172.21.15.82:0/1079804752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c06ed90 0x7fc34c1f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 >> 172.21.15.82:0/1079804752 conn(0x7fc34c0b3670 msgr2=0x7fc34c157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 shutdown_connections 2024-03-14T12:18:18.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.508+0000 7fc352b37640 1 -- 172.21.15.82:0/1079804752 wait complete. 2024-03-14T12:18:18.519 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:18.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.619+0000 7f52ac759640 1 Processor -- start 2024-03-14T12:18:18.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.619+0000 7f52ac759640 1 -- start start 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52ac759640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52a40d0760 0x7f52a40d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52ac759640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52ac759640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a40d1080 con 0x7f52a40d1da0 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52ac759640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a40cf8c0 con 0x7f52a40d0760 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52ac759640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40cfba0 con 0x7f52a40cf370 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52aa4f5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52a40d0760 0x7f52a40d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52aacf6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f52a40d1da0 0x7f52a40d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44802/0 (socket says 172.21.15.82:44802) 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52aacf6640 1 -- 172.21.15.82:0/2626912516 learned_addr learned my addr 172.21.15.82:0/2626912516 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52a9cf4640 1 --2- 172.21.15.82:0/2626912516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52aa4f5640 1 -- 172.21.15.82:0/2626912516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 msgr2=0x7f52a40d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:18.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52aa4f5640 1 --2- 172.21.15.82:0/2626912516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52aa4f5640 1 -- 172.21.15.82:0/2626912516 >> v1:172.21.15.82:6789/0 conn(0x7f52a40d1da0 legacy=0x7f52a40d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52a94f3640 1 -- 172.21.15.82:0/2626912516 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825414156 0 0) 0x7f52a40d1080 con 0x7f52a40d1da0 2024-03-14T12:18:18.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.620+0000 7f52aa4f5640 1 -- 172.21.15.82:0/2626912516 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40b6c40 con 0x7f52a40d0760 2024-03-14T12:18:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.621+0000 7f52aa4f5640 1 --2- 172.21.15.82:0/2626912516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40d0b40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199dde25b05004ea server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.621+0000 7f52a94f3640 1 -- 172.21.15.82:0/2626912516 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0034070 con 0x7f52a40d0760 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.621+0000 7f52a94f3640 1 -- 172.21.15.82:0/2626912516 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a0030610 con 0x7f52a40d0760 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.621+0000 7f52a94f3640 1 -- 172.21.15.82:0/2626912516 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00309e0 con 0x7f52a40d0760 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 -- 172.21.15.82:0/2626912516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 msgr2=0x7f52a40d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 --2- 172.21.15.82:0/2626912516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40d0b40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f52a00097a0 tx=0x7f52a002b960 comp rx=0 tx=0).stop 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 -- 172.21.15.82:0/2626912516 shutdown_connections 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 --2- 172.21.15.82:0/2626912516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 --2- 172.21.15.82:0/2626912516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40d0b40 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 -- 172.21.15.82:0/2626912516 >> 172.21.15.82:0/2626912516 conn(0x7f52a409d030 msgr2=0x7f52a40d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 -- 172.21.15.82:0/2626912516 shutdown_connections 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.622+0000 7f52ac759640 1 -- 172.21.15.82:0/2626912516 wait complete. 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.623+0000 7f52ac759640 1 Processor -- start 2024-03-14T12:18:18.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.623+0000 7f52ac759640 1 -- start start 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52ac759640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52ac759640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52ac759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52a40d1da0 0x7f52a40ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52ac759640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40d4ba0 con 0x7f52a40d0760 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52ac759640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40b7200 con 0x7f52a40d1da0 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52ac759640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41617a0 con 0x7f52a40cf370 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56078/0 (socket says 172.21.15.82:56078) 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 -- 172.21.15.82:0/737446141 learned_addr learned my addr 172.21.15.82:0/737446141 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52aa4f5640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52aacf6640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52a40d1da0 0x7f52a40ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 -- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 msgr2=0x7f52a40c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 -- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52a40d1da0 msgr2=0x7f52a40ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52a40d1da0 0x7f52a40ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 -- 172.21.15.82:0/737446141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40c8a80 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52aacf6640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52a40d1da0 0x7f52a40ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.624+0000 7f52a9cf4640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40c9b20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f529400d620 tx=0x7f529400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.625+0000 7f529affd640 1 -- 172.21.15.82:0/737446141 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400b620 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.625+0000 7f529affd640 1 -- 172.21.15.82:0/737446141 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5294019070 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.625+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a40c8d60 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.625+0000 7f529affd640 1 -- 172.21.15.82:0/737446141 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400ba00 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.625+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a40c9320 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.626+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a404f050 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.626+0000 7f529affd640 1 -- 172.21.15.82:0/737446141 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f529400bba0 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.627+0000 7f529affd640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f528806bbd0 0x7f528806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.627+0000 7f529affd640 1 -- 172.21.15.82:0/737446141 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f529408af80 con 0x7f52a40d0760 2024-03-14T12:18:18.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.627+0000 7f52aa4f5640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f528806bbd0 0x7f528806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.627+0000 7f52aa4f5640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f528806bbd0 0x7f528806e090 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f52a0036000 tx=0x7f52a003b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.631+0000 7f529affd640 1 -- 172.21.15.82:0/737446141 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5294059980 con 0x7f52a40d0760 2024-03-14T12:18:18.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.837+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52a40ce110 con 0x7f52a40d0760 2024-03-14T12:18:18.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.838+0000 7f529affd640 1 -- 172.21.15.82:0/737446141 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f5294059280 con 0x7f52a40d0760 2024-03-14T12:18:18.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:18.838 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T12:16:59.493058+0000","modified":"2024-03-14T12:18:06.362029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":21,"addr":"172.21.15.82:6837/2268648862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2268648862},{"type":"v1","addr":"172.21.15.82:6837","nonce":2268648862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T12:18:18.838 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 40 2024-03-14T12:18:18.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.839+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f528806bbd0 msgr2=0x7f528806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.839+0000 7f52ac759640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f528806bbd0 0x7f528806e090 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f52a0036000 tx=0x7f52a003b040 comp rx=0 tx=0).stop 2024-03-14T12:18:18.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.839+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 msgr2=0x7f52a40c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.839+0000 7f52ac759640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40c9b20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f529400d620 tx=0x7f529400dae0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 shutdown_connections 2024-03-14T12:18:18.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52a40d1da0 0x7f52a40ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52a40d0760 0x7f52a40c9b20 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f528806bbd0 0x7f528806e090 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 --2- 172.21.15.82:0/737446141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52a40cf370 0x7f52a40c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 >> 172.21.15.82:0/737446141 conn(0x7f52a409d030 msgr2=0x7f52a406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:18.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 shutdown_connections 2024-03-14T12:18:18.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.840+0000 7f52ac759640 1 -- 172.21.15.82:0/737446141 wait complete. 2024-03-14T12:18:18.851 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:18:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.950+0000 7fca2a329640 1 Processor -- start 2024-03-14T12:18:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.951+0000 7fca2a329640 1 -- start start 2024-03-14T12:18:18.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.951+0000 7fca2a329640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fca240b8730 0x7fca24168b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.951+0000 7fca2a329640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca240cf8c0 con 0x7fca240d0760 2024-03-14T12:18:18.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.951+0000 7fca2a329640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca240cfba0 con 0x7fca240cf370 2024-03-14T12:18:18.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.951+0000 7fca2a329640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca240b6c40 con 0x7fca240b8730 2024-03-14T12:18:18.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.951+0000 7fca23fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fca240d0760 0x7fca240d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44804/0 (socket says 172.21.15.82:44804) 2024-03-14T12:18:18.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.951+0000 7fca23fff640 1 -- 172.21.15.82:0/994438736 learned_addr learned my addr 172.21.15.82:0/994438736 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:18.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270125772 0 0) 0x7fca240cf8c0 con 0x7fca240d0760 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca288c6640 1 --2- 172.21.15.82:0/994438736 >> v2:172.21.15.164:3300/0 conn(0x7fca240b8730 0x7fca24168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca0c003520 con 0x7fca240d0760 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155066489 0 0) 0x7fca0c003520 con 0x7fca240d0760 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 >> v2:172.21.15.164:3300/0 conn(0x7fca240b8730 msgr2=0x7fca24168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 --2- 172.21.15.82:0/994438736 >> v2:172.21.15.164:3300/0 conn(0x7fca240b8730 0x7fca24168b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 >> v1:172.21.15.164:6789/0 conn(0x7fca240cf370 legacy=0x7fca240d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240b6f20 con 0x7fca240d0760 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fca14001670 con 0x7fca240d0760 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 >> v1:172.21.15.82:6789/0 conn(0x7fca240d0760 legacy=0x7fca240d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca0c0039b0 0x7fca0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c004800 0x7fca0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0c0051c0 0x7fca0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca240b6c40 con 0x7fca0c0051c0 2024-03-14T12:18:18.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c00c070 con 0x7fca0c004800 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c00c270 con 0x7fca0c0039b0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca288c6640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0c0051c0 0x7fca0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca237fe640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c004800 0x7fca0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca23fff640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca0c0039b0 0x7fca0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca288c6640 1 -- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca0c0039b0 msgr2=0x7fca0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca288c6640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca0c0039b0 0x7fca0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca288c6640 1 -- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c004800 msgr2=0x7fca0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca288c6640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c004800 0x7fca0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca288c6640 1 -- 172.21.15.82:0/994438736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c003520 con 0x7fca0c0051c0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.952+0000 7fca237fe640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c004800 0x7fca0c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.953+0000 7fca288c6640 1 -- 172.21.15.82:0/994438736 reap_dead start 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.953+0000 7fca288c6640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0c0051c0 0x7fca0c00b9c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009fc0 tx=0x7fca0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b668689827b70de1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.953+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c009c10 con 0x7fca0c0051c0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.953+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c02cca0 con 0x7fca0c0051c0 2024-03-14T12:18:18.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.953+0000 7fca22ffd640 1 -- 172.21.15.82:0/994438736 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c003bd0 con 0x7fca0c0051c0 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.954+0000 7fca2a329640 1 -- 172.21.15.82:0/994438736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0c0051c0 msgr2=0x7fca0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.954+0000 7fca2a329640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0c0051c0 0x7fca0c00b9c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009fc0 tx=0x7fca0c005830 comp rx=0 tx=0).stop 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.954+0000 7fca2a329640 1 -- 172.21.15.82:0/994438736 shutdown_connections 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.954+0000 7fca2a329640 1 --2- 172.21.15.82:0/994438736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0c0051c0 0x7fca0c00b9c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.954+0000 7fca2a329640 1 -- 172.21.15.82:0/994438736 >> 172.21.15.82:0/994438736 conn(0x7fca2409d030 msgr2=0x7fca240d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.954+0000 7fca2a329640 1 -- 172.21.15.82:0/994438736 shutdown_connections 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.954+0000 7fca2a329640 1 -- 172.21.15.82:0/994438736 wait complete. 2024-03-14T12:18:18.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.955+0000 7fca2a329640 1 Processor -- start 2024-03-14T12:18:18.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.955+0000 7fca2a329640 1 -- start start 2024-03-14T12:18:18.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca2a329640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c0051c0 0x7fca241560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca2a329640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca240cdd60 0x7fca24149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca2a329640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca240cae90 0x7fca240d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca2a329640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca24161920 con 0x7fca240cdd60 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca2a329640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca241617a0 con 0x7fca0c0051c0 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca2a329640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca240d0b40 con 0x7fca240cae90 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca240cdd60 0x7fca24149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca240cdd60 0x7fca24149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56108/0 (socket says 172.21.15.82:56108) 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 -- 172.21.15.82:0/2391706659 learned_addr learned my addr 172.21.15.82:0/2391706659 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca288c6640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca240cae90 0x7fca240d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca23fff640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c0051c0 0x7fca241560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 -- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca240cae90 msgr2=0x7fca240d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca240cae90 0x7fca240d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 -- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c0051c0 msgr2=0x7fca241560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c0051c0 0x7fca241560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 -- 172.21.15.82:0/2391706659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca241f3cd0 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca23fff640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c0051c0 0x7fca241560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca288c6640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca240cae90 0x7fca240d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.956+0000 7fca237fe640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca240cdd60 0x7fca24149ba0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fca1800e4d0 tx=0x7fca1800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.957+0000 7fca20ff9640 1 -- 172.21.15.82:0/2391706659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca180052e0 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.957+0000 7fca20ff9640 1 -- 172.21.15.82:0/2391706659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18018070 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.957+0000 7fca20ff9640 1 -- 172.21.15.82:0/2391706659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18003810 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.957+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca240cca20 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.957+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca240cccc0 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.958+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2404f050 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.958+0000 7fca20ff9640 1 -- 172.21.15.82:0/2391706659 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fca180039b0 con 0x7fca240cdd60 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.959+0000 7fca20ff9640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca1006bbd0 0x7fca1006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.959+0000 7fca23fff640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca1006bbd0 0x7fca1006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.959+0000 7fca20ff9640 1 -- 172.21.15.82:0/2391706659 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fca1808a820 con 0x7fca240cdd60 2024-03-14T12:18:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.959+0000 7fca23fff640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca1006bbd0 0x7fca1006e090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fca14001eb0 tx=0x7fca14009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:18.963+0000 7fca20ff9640 1 -- 172.21.15.82:0/2391706659 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fca18059220 con 0x7fca240cdd60 2024-03-14T12:18:19.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.137+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca240d0540 con 0x7fca240cdd60 2024-03-14T12:18:19.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.138+0000 7fca20ff9640 1 -- 172.21.15.82:0/2391706659 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+17008 (secure 0 0 0) 0x7fca18058b20 con 0x7fca240cdd60 2024-03-14T12:18:19.139 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:19.139 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":73,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:18:05.349077+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-14T12:16:57.716705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-14T12:16:58.458303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.17","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6834/4033072140":"2024-03-15T12:17:15.041576+0000","192.168.0.1:0/895829753":"2024-03-14T13:18:05.110010+0000","172.21.15.164:0/1140720987":"2024-03-14T13:18:05.110143+0000","172.21.15.164:6832/4033072140":"2024-03-15T12:17:15.041576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca1006bbd0 msgr2=0x7fca1006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca1006bbd0 0x7fca1006e090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fca14001eb0 tx=0x7fca14009040 comp rx=0 tx=0).stop 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca240cdd60 msgr2=0x7fca24149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca240cdd60 0x7fca24149ba0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fca1800e4d0 tx=0x7fca1800e990 comp rx=0 tx=0).stop 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 shutdown_connections 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fca1006bbd0 0x7fca1006e090 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca240cae90 0x7fca240d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca240cdd60 0x7fca24149ba0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 --2- 172.21.15.82:0/2391706659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca0c0051c0 0x7fca241560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 >> 172.21.15.82:0/2391706659 conn(0x7fca2409d030 msgr2=0x7fca2415ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:19.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 shutdown_connections 2024-03-14T12:18:19.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.140+0000 7fca2a329640 1 -- 172.21.15.82:0/2391706659 wait complete. 2024-03-14T12:18:19.152 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T12:18:19.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.252+0000 7faf57bd1640 1 Processor -- start 2024-03-14T12:18:19.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf57bd1640 1 -- start start 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf57bd1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf500b4190 0x7faf5015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf57bd1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf500cd0d0 0x7faf5015f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf57bd1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500cd540 con 0x7faf5015b9e0 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf57bd1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500cf730 con 0x7faf500b4190 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf57bd1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500cd810 con 0x7faf500cd0d0 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf5516c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf500b4190 0x7faf5015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf5516c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf500b4190 0x7faf5015bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56120/0 (socket says 172.21.15.82:56120) 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.253+0000 7faf5516c640 1 -- 172.21.15.82:0/4021179823 learned_addr learned my addr 172.21.15.82:0/4021179823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.254+0000 7faf5616e640 1 --2- 172.21.15.82:0/4021179823 >> v2:172.21.15.164:3300/0 conn(0x7faf500cd0d0 0x7faf5015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.254+0000 7faf5516c640 1 -- 172.21.15.82:0/4021179823 >> v2:172.21.15.164:3300/0 conn(0x7faf500cd0d0 msgr2=0x7faf5015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.254+0000 7faf5516c640 1 --2- 172.21.15.82:0/4021179823 >> v2:172.21.15.164:3300/0 conn(0x7faf500cd0d0 0x7faf5015f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.254+0000 7faf5516c640 1 -- 172.21.15.82:0/4021179823 >> v1:172.21.15.164:6789/0 conn(0x7faf5015b9e0 legacy=0x7faf500d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.254+0000 7faf5516c640 1 -- 172.21.15.82:0/4021179823 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500cf980 con 0x7faf500b4190 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.254+0000 7faf5516c640 1 --2- 172.21.15.82:0/4021179823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500b4190 0x7faf5015bdc0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7faf500d14f0 tx=0x7faf4002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4aba7a890ab08f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf5496b640 1 -- 172.21.15.82:0/4021179823 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4002e070 con 0x7faf500b4190 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf5496b640 1 -- 172.21.15.82:0/4021179823 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40037070 con 0x7faf500b4190 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf5496b640 1 -- 172.21.15.82:0/4021179823 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40031a00 con 0x7faf500b4190 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 -- 172.21.15.82:0/4021179823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500b4190 msgr2=0x7faf5015bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 --2- 172.21.15.82:0/4021179823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500b4190 0x7faf5015bdc0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7faf500d14f0 tx=0x7faf4002b880 comp rx=0 tx=0).stop 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 -- 172.21.15.82:0/4021179823 shutdown_connections 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 --2- 172.21.15.82:0/4021179823 >> v2:172.21.15.164:3300/0 conn(0x7faf500cd0d0 0x7faf5015f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 --2- 172.21.15.82:0/4021179823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500b4190 0x7faf5015bdc0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 -- 172.21.15.82:0/4021179823 >> 172.21.15.82:0/4021179823 conn(0x7faf500b3670 msgr2=0x7faf500b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 -- 172.21.15.82:0/4021179823 shutdown_connections 2024-03-14T12:18:19.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.255+0000 7faf57bd1640 1 -- 172.21.15.82:0/4021179823 wait complete. 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.256+0000 7faf57bd1640 1 Processor -- start 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.256+0000 7faf57bd1640 1 -- start start 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf57bd1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4190 0x7faf500b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf57bd1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500cd0d0 0x7faf500b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf57bd1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5015b9e0 0x7faf50158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf57bd1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5015fe80 con 0x7faf500cd0d0 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf57bd1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500cff40 con 0x7faf500b4190 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf57bd1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf50160000 con 0x7faf5015b9e0 2024-03-14T12:18:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500cd0d0 0x7faf500b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500cd0d0 0x7faf500b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56136/0 (socket says 172.21.15.82:56136) 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 -- 172.21.15.82:0/94615458 learned_addr learned my addr 172.21.15.82:0/94615458 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5616e640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5015b9e0 0x7faf50158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 -- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5015b9e0 msgr2=0x7faf50158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5596d640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4190 0x7faf500b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5015b9e0 0x7faf50158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 -- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4190 msgr2=0x7faf500b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4190 0x7faf500b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.257+0000 7faf5516c640 1 -- 172.21.15.82:0/94615458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500b6b80 con 0x7faf500cd0d0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.258+0000 7faf5516c640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500cd0d0 0x7faf500b5430 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7faf4002b8c0 tx=0x7faf400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.258+0000 7faf467fc640 1 -- 172.21.15.82:0/94615458 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40032670 con 0x7faf500cd0d0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.258+0000 7faf467fc640 1 -- 172.21.15.82:0/94615458 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40032830 con 0x7faf500cd0d0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.258+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf500b6e60 con 0x7faf500cd0d0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.258+0000 7faf467fc640 1 -- 172.21.15.82:0/94615458 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40032c20 con 0x7faf500cd0d0 2024-03-14T12:18:19.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.258+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf500b7420 con 0x7faf500cd0d0 2024-03-14T12:18:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.259+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5004f050 con 0x7faf500cd0d0 2024-03-14T12:18:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.259+0000 7faf467fc640 1 -- 172.21.15.82:0/94615458 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faf4004c060 con 0x7faf500cd0d0 2024-03-14T12:18:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.260+0000 7faf467fc640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf3406bc00 0x7faf3406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.260+0000 7faf467fc640 1 -- 172.21.15.82:0/94615458 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5939+0+0 (secure 0 0 0) 0x7faf400ac5b0 con 0x7faf500cd0d0 2024-03-14T12:18:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.260+0000 7faf5596d640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf3406bc00 0x7faf3406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.260+0000 7faf5596d640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf3406bc00 0x7faf3406e0c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faf4c007fd0 tx=0x7faf4c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:19.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.264+0000 7faf467fc640 1 -- 172.21.15.82:0/94615458 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7faf40040070 con 0x7faf500cd0d0 2024-03-14T12:18:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.474+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf5015b560 con 0x7faf500cd0d0 2024-03-14T12:18:19.475 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-14T12:18:19.474+0000 7f8354c85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T12:18:19.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.476+0000 7faf467fc640 1 -- 172.21.15.82:0/94615458 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf4007afb0 con 0x7faf500cd0d0 2024-03-14T12:18:19.476 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T12:18:19.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf3406bc00 msgr2=0x7faf3406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf3406bc00 0x7faf3406e0c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faf4c007fd0 tx=0x7faf4c009040 comp rx=0 tx=0).stop 2024-03-14T12:18:19.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500cd0d0 msgr2=0x7faf500b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500cd0d0 0x7faf500b5430 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7faf4002b8c0 tx=0x7faf400046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 shutdown_connections 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf3406bc00 0x7faf3406e0c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5015b9e0 0x7faf50158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf500cd0d0 0x7faf500b5430 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 --2- 172.21.15.82:0/94615458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4190 0x7faf500b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 >> 172.21.15.82:0/94615458 conn(0x7faf500b3670 msgr2=0x7faf5006e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 shutdown_connections 2024-03-14T12:18:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.479+0000 7faf57bd1640 1 -- 172.21.15.82:0/94615458 wait complete. 2024-03-14T12:18:19.481 INFO:tasks.ceph.mds.c.smithi082.stderr: -499> 2024-03-14T12:18:10.440+0000 7f27bed56640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T12:18:19.491 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T12:18:19.507 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-14T12:18:19.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.595+0000 7fb127420640 1 Processor -- start 2024-03-14T12:18:19.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.595+0000 7fb127420640 1 -- start start 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb127420640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 0x7fb1200b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb127420640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1200b41c0 con 0x7fb120140e70 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb127420640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb120161c10 con 0x7fb120143c60 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb127420640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb120161ef0 con 0x7fb1201626f0 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb1251bc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb120140e70 0x7fb120143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44828/0 (socket says 172.21.15.82:44828) 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb1251bc640 1 -- 172.21.15.82:0/2016828906 learned_addr learned my addr 172.21.15.82:0/2016828906 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865622559 0 0) 0x7fb1200b41c0 con 0x7fb120140e70 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb10c003520 con 0x7fb120140e70 2024-03-14T12:18:19.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb1259bd640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 0x7fb1200b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4102422545 0 0) 0x7fb10c003520 con 0x7fb120140e70 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 msgr2=0x7fb1200b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 0x7fb1200b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 >> v1:172.21.15.164:6789/0 conn(0x7fb120143c60 legacy=0x7fb1200af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1201621d0 con 0x7fb120140e70 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7fb11c001670 con 0x7fb120140e70 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 >> v1:172.21.15.82:6789/0 conn(0x7fb120140e70 legacy=0x7fb120143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb10c0039e0 0x7fb10c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 0x7fb10c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c004d40 0x7fb10c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb120161ef0 con 0x7fb10c004d40 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c00ba70 con 0x7fb10c0039e0 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c00bc70 con 0x7fb1201626f0 2024-03-14T12:18:19.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.596+0000 7fb1259bd640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c004d40 0x7fb10c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1251bc640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb10c0039e0 0x7fb10c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1249bb640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 0x7fb10c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1259bd640 1 -- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 msgr2=0x7fb10c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1259bd640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1201626f0 0x7fb10c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1259bd640 1 -- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb10c0039e0 msgr2=0x7fb10c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1259bd640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb10c0039e0 0x7fb10c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1259bd640 1 -- 172.21.15.82:0/2016828906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c003520 con 0x7fb10c004d40 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.597+0000 7fb1259bd640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c004d40 0x7fb10c0051c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb120071640 tx=0x7fb10c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41ca24357528a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118039070 con 0x7fb10c004d40 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11802cb90 con 0x7fb10c004d40 2024-03-14T12:18:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb117fff640 1 -- 172.21.15.82:0/2016828906 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118009c10 con 0x7fb10c004d40 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb127420640 1 -- 172.21.15.82:0/2016828906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c004d40 msgr2=0x7fb10c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb127420640 1 --2- 172.21.15.82:0/2016828906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c004d40 0x7fb10c0051c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb120071640 tx=0x7fb10c005830 comp rx=0 tx=0).stop 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb1259bd640 1 -- 172.21.15.82:0/2016828906 reap_dead start 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb127420640 1 -- 172.21.15.82:0/2016828906 shutdown_connections 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb127420640 1 -- 172.21.15.82:0/2016828906 >> 172.21.15.82:0/2016828906 conn(0x7fb1200b2830 msgr2=0x7fb1200b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb127420640 1 -- 172.21.15.82:0/2016828906 shutdown_connections 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.598+0000 7fb127420640 1 -- 172.21.15.82:0/2016828906 wait complete. 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.599+0000 7fb127420640 1 Processor -- start 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.599+0000 7fb127420640 1 -- start start 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb127420640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120156f30 0x7fb1200cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb127420640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1200cfe50 0x7fb12006e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb127420640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12006eca0 0x7fb1201422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb127420640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200b3f80 con 0x7fb120156f30 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb127420640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200b3e00 con 0x7fb12006eca0 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb127420640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12015eeb0 con 0x7fb1200cfe50 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120156f30 0x7fb1200cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120156f30 0x7fb1200cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56168/0 (socket says 172.21.15.82:56168) 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 -- 172.21.15.82:0/2663357522 learned_addr learned my addr 172.21.15.82:0/2663357522 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1259bd640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12006eca0 0x7fb1201422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1249bb640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1200cfe50 0x7fb12006e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 -- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1200cfe50 msgr2=0x7fb12006e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1200cfe50 0x7fb12006e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 -- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12006eca0 msgr2=0x7fb1201422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12006eca0 0x7fb1201422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1251bc640 1 -- 172.21.15.82:0/2663357522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120157310 con 0x7fb120156f30 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.600+0000 7fb1249bb640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1200cfe50 0x7fb12006e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.601+0000 7fb1251bc640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120156f30 0x7fb1200cdf20 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0072c0 tx=0x7fb11c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.601+0000 7fb115ffb640 1 -- 172.21.15.82:0/2663357522 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c00f9a0 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.601+0000 7fb115ffb640 1 -- 172.21.15.82:0/2663357522 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c018070 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.601+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1200ce5b0 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.601+0000 7fb115ffb640 1 -- 172.21.15.82:0/2663357522 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c00fd80 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.601+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1200ceb70 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.602+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12004f050 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.602+0000 7fb115ffb640 1 -- 172.21.15.82:0/2663357522 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb11c01d460 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.602+0000 7fb115ffb640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb10406bbd0 0x7fb10406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.603+0000 7fb115ffb640 1 -- 172.21.15.82:0/2663357522 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fb11c08ad50 con 0x7fb120156f30 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.603+0000 7fb1249bb640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb10406bbd0 0x7fb10406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.603+0000 7fb1249bb640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb10406bbd0 0x7fb10406e090 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb12015f9f0 tx=0x7fb110009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:19.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.607+0000 7fb115ffb640 1 -- 172.21.15.82:0/2663357522 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb11c08f070 con 0x7fb120156f30 2024-03-14T12:18:19.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:19.814+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb1200b2440 con 0x7fb120156f30 2024-03-14T12:18:20.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.480+0000 7fb115ffb640 1 -- 172.21.15.82:0/2663357522 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb11c059700 con 0x7fb120156f30 2024-03-14T12:18:20.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb10406bbd0 msgr2=0x7fb10406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:20.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb10406bbd0 0x7fb10406e090 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb12015f9f0 tx=0x7fb110009040 comp rx=0 tx=0).stop 2024-03-14T12:18:20.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120156f30 msgr2=0x7fb1200cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:20.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120156f30 0x7fb1200cdf20 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0072c0 tx=0x7fb11c007780 comp rx=0 tx=0).stop 2024-03-14T12:18:20.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 shutdown_connections 2024-03-14T12:18:20.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12006eca0 0x7fb1201422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fb10406bbd0 0x7fb10406e090 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.482+0000 7fb127420640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1200cfe50 0x7fb12006e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.483+0000 7fb127420640 1 --2- 172.21.15.82:0/2663357522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120156f30 0x7fb1200cdf20 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.483+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 >> 172.21.15.82:0/2663357522 conn(0x7fb1200b2830 msgr2=0x7fb1200ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.483+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 shutdown_connections 2024-03-14T12:18:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.483+0000 7fb127420640 1 -- 172.21.15.82:0/2663357522 wait complete. 2024-03-14T12:18:20.496 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T12:18:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.644+0000 7fa341eea640 1 Processor -- start 2024-03-14T12:18:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.645+0000 7fa341eea640 1 -- start start 2024-03-14T12:18:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.645+0000 7fa341eea640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0d4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.645+0000 7fa341eea640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa33c0c9020 0x7fa33c0d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.645+0000 7fa341eea640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c161a70 con 0x7fa33c161690 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.645+0000 7fa341eea640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c0cf5e0 con 0x7fa33c0c9020 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.645+0000 7fa341eea640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0cf8c0 con 0x7fa33c0d0e50 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33affd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa33c0c9020 0x7fa33c0d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa33c0c9020 0x7fa33c0d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:38244/0 (socket says 172.21.15.82:38244) 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33affd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0d4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:46854/0 (socket says 172.21.15.82:46854) 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33bfff640 1 -- 172.21.15.82:0/3539097132 learned_addr learned my addr 172.21.15.82:0/3539097132 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33b7fe640 1 --1- 172.21.15.82:0/3539097132 >> v1:172.21.15.164:6789/0 conn(0x7fa33c161690 0x7fa33c0d0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:36280/0 (socket says 172.21.15.82:36280) 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33a7fc640 1 -- 172.21.15.82:0/3539097132 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288917997 0 0) 0x7fa33c161a70 con 0x7fa33c161690 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33a7fc640 1 -- 172.21.15.82:0/3539097132 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa324003520 con 0x7fa33c161690 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33bfff640 1 -- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 msgr2=0x7fa33c0d4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33bfff640 1 --2- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0d4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33bfff640 1 -- 172.21.15.82:0/3539097132 >> v1:172.21.15.164:6789/0 conn(0x7fa33c161690 legacy=0x7fa33c0d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:20.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33bfff640 1 -- 172.21.15.82:0/3539097132 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c0cfba0 con 0x7fa33c0c9020 2024-03-14T12:18:20.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.646+0000 7fa33affd640 1 --2- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:18:20.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.647+0000 7fa33bfff640 1 --2- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 0x7fa33c0d19d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa334009e50 tx=0x7fa33402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a20c443d1e9f14c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:20.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.647+0000 7fa33a7fc640 1 -- 172.21.15.82:0/3539097132 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33402cb00 con 0x7fa33c0c9020 2024-03-14T12:18:20.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.647+0000 7fa33a7fc640 1 -- 172.21.15.82:0/3539097132 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33402cca0 con 0x7fa33c0c9020 2024-03-14T12:18:20.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.647+0000 7fa33a7fc640 1 -- 172.21.15.82:0/3539097132 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334034610 con 0x7fa33c0c9020 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 -- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 msgr2=0x7fa33c0d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 --2- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 0x7fa33c0d19d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa334009e50 tx=0x7fa33402bf60 comp rx=0 tx=0).stop 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 -- 172.21.15.82:0/3539097132 shutdown_connections 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 --2- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 0x7fa33c0d19d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 --2- 172.21.15.82:0/3539097132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 -- 172.21.15.82:0/3539097132 >> 172.21.15.82:0/3539097132 conn(0x7fa33c0d3e00 msgr2=0x7fa33c0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 -- 172.21.15.82:0/3539097132 shutdown_connections 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.648+0000 7fa341eea640 1 -- 172.21.15.82:0/3539097132 wait complete. 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.649+0000 7fa341eea640 1 Processor -- start 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.649+0000 7fa341eea640 1 -- start start 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa341eea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 0x7fa33c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:20.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa341eea640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa341eea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa33c161690 0x7fa33c0b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa341eea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0d2090 con 0x7fa33c161690 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa341eea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0d1f10 con 0x7fa33c0c9020 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa341eea640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0d2210 con 0x7fa33c0d0e50 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa33c161690 0x7fa33c0b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa33c161690 0x7fa33c0b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56172/0 (socket says 172.21.15.82:56172) 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 -- 172.21.15.82:0/939657402 learned_addr learned my addr 172.21.15.82:0/939657402 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33b7fe640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 0x7fa33c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33affd640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 -- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 msgr2=0x7fa33c0cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 -- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 msgr2=0x7fa33c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 0x7fa33c15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:20.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 -- 172.21.15.82:0/939657402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c0cc7d0 con 0x7fa33c161690 2024-03-14T12:18:20.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.650+0000 7fa33bfff640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa33c161690 0x7fa33c0b69f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa334009e50 tx=0x7fa334004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:20.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.651+0000 7fa323fff640 1 -- 172.21.15.82:0/939657402 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334043020 con 0x7fa33c161690 2024-03-14T12:18:20.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.651+0000 7fa323fff640 1 -- 172.21.15.82:0/939657402 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33402ce50 con 0x7fa33c161690 2024-03-14T12:18:20.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.651+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c0b9810 con 0x7fa33c161690 2024-03-14T12:18:20.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.651+0000 7fa323fff640 1 -- 172.21.15.82:0/939657402 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334009510 con 0x7fa33c161690 2024-03-14T12:18:20.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.651+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c0cbcf0 con 0x7fa33c161690 2024-03-14T12:18:20.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.652+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c04f050 con 0x7fa33c161690 2024-03-14T12:18:20.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.652+0000 7fa323fff640 1 -- 172.21.15.82:0/939657402 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa334046020 con 0x7fa33c161690 2024-03-14T12:18:20.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.652+0000 7fa323fff640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32806bbd0 0x7fa32806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:20.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.653+0000 7fa323fff640 1 -- 172.21.15.82:0/939657402 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fa3340acef0 con 0x7fa33c161690 2024-03-14T12:18:20.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.653+0000 7fa33b7fe640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32806bbd0 0x7fa32806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:20.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.653+0000 7fa33b7fe640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32806bbd0 0x7fa32806e090 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa32c000f60 tx=0x7fa32c007cf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:20.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.657+0000 7fa323fff640 1 -- 172.21.15.82:0/939657402 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa33404d030 con 0x7fa33c161690 2024-03-14T12:18:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:20.827+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa33c06c890 con 0x7fa33c161690 2024-03-14T12:18:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.481+0000 7fa323fff640 1 -- 172.21.15.82:0/939657402 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa33407b2e0 con 0x7fa33c161690 2024-03-14T12:18:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-03-14T12:18:21.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32806bbd0 msgr2=0x7fa32806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:21.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32806bbd0 0x7fa32806e090 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa32c000f60 tx=0x7fa32c007cf0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa33c161690 msgr2=0x7fa33c0b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa33c161690 0x7fa33c0b69f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa334009e50 tx=0x7fa334004a40 comp rx=0 tx=0).stop 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 shutdown_connections 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa33c161690 0x7fa33c0b69f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fa32806bbd0 0x7fa32806e090 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa33c0d0e50 0x7fa33c0cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 --2- 172.21.15.82:0/939657402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c0c9020 0x7fa33c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 >> 172.21.15.82:0/939657402 conn(0x7fa33c0d3e00 msgr2=0x7fa33c0c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 shutdown_connections 2024-03-14T12:18:21.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.485+0000 7fa341eea640 1 -- 172.21.15.82:0/939657402 wait complete. 2024-03-14T12:18:21.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T12:18:21.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.603+0000 7feaf880b640 1 Processor -- start 2024-03-14T12:18:21.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.603+0000 7feaf880b640 1 -- start start 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.604+0000 7feaf880b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaf40716d0 0x7feaf4165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.604+0000 7feaf880b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf415e7e0 con 0x7feaf415ad50 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.604+0000 7feaf880b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf415eac0 con 0x7feaf406b420 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.604+0000 7feaf880b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415f3f0 con 0x7feaf40716d0 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.604+0000 7feaf259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feaf415ad50 0x7feaf406ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42792/0 (socket says 172.21.15.82:42792) 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.604+0000 7feaf259c640 1 -- 172.21.15.82:0/2652630976 learned_addr learned my addr 172.21.15.82:0/2652630976 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250550330 0 0) 0x7feaf415e7e0 con 0x7feaf415ad50 2024-03-14T12:18:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feadc003520 con 0x7feaf415ad50 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf2d9d640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaf40716d0 0x7feaf4165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2262697647 0 0) 0x7feadc003520 con 0x7feaf415ad50 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaf40716d0 msgr2=0x7feaf4165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaf40716d0 0x7feaf4165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 >> v1:172.21.15.164:6789/0 conn(0x7feaf406b420 legacy=0x7feaf4071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf415eda0 con 0x7feaf415ad50 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7feae4001670 con 0x7feaf415ad50 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 >> v1:172.21.15.82:6789/0 conn(0x7feaf415ad50 legacy=0x7feaf406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feadc0039e0 0x7feadc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feadc004800 0x7feadc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc0051c0 0x7feadc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415f3f0 con 0x7feadc004800 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc00c090 con 0x7feadc0039e0 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc00c290 con 0x7feadc0051c0 2024-03-14T12:18:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf2d9d640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaf40716d0 0x7feaf4165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf1d9b640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feadc004800 0x7feadc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf259c640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feadc0039e0 0x7feadc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf2d9d640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc0051c0 0x7feadc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf1d9b640 1 -- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc0051c0 msgr2=0x7feadc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf1d9b640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc0051c0 0x7feadc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf1d9b640 1 -- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feadc0039e0 msgr2=0x7feadc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf1d9b640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feadc0039e0 0x7feadc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.605+0000 7feaf1d9b640 1 -- 172.21.15.82:0/2652630976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc003520 con 0x7feadc004800 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.606+0000 7feaf1d9b640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feadc004800 0x7feadc004c80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7feae8001750 tx=0x7feadc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b686f4d5d2532ab server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.606+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8001370 con 0x7feadc004800 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.606+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae80052d0 con 0x7feadc004800 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.606+0000 7feaf159a640 1 -- 172.21.15.82:0/2652630976 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8003830 con 0x7feadc004800 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.607+0000 7feaf880b640 1 -- 172.21.15.82:0/2652630976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feadc004800 msgr2=0x7feadc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.607+0000 7feaf880b640 1 --2- 172.21.15.82:0/2652630976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feadc004800 0x7feadc004c80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7feae8001750 tx=0x7feadc005830 comp rx=0 tx=0).stop 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.607+0000 7feaf2d9d640 1 -- 172.21.15.82:0/2652630976 reap_dead start 2024-03-14T12:18:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.607+0000 7feaf880b640 1 -- 172.21.15.82:0/2652630976 shutdown_connections 2024-03-14T12:18:21.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.607+0000 7feaf880b640 1 -- 172.21.15.82:0/2652630976 >> 172.21.15.82:0/2652630976 conn(0x7feaf40b17b0 msgr2=0x7feaf40b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:21.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.607+0000 7feaf880b640 1 -- 172.21.15.82:0/2652630976 shutdown_connections 2024-03-14T12:18:21.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.607+0000 7feaf880b640 1 -- 172.21.15.82:0/2652630976 wait complete. 2024-03-14T12:18:21.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.608+0000 7feaf880b640 1 Processor -- start 2024-03-14T12:18:21.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.608+0000 7feaf880b640 1 -- start start 2024-03-14T12:18:21.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf880b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc004800 0x7feaf41460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf880b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf415b130 0x7feaf414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf880b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feaf4150220 0x7feaf415c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf880b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4072710 con 0x7feaf4150220 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf880b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4072590 con 0x7feaf415b130 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf880b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4072890 con 0x7feadc004800 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feaf4150220 0x7feaf415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feaf4150220 0x7feaf415c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55590/0 (socket says 172.21.15.82:55590) 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 -- 172.21.15.82:0/467596507 learned_addr learned my addr 172.21.15.82:0/467596507 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf1d9b640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf415b130 0x7feaf414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf259c640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc004800 0x7feaf41460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 -- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc004800 msgr2=0x7feaf41460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc004800 0x7feaf41460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 -- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf415b130 msgr2=0x7feaf414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf415b130 0x7feaf414fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 -- 172.21.15.82:0/467596507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf415b680 con 0x7feaf4150220 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.609+0000 7feaf2d9d640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feaf4150220 0x7feaf415c720 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7feaec007d60 tx=0x7feaec010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.610+0000 7feae2ffd640 1 -- 172.21.15.82:0/467596507 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec01c070 con 0x7feaf4150220 2024-03-14T12:18:21.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.610+0000 7feae2ffd640 1 -- 172.21.15.82:0/467596507 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec010d50 con 0x7feaf4150220 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.610+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf415b960 con 0x7feaf4150220 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.610+0000 7feae2ffd640 1 -- 172.21.15.82:0/467596507 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec003bb0 con 0x7feaf4150220 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.610+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf415bf20 con 0x7feaf4150220 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.611+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf404f050 con 0x7feaf4150220 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.611+0000 7feae2ffd640 1 -- 172.21.15.82:0/467596507 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7feaec01f070 con 0x7feaf4150220 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.611+0000 7feae2ffd640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fead406bc00 0x7fead406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.611+0000 7feae2ffd640 1 -- 172.21.15.82:0/467596507 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5587+0+0 (secure 0 0 0) 0x7feaec08a890 con 0x7feaf4150220 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.611+0000 7feaf259c640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fead406bc00 0x7fead406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:21.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.612+0000 7feaf259c640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fead406bc00 0x7fead406e0c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feae4007c30 tx=0x7feae4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:21.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.616+0000 7feae2ffd640 1 -- 172.21.15.82:0/467596507 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7feaec058e30 con 0x7feaf4150220 2024-03-14T12:18:21.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:21.787+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feaf40cf450 con 0x7feaf4150220 2024-03-14T12:18:22.323 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:22.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.489+0000 7feae2ffd640 1 -- 172.21.15.82:0/467596507 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) v1 ==== 169+0+0 (secure 0 0 0) 0x7feaec05ced0 con 0x7feaf4150220 2024-03-14T12:18:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fead406bc00 msgr2=0x7fead406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fead406bc00 0x7fead406e0c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feae4007c30 tx=0x7feae4009040 comp rx=0 tx=0).stop 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feaf4150220 msgr2=0x7feaf415c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feaf4150220 0x7feaf415c720 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7feaec007d60 tx=0x7feaec010530 comp rx=0 tx=0).stop 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 shutdown_connections 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feaf4150220 0x7feaf415c720 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf415b130 0x7feaf414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fead406bc00 0x7fead406e0c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 --2- 172.21.15.82:0/467596507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feadc004800 0x7feaf41460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 >> 172.21.15.82:0/467596507 conn(0x7feaf40b17b0 msgr2=0x7feaf40cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:22.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 shutdown_connections 2024-03-14T12:18:22.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.492+0000 7feaf880b640 1 -- 172.21.15.82:0/467596507 wait complete. 2024-03-14T12:18:22.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-14T12:18:22.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.612+0000 7fc3cc590640 1 Processor -- start 2024-03-14T12:18:22.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.612+0000 7fc3cc590640 1 -- start start 2024-03-14T12:18:22.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3cc590640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40cfe30 0x7fc3c4143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3cc590640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc3c40d0210 0x7fc3c4144520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3cc590640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c40b34e0 con 0x7fc3c415e770 2024-03-14T12:18:22.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3cc590640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c415f4a0 con 0x7fc3c40d0210 2024-03-14T12:18:22.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3cc590640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40b3780 con 0x7fc3c40cfe30 2024-03-14T12:18:22.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3ca32c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40cfe30 0x7fc3c4143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3c9b2b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc3c40d0210 0x7fc3c4144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3ca32c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40cfe30 0x7fc3c4143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.82:59206/0 (socket says 172.21.15.82:59206) 2024-03-14T12:18:22.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3ca32c640 1 -- 172.21.15.82:0/3288486437 learned_addr learned my addr 172.21.15.82:0/3288486437 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:22.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3c9b2b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc3c40d0210 0x7fc3c4144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:42692/0 (socket says 172.21.15.82:42692) 2024-03-14T12:18:22.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3c9b2b640 1 -- 172.21.15.82:0/3288486437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40cfe30 msgr2=0x7fc3c4143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3c9b2b640 1 --2- 172.21.15.82:0/3288486437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40cfe30 0x7fc3c4143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3c9b2b640 1 -- 172.21.15.82:0/3288486437 >> v1:172.21.15.164:6789/0 conn(0x7fc3c415e770 legacy=0x7fc3c415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3c9b2b640 1 -- 172.21.15.82:0/3288486437 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40b3a60 con 0x7fc3c40d0210 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.613+0000 7fc3c932a640 1 -- 172.21.15.82:0/3288486437 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051290043 0 0) 0x7fc3c40b34e0 con 0x7fc3c415e770 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.614+0000 7fc3c9b2b640 1 --2- 172.21.15.82:0/3288486437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40d0210 0x7fc3c4144520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097a0 tx=0x7fc3b402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62355ff6338a4c17 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.614+0000 7fc3c932a640 1 -- 172.21.15.82:0/3288486437 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4035020 con 0x7fc3c40d0210 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.614+0000 7fc3c932a640 1 -- 172.21.15.82:0/3288486437 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b4031830 con 0x7fc3c40d0210 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.614+0000 7fc3c932a640 1 -- 172.21.15.82:0/3288486437 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4031c00 con 0x7fc3c40d0210 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 -- 172.21.15.82:0/3288486437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40d0210 msgr2=0x7fc3c4144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 --2- 172.21.15.82:0/3288486437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40d0210 0x7fc3c4144520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097a0 tx=0x7fc3b402b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 -- 172.21.15.82:0/3288486437 shutdown_connections 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 --2- 172.21.15.82:0/3288486437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40d0210 0x7fc3c4144520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 --2- 172.21.15.82:0/3288486437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40cfe30 0x7fc3c4143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 -- 172.21.15.82:0/3288486437 >> 172.21.15.82:0/3288486437 conn(0x7fc3c406aef0 msgr2=0x7fc3c406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:22.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 -- 172.21.15.82:0/3288486437 shutdown_connections 2024-03-14T12:18:22.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.615+0000 7fc3cc590640 1 -- 172.21.15.82:0/3288486437 wait complete. 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.616+0000 7fc3cc590640 1 Processor -- start 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.616+0000 7fc3cc590640 1 -- start start 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cc590640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40cfe30 0x7fc3c413fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cc590640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40d0210 0x7fc3c4141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cc590640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c415e770 0x7fc3c415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cc590640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40b43e0 con 0x7fc3c415e770 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cc590640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40b4260 con 0x7fc3c40cfe30 2024-03-14T12:18:22.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cc590640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c406d870 con 0x7fc3c40d0210 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cab2d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c415e770 0x7fc3c415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3ca32c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40cfe30 0x7fc3c413fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3ca32c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40cfe30 0x7fc3c413fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:42708/0 (socket says 172.21.15.82:42708) 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cab2d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c415e770 0x7fc3c415b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55604/0 (socket says 172.21.15.82:55604) 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3ca32c640 1 -- 172.21.15.82:0/445234819 learned_addr learned my addr 172.21.15.82:0/445234819 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3c9b2b640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40d0210 0x7fc3c4141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cab2d640 1 -- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40d0210 msgr2=0x7fc3c4141540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cab2d640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40d0210 0x7fc3c4141540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cab2d640 1 -- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40cfe30 msgr2=0x7fc3c413fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cab2d640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40cfe30 0x7fc3c413fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3cab2d640 1 -- 172.21.15.82:0/445234819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40af650 con 0x7fc3c415e770 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.617+0000 7fc3ca32c640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40cfe30 0x7fc3c413fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.618+0000 7fc3cab2d640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c415e770 0x7fc3c415b180 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c750 tx=0x7fc3bc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.618+0000 7fc3baffd640 1 -- 172.21.15.82:0/445234819 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc003b70 con 0x7fc3c415e770 2024-03-14T12:18:22.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.618+0000 7fc3baffd640 1 -- 172.21.15.82:0/445234819 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc003d30 con 0x7fc3c415e770 2024-03-14T12:18:22.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.618+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c40af930 con 0x7fc3c415e770 2024-03-14T12:18:22.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.618+0000 7fc3baffd640 1 -- 172.21.15.82:0/445234819 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0052b0 con 0x7fc3c415e770 2024-03-14T12:18:22.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.618+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c40afef0 con 0x7fc3c415e770 2024-03-14T12:18:22.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.619+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c404f050 con 0x7fc3c415e770 2024-03-14T12:18:22.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.619+0000 7fc3baffd640 1 -- 172.21.15.82:0/445234819 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc3bc005450 con 0x7fc3c415e770 2024-03-14T12:18:22.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.620+0000 7fc3baffd640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3a806bbd0 0x7fc3a806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.620+0000 7fc3ca32c640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3a806bbd0 0x7fc3a806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.620+0000 7fc3baffd640 1 -- 172.21.15.82:0/445234819 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc3bc08a530 con 0x7fc3c415e770 2024-03-14T12:18:22.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.620+0000 7fc3ca32c640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3a806bbd0 0x7fc3a806e090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00094a0 tx=0x7fc3c0009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:22.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.624+0000 7fc3baffd640 1 -- 172.21.15.82:0/445234819 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc3bc058c30 con 0x7fc3c415e770 2024-03-14T12:18:22.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.798+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc3c406d300 con 0x7fc3c415e770 2024-03-14T12:18:22.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.801+0000 7fc3baffd640 1 -- 172.21.15.82:0/445234819 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc3bc05ccd0 con 0x7fc3c415e770 2024-03-14T12:18:22.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.803+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3a806bbd0 msgr2=0x7fc3a806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.803+0000 7fc3cc590640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3a806bbd0 0x7fc3a806e090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00094a0 tx=0x7fc3c0009240 comp rx=0 tx=0).stop 2024-03-14T12:18:22.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.803+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c415e770 msgr2=0x7fc3c415b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c415e770 0x7fc3c415b180 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c750 tx=0x7fc3bc00cc10 comp rx=0 tx=0).stop 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 shutdown_connections 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c415e770 0x7fc3c415b180 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc3a806bbd0 0x7fc3a806e090 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc3c40d0210 0x7fc3c4141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 --2- 172.21.15.82:0/445234819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3c40cfe30 0x7fc3c413fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 >> 172.21.15.82:0/445234819 conn(0x7fc3c406aef0 msgr2=0x7fc3c415c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 shutdown_connections 2024-03-14T12:18:22.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.804+0000 7fc3cc590640 1 -- 172.21.15.82:0/445234819 wait complete. 2024-03-14T12:18:22.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_reconnect' 2024-03-14T12:18:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.917+0000 7f9891220640 1 Processor -- start 2024-03-14T12:18:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.917+0000 7f9891220640 1 -- start start 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f9891220640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f9891220640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c15ebe0 con 0x7f988c143f60 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f9891220640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c15eec0 con 0x7f988c071740 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f9891220640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c15f1a0 con 0x7f988c15e670 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f988b59e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f988c143f60 0x7f988c15b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42798/0 (socket says 172.21.15.82:42798) 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f988b59e640 1 -- 172.21.15.82:0/1052213578 learned_addr learned my addr 172.21.15.82:0/1052213578 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f988a59c640 1 --2- 172.21.15.82:0/1052213578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f9889d9b640 1 -- 172.21.15.82:0/1052213578 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250440182 0 0) 0x7f988c15ebe0 con 0x7f988c143f60 2024-03-14T12:18:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.918+0000 7f9889d9b640 1 -- 172.21.15.82:0/1052213578 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9874003520 con 0x7f988c143f60 2024-03-14T12:18:22.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.919+0000 7f988a59c640 1 -- 172.21.15.82:0/1052213578 >> v1:172.21.15.164:6789/0 conn(0x7f988c071740 legacy=0x7f988c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.919+0000 7f9889d9b640 1 -- 172.21.15.82:0/1052213578 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56296803 0 0) 0x7f988c15eec0 con 0x7f988c071740 2024-03-14T12:18:22.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.919+0000 7f988a59c640 1 -- 172.21.15.82:0/1052213578 >> v1:172.21.15.82:6789/0 conn(0x7f988c143f60 legacy=0x7f988c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.919+0000 7f988a59c640 1 -- 172.21.15.82:0/1052213578 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c15f480 con 0x7f988c15e670 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.919+0000 7f988a59c640 1 --2- 172.21.15.82:0/1052213578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c0b2940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98800097d0 tx=0x7f988002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9301a1b3d43a3c5e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.919+0000 7f9889d9b640 1 -- 172.21.15.82:0/1052213578 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880035020 con 0x7f988c15e670 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9889d9b640 1 -- 172.21.15.82:0/1052213578 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880031890 con 0x7f988c15e670 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9889d9b640 1 -- 172.21.15.82:0/1052213578 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880031c60 con 0x7f988c15e670 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9891220640 1 -- 172.21.15.82:0/1052213578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 msgr2=0x7f988c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9891220640 1 --2- 172.21.15.82:0/1052213578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c0b2940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98800097d0 tx=0x7f988002b930 comp rx=0 tx=0).stop 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9891220640 1 -- 172.21.15.82:0/1052213578 shutdown_connections 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9891220640 1 --2- 172.21.15.82:0/1052213578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c0b2940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9891220640 1 -- 172.21.15.82:0/1052213578 >> 172.21.15.82:0/1052213578 conn(0x7f988c0b1c70 msgr2=0x7f988c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9891220640 1 -- 172.21.15.82:0/1052213578 shutdown_connections 2024-03-14T12:18:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.920+0000 7f9891220640 1 -- 172.21.15.82:0/1052213578 wait complete. 2024-03-14T12:18:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.921+0000 7f9891220640 1 Processor -- start 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f9891220640 1 -- start start 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f9891220640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c071740 0x7f988c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f9891220640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988c143f60 0x7f988c1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f9891220640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c1f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f9891220640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c144670 con 0x7f988c143f60 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f9891220640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c15b080 con 0x7f988c071740 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f9891220640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c1447f0 con 0x7f988c15e670 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f988a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988c143f60 0x7f988c1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f988a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988c143f60 0x7f988c1f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55620/0 (socket says 172.21.15.82:55620) 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f988a59c640 1 -- 172.21.15.82:0/3846060884 learned_addr learned my addr 172.21.15.82:0/3846060884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f988ad9d640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c071740 0x7f988c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.922+0000 7f988b59e640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c1f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988a59c640 1 -- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 msgr2=0x7f988c1f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988a59c640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c1f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988a59c640 1 -- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c071740 msgr2=0x7f988c158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988a59c640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c071740 0x7f988c158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988a59c640 1 -- 172.21.15.82:0/3846060884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c1589d0 con 0x7f988c143f60 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988ad9d640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c071740 0x7f988c158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988b59e640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c1f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f988a59c640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988c143f60 0x7f988c1f5610 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9880000c00 tx=0x7f98800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f98737fe640 1 -- 172.21.15.82:0/3846060884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880035050 con 0x7f988c143f60 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f98737fe640 1 -- 172.21.15.82:0/3846060884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988003bd70 con 0x7f988c143f60 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c15e3f0 con 0x7f988c143f60 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f98737fe640 1 -- 172.21.15.82:0/3846060884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880045630 con 0x7f988c143f60 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.923+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c1f2b40 con 0x7f988c143f60 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.924+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c04f050 con 0x7f988c143f60 2024-03-14T12:18:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.924+0000 7f98737fe640 1 -- 172.21.15.82:0/3846060884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f988004c060 con 0x7f988c143f60 2024-03-14T12:18:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.925+0000 7f98737fe640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f987806bc00 0x7f987806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.925+0000 7f98737fe640 1 -- 172.21.15.82:0/3846060884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f98800aca50 con 0x7f988c143f60 2024-03-14T12:18:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.925+0000 7f988ad9d640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f987806bc00 0x7f987806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.925+0000 7f988ad9d640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f987806bc00 0x7f987806e0c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f987c007fd0 tx=0x7f987c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:22.929+0000 7f98737fe640 1 -- 172.21.15.82:0/3846060884 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f988c1589d0 con 0x7f988c143f60 2024-03-14T12:18:23.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.164+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_reconnect"]} v 0) v1 -- 0x7f988c0d07b0 con 0x7f988c143f60 2024-03-14T12:18:23.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.490+0000 7f98737fe640 1 -- 172.21.15.82:0/3846060884 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_on_client_reconnect"]}]=0 v255) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9880040070 con 0x7f988c143f60 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f987806bc00 msgr2=0x7f987806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f987806bc00 0x7f987806e0c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f987c007fd0 tx=0x7f987c009040 comp rx=0 tx=0).stop 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988c143f60 msgr2=0x7f988c1f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988c143f60 0x7f988c1f5610 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9880000c00 tx=0x7f98800046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 shutdown_connections 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f987806bc00 0x7f987806e0c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f988c15e670 0x7f988c1f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f988c143f60 0x7f988c1f5610 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 --2- 172.21.15.82:0/3846060884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c071740 0x7f988c158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 >> 172.21.15.82:0/3846060884 conn(0x7f988c0b1c70 msgr2=0x7f988c0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 shutdown_connections 2024-03-14T12:18:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:23.492+0000 7f9891220640 1 -- 172.21.15.82:0/3846060884 wait complete. 2024-03-14T12:18:23.504 INFO:tasks.cephfs_test_runner:That the per-client labelled metrics are generated during client reconnect ... ok 2024-03-14T12:18:23.504 INFO:tasks.cephfs_test_runner: 2024-03-14T12:18:23.504 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-14T12:18:23.504 INFO:tasks.cephfs_test_runner:Ran 4 tests in 244.063s 2024-03-14T12:18:23.504 INFO:tasks.cephfs_test_runner: 2024-03-14T12:18:23.504 INFO:tasks.cephfs_test_runner:OK 2024-03-14T12:18:23.505 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-14T12:18:23.514 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-14T12:18:23.523 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-14T12:18:23.523 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:18:23.523 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:23.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:18:23.540 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-14T12:18:23.540 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:23.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=919 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-14T12:18:23.595 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:18:23.595 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:23.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:18:23.614 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-14T12:18:23.614 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T12:18:23.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-14T12:18:23.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:23.672 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:18:23.686 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-14T12:18:23.687 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T12:18:23.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:23.687 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-14T12:18:23.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:23.755 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T12:18:23.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:23.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-14T12:18:23.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:23.915 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-14T12:18:23.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:23.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-14T12:18:24.000 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-03-14T12:18:24.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:18:24.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:24.002 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:18:24.020 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-14T12:18:24.020 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T12:18:24.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:24.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-14T12:18:24.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:24.092 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T12:18:24.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:24.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-03-14T12:18:24.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:24.265 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-14T12:18:24.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T12:18:24.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-03-14T12:18:24.353 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-03-14T12:18:24.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:18:24.355 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-14T12:18:24.366 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-14T12:18:24.367 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:18:24.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.467+0000 7ff5ebdf1640 1 Processor -- start 2024-03-14T12:18:24.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.468+0000 7ff5ebdf1640 1 -- start start 2024-03-14T12:18:24.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.468+0000 7ff5ebdf1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5e4151230 0x7ff5e40cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.468+0000 7ff5ebdf1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e40b3da0 con 0x7ff5e415dde0 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5ebdf1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e40b4080 con 0x7ff5e40b4b70 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5ebdf1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e4070d40 con 0x7ff5e4151230 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5e9b8d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5e4151230 0x7ff5e40cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5ea38e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff5e415dde0 0x7ff5e406cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42814/0 (socket says 172.21.15.82:42814) 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5ea38e640 1 -- 172.21.15.82:0/3895008104 learned_addr learned my addr 172.21.15.82:0/3895008104 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5e9b8d640 1 -- 172.21.15.82:0/3895008104 >> v1:172.21.15.164:6789/0 conn(0x7ff5e40b4b70 legacy=0x7ff5e4070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5e9b8d640 1 -- 172.21.15.82:0/3895008104 >> v1:172.21.15.82:6789/0 conn(0x7ff5e415dde0 legacy=0x7ff5e406cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5e8b8b640 1 -- 172.21.15.82:0/3895008104 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531763332 0 0) 0x7ff5e40b3da0 con 0x7ff5e415dde0 2024-03-14T12:18:24.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.469+0000 7ff5e9b8d640 1 -- 172.21.15.82:0/3895008104 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40b4360 con 0x7ff5e4151230 2024-03-14T12:18:24.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5e9b8d640 1 --2- 172.21.15.82:0/3895008104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e4151230 0x7ff5e40cf0a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097a0 tx=0x7ff5d802ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b104a6348c67ab71 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:24.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5e8b8b640 1 -- 172.21.15.82:0/3895008104 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802e830 con 0x7ff5e4151230 2024-03-14T12:18:24.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5e8b8b640 1 -- 172.21.15.82:0/3895008104 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d802e9d0 con 0x7ff5e4151230 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5e8b8b640 1 -- 172.21.15.82:0/3895008104 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802eda0 con 0x7ff5e4151230 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3895008104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e4151230 msgr2=0x7ff5e40cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3895008104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e4151230 0x7ff5e40cf0a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097a0 tx=0x7ff5d802ca70 comp rx=0 tx=0).stop 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3895008104 shutdown_connections 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3895008104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e4151230 0x7ff5e40cf0a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3895008104 >> 172.21.15.82:0/3895008104 conn(0x7ff5e40b1c70 msgr2=0x7ff5e40cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.470+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3895008104 shutdown_connections 2024-03-14T12:18:24.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.471+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3895008104 wait complete. 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 Processor -- start 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 -- start start 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e40b4b70 0x7ff5e40c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5e4151230 0x7ff5e4148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5e415dde0 0x7ff5e415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e406c690 con 0x7ff5e40b4b70 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e406c3b0 con 0x7ff5e415dde0 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5ebdf1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40cf5e0 con 0x7ff5e4151230 2024-03-14T12:18:24.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5e9b8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e40b4b70 0x7ff5e40c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5e9b8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e40b4b70 0x7ff5e40c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55626/0 (socket says 172.21.15.82:55626) 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.472+0000 7ff5e9b8d640 1 -- 172.21.15.82:0/3954706160 learned_addr learned my addr 172.21.15.82:0/3954706160 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5ea38e640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5e415dde0 0x7ff5e415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e938c640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5e4151230 0x7ff5e4148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e9b8d640 1 -- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5e4151230 msgr2=0x7ff5e4148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e9b8d640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5e4151230 0x7ff5e4148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e9b8d640 1 -- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5e415dde0 msgr2=0x7ff5e415e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e9b8d640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5e415dde0 0x7ff5e415e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e9b8d640 1 -- 172.21.15.82:0/3954706160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40ca7d0 con 0x7ff5e40b4b70 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e938c640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5e4151230 0x7ff5e4148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5e9b8d640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e40b4b70 0x7ff5e40c9f50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80098c0 tx=0x7ff5d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5d67fc640 1 -- 172.21.15.82:0/3954706160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802e7d0 con 0x7ff5e40b4b70 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5d67fc640 1 -- 172.21.15.82:0/3954706160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d8038cc0 con 0x7ff5e40b4b70 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e40caab0 con 0x7ff5e40b4b70 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5d67fc640 1 -- 172.21.15.82:0/3954706160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802f640 con 0x7ff5e40b4b70 2024-03-14T12:18:24.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.473+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e40cb350 con 0x7ff5e40b4b70 2024-03-14T12:18:24.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.474+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e40cac80 con 0x7ff5e40b4b70 2024-03-14T12:18:24.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.474+0000 7ff5d67fc640 1 -- 172.21.15.82:0/3954706160 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff5d804a400 con 0x7ff5e40b4b70 2024-03-14T12:18:24.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.475+0000 7ff5d67fc640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5c806bbd0 0x7ff5c806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.475+0000 7ff5d67fc640 1 -- 172.21.15.82:0/3954706160 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff5d80ac630 con 0x7ff5e40b4b70 2024-03-14T12:18:24.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.475+0000 7ff5e938c640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5c806bbd0 0x7ff5c806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.475+0000 7ff5e938c640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5c806bbd0 0x7ff5c806e090 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40cf040 tx=0x7ff5e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:24.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.479+0000 7ff5d67fc640 1 -- 172.21.15.82:0/3954706160 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff5d804f060 con 0x7ff5e40b4b70 2024-03-14T12:18:24.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.649+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5e404f050 con 0x7ff5c806bbd0 2024-03-14T12:18:24.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.650+0000 7ff5d67fc640 1 -- 172.21.15.82:0/3954706160 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13481 (secure 0 0 0) 0x7ff5e404f050 con 0x7ff5c806bbd0 2024-03-14T12:18:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:24.651 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:18:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5c806bbd0 msgr2=0x7ff5c806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5c806bbd0 0x7ff5c806e090 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40cf040 tx=0x7ff5e0009040 comp rx=0 tx=0).stop 2024-03-14T12:18:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e40b4b70 msgr2=0x7ff5e40c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e40b4b70 0x7ff5e40c9f50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80098c0 tx=0x7ff5d80046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 shutdown_connections 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5e415dde0 0x7ff5e415e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff5c806bbd0 0x7ff5c806e090 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5e4151230 0x7ff5e4148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 --2- 172.21.15.82:0/3954706160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e40b4b70 0x7ff5e40c9f50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 >> 172.21.15.82:0/3954706160 conn(0x7ff5e40b1c70 msgr2=0x7ff5e406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 shutdown_connections 2024-03-14T12:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.652+0000 7ff5ebdf1640 1 -- 172.21.15.82:0/3954706160 wait complete. 2024-03-14T12:18:24.663 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-03-14T12:18:23.553891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266172,"kb_used_data":8396,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708548,"statfs":{"total":773094113280,"available":772821553152,"internally_reserved":0,"allocated":8597504,"data_stored":4306313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":263835187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4447,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":-79,"num_read_kb":-84,"num_write":-53,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000950"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":274,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-14T12:18:23.508119+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:18:23.508119+0000","last_peered":"2024-03-14T12:18:23.508119+0000","last_clean":"2024-03-14T12:18:23.508119+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:18:23.508119+0000","last_undegraded":"2024-03-14T12:18:23.508119+0000","last_fullsized":"2024-03-14T12:18:23.508119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":978408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":388040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36236,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335604,"statfs":{"total":96636764160,"available":96599658496,"internally_reserved":0,"allocated":917504,"data_stored":388040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":921600,"data_stored":386213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":984179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:18:24.663 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:18:24.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.763+0000 7faf62b7e640 1 Processor -- start 2024-03-14T12:18:24.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.763+0000 7faf62b7e640 1 -- start start 2024-03-14T12:18:24.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf62b7e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf5c15e670 0x7faf5c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf62b7e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf5c143f60 0x7faf5c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf62b7e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c15ebe0 con 0x7faf5c071740 2024-03-14T12:18:24.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf62b7e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c15eec0 con 0x7faf5c143f60 2024-03-14T12:18:24.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf62b7e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c15f1a0 con 0x7faf5c15e670 2024-03-14T12:18:24.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf6111b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf5c143f60 0x7faf5c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf6091a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faf5c071740 0x7faf5c071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42826/0 (socket says 172.21.15.82:42826) 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf6091a640 1 -- 172.21.15.82:0/1161269716 learned_addr learned my addr 172.21.15.82:0/1161269716 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf5bfff640 1 --2- 172.21.15.82:0/1161269716 >> v2:172.21.15.164:3300/0 conn(0x7faf5c15e670 0x7faf5c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf6111b640 1 -- 172.21.15.82:0/1161269716 >> v2:172.21.15.164:3300/0 conn(0x7faf5c15e670 msgr2=0x7faf5c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf6111b640 1 --2- 172.21.15.82:0/1161269716 >> v2:172.21.15.164:3300/0 conn(0x7faf5c15e670 0x7faf5c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf6111b640 1 -- 172.21.15.82:0/1161269716 >> v1:172.21.15.82:6789/0 conn(0x7faf5c071740 legacy=0x7faf5c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf5b7fe640 1 -- 172.21.15.82:0/1161269716 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199546495 0 0) 0x7faf5c15ebe0 con 0x7faf5c071740 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.764+0000 7faf6111b640 1 -- 172.21.15.82:0/1161269716 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c15f480 con 0x7faf5c143f60 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf6111b640 1 --2- 172.21.15.82:0/1161269716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c143f60 0x7faf5c15b610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7faf54009d90 tx=0x7faf5402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37bdd93a4333c05 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf5b7fe640 1 -- 172.21.15.82:0/1161269716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5403b070 con 0x7faf5c143f60 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf5b7fe640 1 -- 172.21.15.82:0/1161269716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54036440 con 0x7faf5c143f60 2024-03-14T12:18:24.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf5b7fe640 1 -- 172.21.15.82:0/1161269716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54036810 con 0x7faf5c143f60 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf62b7e640 1 -- 172.21.15.82:0/1161269716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c143f60 msgr2=0x7faf5c15b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf62b7e640 1 --2- 172.21.15.82:0/1161269716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c143f60 0x7faf5c15b610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7faf54009d90 tx=0x7faf5402e0c0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf62b7e640 1 -- 172.21.15.82:0/1161269716 shutdown_connections 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf62b7e640 1 --2- 172.21.15.82:0/1161269716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c143f60 0x7faf5c15b610 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf62b7e640 1 --2- 172.21.15.82:0/1161269716 >> v2:172.21.15.164:3300/0 conn(0x7faf5c15e670 0x7faf5c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.765+0000 7faf62b7e640 1 -- 172.21.15.82:0/1161269716 >> 172.21.15.82:0/1161269716 conn(0x7faf5c0b1c70 msgr2=0x7faf5c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.766+0000 7faf62b7e640 1 -- 172.21.15.82:0/1161269716 shutdown_connections 2024-03-14T12:18:24.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.766+0000 7faf62b7e640 1 -- 172.21.15.82:0/1161269716 wait complete. 2024-03-14T12:18:24.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.767+0000 7faf62b7e640 1 Processor -- start 2024-03-14T12:18:24.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.767+0000 7faf62b7e640 1 -- start start 2024-03-14T12:18:24.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.767+0000 7faf62b7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c071740 0x7faf5c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf62b7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c143f60 0x7faf5c158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf62b7e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5c15e670 0x7faf5c139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf62b7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c144670 con 0x7faf5c071740 2024-03-14T12:18:24.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf62b7e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c15b080 con 0x7faf5c143f60 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf62b7e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1447f0 con 0x7faf5c15e670 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c071740 0x7faf5c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c071740 0x7faf5c15e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55650/0 (socket says 172.21.15.82:55650) 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 -- 172.21.15.82:0/2486591537 learned_addr learned my addr 172.21.15.82:0/2486591537 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf5bfff640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c143f60 0x7faf5c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 -- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5c15e670 msgr2=0x7faf5c139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6111b640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5c15e670 0x7faf5c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5c15e670 0x7faf5c139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 -- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c143f60 msgr2=0x7faf5c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c143f60 0x7faf5c158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 -- 172.21.15.82:0/2486591537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c158860 con 0x7faf5c071740 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf5bfff640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c143f60 0x7faf5c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:24.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf6091a640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c071740 0x7faf5c15e0f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c430 tx=0x7faf5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf597fa640 1 -- 172.21.15.82:0/2486591537 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500074e0 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf597fa640 1 -- 172.21.15.82:0/2486591537 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf500047e0 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c15d050 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf597fa640 1 -- 172.21.15.82:0/2486591537 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50004bd0 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.768+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c15d610 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.769+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c04f050 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.770+0000 7faf597fa640 1 -- 172.21.15.82:0/2486591537 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7faf50011070 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.770+0000 7faf597fa640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bc00 0x7faf4006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.770+0000 7faf597fa640 1 -- 172.21.15.82:0/2486591537 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7faf5008a400 con 0x7faf5c071740 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.770+0000 7faf5bfff640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bc00 0x7faf4006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:24.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.770+0000 7faf5bfff640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bc00 0x7faf4006e0c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faf5c13eeb0 tx=0x7faf48008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.774+0000 7faf597fa640 1 -- 172.21.15.82:0/2486591537 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7faf5000e280 con 0x7faf5c071740 2024-03-14T12:18:24.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.944+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf5c0d07b0 con 0x7faf4006bc00 2024-03-14T12:18:24.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.945+0000 7faf597fa640 1 -- 172.21.15.82:0/2486591537 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13481 (secure 0 0 0) 0x7faf5c0d07b0 con 0x7faf4006bc00 2024-03-14T12:18:24.946 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:24.946 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bc00 msgr2=0x7faf4006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bc00 0x7faf4006e0c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faf5c13eeb0 tx=0x7faf48008040 comp rx=0 tx=0).stop 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c071740 msgr2=0x7faf5c15e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c071740 0x7faf5c15e0f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c430 tx=0x7faf5000c8f0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 shutdown_connections 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7faf4006bc00 0x7faf4006e0c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf5c15e670 0x7faf5c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf5c143f60 0x7faf5c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 --2- 172.21.15.82:0/2486591537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf5c071740 0x7faf5c15e0f0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 >> 172.21.15.82:0/2486591537 conn(0x7faf5c0b1c70 msgr2=0x7faf5c0ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:24.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 shutdown_connections 2024-03-14T12:18:24.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:24.947+0000 7faf62b7e640 1 -- 172.21.15.82:0/2486591537 wait complete. 2024-03-14T12:18:24.958 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-03-14T12:18:23.553891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266172,"kb_used_data":8396,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708548,"statfs":{"total":773094113280,"available":772821553152,"internally_reserved":0,"allocated":8597504,"data_stored":4306313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":263835187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4447,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":-79,"num_read_kb":-84,"num_write":-53,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000950"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":274,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-14T12:18:23.508119+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:18:23.508119+0000","last_peered":"2024-03-14T12:18:23.508119+0000","last_clean":"2024-03-14T12:18:23.508119+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:18:23.508119+0000","last_undegraded":"2024-03-14T12:18:23.508119+0000","last_fullsized":"2024-03-14T12:18:23.508119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":978408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":388040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36236,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335604,"statfs":{"total":96636764160,"available":96599658496,"internally_reserved":0,"allocated":917504,"data_stored":388040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":921600,"data_stored":386213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":984179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:18:24.962 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-14T12:18:24.962 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:18:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.061+0000 7fc52c49a640 1 Processor -- start 2024-03-14T12:18:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.061+0000 7fc52c49a640 1 -- start start 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.062+0000 7fc52c49a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5240d0760 0x7fc5240d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.062+0000 7fc52c49a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240b7720 0x7fc5240b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.062+0000 7fc52c49a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5240d1080 con 0x7fc5240cf370 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.062+0000 7fc52c49a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240cf8c0 con 0x7fc5240d0760 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.062+0000 7fc52c49a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240cfba0 con 0x7fc5240b7720 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52a236640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5240d0760 0x7fc5240d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52aa37640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240b7720 0x7fc5240b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52a236640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5240d0760 0x7fc5240d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:42760/0 (socket says 172.21.15.82:42760) 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52a236640 1 -- 172.21.15.82:0/1374932591 learned_addr learned my addr 172.21.15.82:0/1374932591 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc529a35640 1 --1- 172.21.15.82:0/1374932591 >> v1:172.21.15.164:6789/0 conn(0x7fc5240cf370 0x7fc5240d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:58576/0 (socket says 172.21.15.82:58576) 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52a236640 1 -- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240b7720 msgr2=0x7fc5240b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52a236640 1 --2- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240b7720 0x7fc5240b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52a236640 1 -- 172.21.15.82:0/1374932591 >> v1:172.21.15.164:6789/0 conn(0x7fc5240cf370 legacy=0x7fc5240d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52a236640 1 -- 172.21.15.82:0/1374932591 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240b6c40 con 0x7fc5240d0760 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc529234640 1 -- 172.21.15.82:0/1374932591 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859885871 0 0) 0x7fc5240d1080 con 0x7fc5240cf370 2024-03-14T12:18:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.063+0000 7fc52aa37640 1 --2- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240b7720 0x7fc5240b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.064+0000 7fc52a236640 1 --2- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240d0760 0x7fc5240d0b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc520009e50 tx=0x7fc52002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92a44fa478429503 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.064+0000 7fc529234640 1 -- 172.21.15.82:0/1374932591 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52002cb90 con 0x7fc5240d0760 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.064+0000 7fc529234640 1 -- 172.21.15.82:0/1374932591 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52002cd30 con 0x7fc5240d0760 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.064+0000 7fc529234640 1 -- 172.21.15.82:0/1374932591 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520034610 con 0x7fc5240d0760 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.064+0000 7fc52c49a640 1 -- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240d0760 msgr2=0x7fc5240d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.064+0000 7fc52c49a640 1 --2- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240d0760 0x7fc5240d0b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc520009e50 tx=0x7fc52002bff0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.065+0000 7fc52c49a640 1 -- 172.21.15.82:0/1374932591 shutdown_connections 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.065+0000 7fc52c49a640 1 --2- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240b7720 0x7fc5240b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.065+0000 7fc52c49a640 1 --2- 172.21.15.82:0/1374932591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240d0760 0x7fc5240d0b40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.065+0000 7fc52c49a640 1 -- 172.21.15.82:0/1374932591 >> 172.21.15.82:0/1374932591 conn(0x7fc52409d030 msgr2=0x7fc5240d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.065+0000 7fc52c49a640 1 -- 172.21.15.82:0/1374932591 shutdown_connections 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.065+0000 7fc52c49a640 1 -- 172.21.15.82:0/1374932591 wait complete. 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 Processor -- start 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 -- start start 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240b7720 0x7fc52415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240cf370 0x7fc5240cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240d0760 0x7fc524147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5241617a0 con 0x7fc5240b7720 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240b7200 con 0x7fc5240cf370 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.066+0000 7fc52c49a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc524161920 con 0x7fc5240d0760 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240b7720 0x7fc52415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240b7720 0x7fc52415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55660/0 (socket says 172.21.15.82:55660) 2024-03-14T12:18:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 -- 172.21.15.82:0/182379377 learned_addr learned my addr 172.21.15.82:0/182379377 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc529a35640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240cf370 0x7fc5240cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 -- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240d0760 msgr2=0x7fc524147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52aa37640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240d0760 0x7fc524147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240d0760 0x7fc524147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 -- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240cf370 msgr2=0x7fc5240cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240cf370 0x7fc5240cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 -- 172.21.15.82:0/182379377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524160450 con 0x7fc5240b7720 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52a236640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240b7720 0x7fc52415ea60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc520036040 tx=0x7fc520004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc51affd640 1 -- 172.21.15.82:0/182379377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520043030 con 0x7fc5240b7720 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc51affd640 1 -- 172.21.15.82:0/182379377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52003e450 con 0x7fc5240b7720 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52415f3b0 con 0x7fc5240b7720 2024-03-14T12:18:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc51affd640 1 -- 172.21.15.82:0/182379377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520009510 con 0x7fc5240b7720 2024-03-14T12:18:25.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.067+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52415f970 con 0x7fc5240b7720 2024-03-14T12:18:25.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.068+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52404f050 con 0x7fc5240b7720 2024-03-14T12:18:25.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.069+0000 7fc51affd640 1 -- 172.21.15.82:0/182379377 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc520046070 con 0x7fc5240b7720 2024-03-14T12:18:25.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.069+0000 7fc51affd640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50806bbd0 0x7fc50806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.069+0000 7fc51affd640 1 -- 172.21.15.82:0/182379377 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc5200acab0 con 0x7fc5240b7720 2024-03-14T12:18:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.069+0000 7fc529a35640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50806bbd0 0x7fc50806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.070+0000 7fc529a35640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50806bbd0 0x7fc50806e090 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc514007fd0 tx=0x7fc514009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.073+0000 7fc51affd640 1 -- 172.21.15.82:0/182379377 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc52004d070 con 0x7fc5240b7720 2024-03-14T12:18:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.243+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5240d0540 con 0x7fc50806bbd0 2024-03-14T12:18:25.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.244+0000 7fc51affd640 1 -- 172.21.15.82:0/182379377 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13481 (secure 0 0 0) 0x7fc5240d0540 con 0x7fc50806bbd0 2024-03-14T12:18:25.244 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:25.245 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50806bbd0 msgr2=0x7fc50806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50806bbd0 0x7fc50806e090 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc514007fd0 tx=0x7fc514009040 comp rx=0 tx=0).stop 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240b7720 msgr2=0x7fc52415ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240b7720 0x7fc52415ea60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc520036040 tx=0x7fc520004a40 comp rx=0 tx=0).stop 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 shutdown_connections 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc50806bbd0 0x7fc50806e090 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5240d0760 0x7fc524147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5240cf370 0x7fc5240cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 --2- 172.21.15.82:0/182379377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240b7720 0x7fc52415ea60 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 >> 172.21.15.82:0/182379377 conn(0x7fc52409d030 msgr2=0x7fc5240cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 shutdown_connections 2024-03-14T12:18:25.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.246+0000 7fc52c49a640 1 -- 172.21.15.82:0/182379377 wait complete. 2024-03-14T12:18:25.257 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-03-14T12:18:23.553891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266172,"kb_used_data":8396,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708548,"statfs":{"total":773094113280,"available":772821553152,"internally_reserved":0,"allocated":8597504,"data_stored":4306313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":263835187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4447,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":-79,"num_read_kb":-84,"num_write":-53,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000950"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":274,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-14T12:18:23.508119+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:18:23.508119+0000","last_peered":"2024-03-14T12:18:23.508119+0000","last_clean":"2024-03-14T12:18:23.508119+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:18:23.508119+0000","last_undegraded":"2024-03-14T12:18:23.508119+0000","last_fullsized":"2024-03-14T12:18:23.508119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":978408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":388040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36236,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335604,"statfs":{"total":96636764160,"available":96599658496,"internally_reserved":0,"allocated":917504,"data_stored":388040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":921600,"data_stored":386213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":984179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:18:25.257 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T12:18:25.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.356+0000 7fc909032640 1 Processor -- start 2024-03-14T12:18:25.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.356+0000 7fc909032640 1 -- start start 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc909032640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc9040d0760 0x7fc9040d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc909032640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040cf370 0x7fc9040d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc909032640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9040d1080 con 0x7fc9040d1da0 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc909032640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040cf8c0 con 0x7fc9040d0760 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc909032640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040cfba0 con 0x7fc9040cf370 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc90359e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc9040d1da0 0x7fc9040d2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42828/0 (socket says 172.21.15.82:42828) 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc90359e640 1 -- 172.21.15.82:0/2293585471 learned_addr learned my addr 172.21.15.82:0/2293585471 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc902d9d640 1 --2- 172.21.15.82:0/2293585471 >> v2:172.21.15.164:3300/0 conn(0x7fc9040d0760 0x7fc9040d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.357+0000 7fc90259c640 1 --2- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040cf370 0x7fc9040d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.358+0000 7fc901d9b640 1 -- 172.21.15.82:0/2293585471 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922597361 0 0) 0x7fc9040d1080 con 0x7fc9040d1da0 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.358+0000 7fc901d9b640 1 -- 172.21.15.82:0/2293585471 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ec003520 con 0x7fc9040d1da0 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.358+0000 7fc901d9b640 1 -- 172.21.15.82:0/2293585471 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501371263 0 0) 0x7fc8ec003520 con 0x7fc9040d1da0 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.358+0000 7fc902d9d640 1 -- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040cf370 msgr2=0x7fc9040d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.358+0000 7fc902d9d640 1 --2- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040cf370 0x7fc9040d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.358+0000 7fc902d9d640 1 -- 172.21.15.82:0/2293585471 >> v1:172.21.15.82:6789/0 conn(0x7fc9040d1da0 legacy=0x7fc9040d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.358+0000 7fc902d9d640 1 -- 172.21.15.82:0/2293585471 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040b6c40 con 0x7fc9040d0760 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.359+0000 7fc902d9d640 1 --2- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040d0760 0x7fc9040d0b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009d90 tx=0x7fc8f402be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f66a9c1bc8b742fe server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.359+0000 7fc901d9b640 1 -- 172.21.15.82:0/2293585471 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4039070 con 0x7fc9040d0760 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.359+0000 7fc901d9b640 1 -- 172.21.15.82:0/2293585471 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f402c930 con 0x7fc9040d0760 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.359+0000 7fc901d9b640 1 -- 172.21.15.82:0/2293585471 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc001670 con 0x7fc9040d0760 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.359+0000 7fc909032640 1 -- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040d0760 msgr2=0x7fc9040d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.359+0000 7fc909032640 1 --2- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040d0760 0x7fc9040d0b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009d90 tx=0x7fc8f402be00 comp rx=0 tx=0).stop 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.360+0000 7fc909032640 1 -- 172.21.15.82:0/2293585471 shutdown_connections 2024-03-14T12:18:25.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.360+0000 7fc909032640 1 --2- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040cf370 0x7fc9040d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.360+0000 7fc909032640 1 --2- 172.21.15.82:0/2293585471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040d0760 0x7fc9040d0b40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.360+0000 7fc909032640 1 -- 172.21.15.82:0/2293585471 >> 172.21.15.82:0/2293585471 conn(0x7fc90409d030 msgr2=0x7fc9040d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:25.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.360+0000 7fc909032640 1 -- 172.21.15.82:0/2293585471 shutdown_connections 2024-03-14T12:18:25.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.360+0000 7fc909032640 1 -- 172.21.15.82:0/2293585471 wait complete. 2024-03-14T12:18:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.361+0000 7fc909032640 1 Processor -- start 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.361+0000 7fc909032640 1 -- start start 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc909032640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040cf370 0x7fc9041f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc909032640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040d0760 0x7fc9040c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc909032640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9040d1da0 0x7fc90415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc909032640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040d4ba0 con 0x7fc9040d1da0 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc909032640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040b7200 con 0x7fc9040cf370 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc909032640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041617a0 con 0x7fc9040d0760 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9040d1da0 0x7fc90415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9040d1da0 0x7fc90415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55668/0 (socket says 172.21.15.82:55668) 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 -- 172.21.15.82:0/2760786509 learned_addr learned my addr 172.21.15.82:0/2760786509 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90259c640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040d0760 0x7fc9040c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc902d9d640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040cf370 0x7fc9041f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 -- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040d0760 msgr2=0x7fc9040c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040d0760 0x7fc9040c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 -- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040cf370 msgr2=0x7fc9041f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040cf370 0x7fc9041f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 -- 172.21.15.82:0/2760786509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040c8340 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.362+0000 7fc90359e640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9040d1da0 0x7fc90415c7d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c580 tx=0x7fc8fc00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.363+0000 7fc8eb7fe640 1 -- 172.21.15.82:0/2760786509 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc005ce0 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.363+0000 7fc8eb7fe640 1 -- 172.21.15.82:0/2760786509 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc0070f0 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.363+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040c8620 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.363+0000 7fc8eb7fe640 1 -- 172.21.15.82:0/2760786509 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0074e0 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.363+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040c9080 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.364+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90404f050 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.364+0000 7fc8eb7fe640 1 -- 172.21.15.82:0/2760786509 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc8fc01b400 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.364+0000 7fc8eb7fe640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8f006bbd0 0x7fc8f006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.365+0000 7fc8eb7fe640 1 -- 172.21.15.82:0/2760786509 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc8fc08a490 con 0x7fc9040d1da0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.365+0000 7fc902d9d640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8f006bbd0 0x7fc8f006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.365+0000 7fc902d9d640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8f006bbd0 0x7fc8f006e090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4036000 tx=0x7fc8f4030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:25.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.369+0000 7fc8eb7fe640 1 -- 172.21.15.82:0/2760786509 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc8fc058c10 con 0x7fc9040d1da0 2024-03-14T12:18:25.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.542+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9040d0540 con 0x7fc9040d1da0 2024-03-14T12:18:25.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.543+0000 7fc8eb7fe640 1 -- 172.21.15.82:0/2760786509 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+13264 (secure 0 0 0) 0x7fc8fc05ccb0 con 0x7fc9040d1da0 2024-03-14T12:18:25.543 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":76,"fsid":"30427ff5-7d8c-458f-ac8c-6cdc36ecf1a2","created":"2024-03-14T12:13:27.374155+0000","modified":"2024-03-14T12:18:22.483852+0000","last_up_change":"2024-03-14T12:13:38.426422+0000","last_in_change":"2024-03-14T12:13:29.566529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T12:13:39.541273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbe5962a-eef4-46b1-8a89-9d92797b47f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6801","nonce":3139126855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6803","nonce":3139126855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6807","nonce":3139126855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3139126855},{"type":"v1","addr":"172.21.15.82:6805","nonce":3139126855}]},"public_addr":"172.21.15.82:6801/3139126855","cluster_addr":"172.21.15.82:6803/3139126855","heartbeat_back_addr":"172.21.15.82:6807/3139126855","heartbeat_front_addr":"172.21.15.82:6805/3139126855","state":["exists","up"]},{"osd":1,"uuid":"8856df27-763b-47d7-853b-29410857ca0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6817","nonce":2003973578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6819","nonce":2003973578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6823","nonce":2003973578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2003973578},{"type":"v1","addr":"172.21.15.82:6821","nonce":2003973578}]},"public_addr":"172.21.15.82:6817/2003973578","cluster_addr":"172.21.15.82:6819/2003973578","heartbeat_back_addr":"172.21.15.82:6823/2003973578","heartbeat_front_addr":"172.21.15.82:6821/2003973578","state":["exists","up"]},{"osd":2,"uuid":"7ac8e577-9a2e-4ace-aa16-7ab7ac5cce57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6809","nonce":2455719014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6811","nonce":2455719014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6815","nonce":2455719014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2455719014},{"type":"v1","addr":"172.21.15.82:6813","nonce":2455719014}]},"public_addr":"172.21.15.82:6809/2455719014","cluster_addr":"172.21.15.82:6811/2455719014","heartbeat_back_addr":"172.21.15.82:6815/2455719014","heartbeat_front_addr":"172.21.15.82:6813/2455719014","state":["exists","up"]},{"osd":3,"uuid":"e8b7f18c-6a7f-48b9-a7e4-481dc021cec8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6825","nonce":994456991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6827","nonce":994456991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6831","nonce":994456991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":994456991},{"type":"v1","addr":"172.21.15.82:6829","nonce":994456991}]},"public_addr":"172.21.15.82:6825/994456991","cluster_addr":"172.21.15.82:6827/994456991","heartbeat_back_addr":"172.21.15.82:6831/994456991","heartbeat_front_addr":"172.21.15.82:6829/994456991","state":["exists","up"]},{"osd":4,"uuid":"00d8cc9a-7044-475d-a609-45a893e543c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6801","nonce":2779428821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6803","nonce":2779428821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6807","nonce":2779428821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2779428821},{"type":"v1","addr":"172.21.15.164:6805","nonce":2779428821}]},"public_addr":"172.21.15.164:6801/2779428821","cluster_addr":"172.21.15.164:6803/2779428821","heartbeat_back_addr":"172.21.15.164:6807/2779428821","heartbeat_front_addr":"172.21.15.164:6805/2779428821","state":["exists","up"]},{"osd":5,"uuid":"f28ae07f-38af-4f2a-81ad-69db2b9f57a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6819","nonce":2718563770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6823","nonce":2718563770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6829","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6831","nonce":2718563770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6825","nonce":2718563770},{"type":"v1","addr":"172.21.15.164:6827","nonce":2718563770}]},"public_addr":"172.21.15.164:6819/2718563770","cluster_addr":"172.21.15.164:6823/2718563770","heartbeat_back_addr":"172.21.15.164:6831/2718563770","heartbeat_front_addr":"172.21.15.164:6827/2718563770","state":["exists","up"]},{"osd":6,"uuid":"d4030c0e-7f55-42bf-8f3c-a25e05fd6218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6809","nonce":3403808739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6811","nonce":3403808739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6815","nonce":3403808739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3403808739},{"type":"v1","addr":"172.21.15.164:6813","nonce":3403808739}]},"public_addr":"172.21.15.164:6809/3403808739","cluster_addr":"172.21.15.164:6811/3403808739","heartbeat_back_addr":"172.21.15.164:6815/3403808739","heartbeat_front_addr":"172.21.15.164:6813/3403808739","state":["exists","up"]},{"osd":7,"uuid":"ad7ab054-0c02-4739-9e9d-8c711dd9fc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6818","nonce":369064365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6822","nonce":369064365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6830","nonce":369064365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":369064365},{"type":"v1","addr":"172.21.15.164:6826","nonce":369064365}]},"public_addr":"172.21.15.164:6818/369064365","cluster_addr":"172.21.15.164:6822/369064365","heartbeat_back_addr":"172.21.15.164:6830/369064365","heartbeat_front_addr":"172.21.15.164:6826/369064365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.707701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750332+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.760002+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.792434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.641489+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.702780+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.681416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T12:13:34.750228+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/2268648862":"2024-03-15T12:18:19.476224+0000","172.21.15.82:6836/2268648862":"2024-03-15T12:18:19.476224+0000","172.21.15.164:6834/4033072140":"2024-03-15T12:17:15.041576+0000","192.168.0.1:0/895829753":"2024-03-14T13:18:05.110010+0000","172.21.15.164:0/1140720987":"2024-03-14T13:18:05.110143+0000","172.21.15.164:6832/4033072140":"2024-03-15T12:17:15.041576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T12:18:25.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8f006bbd0 msgr2=0x7fc8f006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8f006bbd0 0x7fc8f006e090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4036000 tx=0x7fc8f4030000 comp rx=0 tx=0).stop 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9040d1da0 msgr2=0x7fc90415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9040d1da0 0x7fc90415c7d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c580 tx=0x7fc8fc00ca40 comp rx=0 tx=0).stop 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 shutdown_connections 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9040d1da0 0x7fc90415c7d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fc8f006bbd0 0x7fc8f006e090 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9040d0760 0x7fc9040c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 --2- 172.21.15.82:0/2760786509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9040cf370 0x7fc9041f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 >> 172.21.15.82:0/2760786509 conn(0x7fc90409d030 msgr2=0x7fc9040b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 shutdown_connections 2024-03-14T12:18:25.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:25.545+0000 7fc909032640 1 -- 172.21.15.82:0/2760786509 wait complete. 2024-03-14T12:18:26.557 INFO:tasks.ceph:Scrubbing osd.0 2024-03-14T12:18:26.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.671+0000 7fcc0c359640 1 Processor -- start 2024-03-14T12:18:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.671+0000 7fcc0c359640 1 -- start start 2024-03-14T12:18:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0c359640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcc040ce5f0 0x7fcc040b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0c359640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcc0415f020 0x7fcc0415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0c359640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040cf290 con 0x7fcc040b2cc0 2024-03-14T12:18:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0c359640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040d0d30 con 0x7fcc040ce5f0 2024-03-14T12:18:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0c359640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040cf570 con 0x7fcc0415f020 2024-03-14T12:18:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0a0f5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcc040ce5f0 0x7fcc040b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0a0f5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcc040ce5f0 0x7fcc040b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55676/0 (socket says 172.21.15.82:55676) 2024-03-14T12:18:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.672+0000 7fcc0a0f5640 1 -- 172.21.15.82:0/2695841524 learned_addr learned my addr 172.21.15.82:0/2695841524 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.673+0000 7fcc0a8f6640 1 --2- 172.21.15.82:0/2695841524 >> v2:172.21.15.164:3300/0 conn(0x7fcc0415f020 0x7fcc0415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.673+0000 7fcc0a0f5640 1 -- 172.21.15.82:0/2695841524 >> v2:172.21.15.164:3300/0 conn(0x7fcc0415f020 msgr2=0x7fcc0415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.673+0000 7fcc0a0f5640 1 --2- 172.21.15.82:0/2695841524 >> v2:172.21.15.164:3300/0 conn(0x7fcc0415f020 0x7fcc0415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.673+0000 7fcc0a0f5640 1 -- 172.21.15.82:0/2695841524 >> v1:172.21.15.164:6789/0 conn(0x7fcc040b2cc0 legacy=0x7fcc040ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.673+0000 7fcc0a0f5640 1 -- 172.21.15.82:0/2695841524 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040d0270 con 0x7fcc040ce5f0 2024-03-14T12:18:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.673+0000 7fcc0a0f5640 1 --2- 172.21.15.82:0/2695841524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc040b42a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcc04071380 tx=0x7fcc0002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fac5bc57afb8ef server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc090f3640 1 -- 172.21.15.82:0/2695841524 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0002c9f0 con 0x7fcc040ce5f0 2024-03-14T12:18:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc090f3640 1 -- 172.21.15.82:0/2695841524 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0002cb90 con 0x7fcc040ce5f0 2024-03-14T12:18:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc090f3640 1 -- 172.21.15.82:0/2695841524 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000343f0 con 0x7fcc040ce5f0 2024-03-14T12:18:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 -- 172.21.15.82:0/2695841524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 msgr2=0x7fcc040b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 --2- 172.21.15.82:0/2695841524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc040b42a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcc04071380 tx=0x7fcc0002be50 comp rx=0 tx=0).stop 2024-03-14T12:18:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 -- 172.21.15.82:0/2695841524 shutdown_connections 2024-03-14T12:18:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 --2- 172.21.15.82:0/2695841524 >> v2:172.21.15.164:3300/0 conn(0x7fcc0415f020 0x7fcc0415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 --2- 172.21.15.82:0/2695841524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc040b42a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 -- 172.21.15.82:0/2695841524 >> 172.21.15.82:0/2695841524 conn(0x7fcc040b3780 msgr2=0x7fcc040b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 -- 172.21.15.82:0/2695841524 shutdown_connections 2024-03-14T12:18:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.674+0000 7fcc0c359640 1 -- 172.21.15.82:0/2695841524 wait complete. 2024-03-14T12:18:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.675+0000 7fcc0c359640 1 Processor -- start 2024-03-14T12:18:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.675+0000 7fcc0c359640 1 -- start start 2024-03-14T12:18:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0c359640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc040b2cc0 0x7fcc040c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0c359640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc0406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0c359640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc0415f020 0x7fcc0414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0c359640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415f9a0 con 0x7fcc040ce5f0 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0c359640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040d0830 con 0x7fcc0415f020 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0c359640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040b1520 con 0x7fcc040b2cc0 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc098f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc0406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc098f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc0406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55678/0 (socket says 172.21.15.82:55678) 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc098f4640 1 -- 172.21.15.82:0/2232378289 learned_addr learned my addr 172.21.15.82:0/2232378289 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0a8f6640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc0415f020 0x7fcc0414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.676+0000 7fcc0a0f5640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc040b2cc0 0x7fcc040c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc098f4640 1 -- 172.21.15.82:0/2232378289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc040b2cc0 msgr2=0x7fcc040c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc098f4640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc040b2cc0 0x7fcc040c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc098f4640 1 -- 172.21.15.82:0/2232378289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc0415f020 msgr2=0x7fcc0414f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc098f4640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc0415f020 0x7fcc0414f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc098f4640 1 -- 172.21.15.82:0/2232378289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040c5c20 con 0x7fcc040ce5f0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc098f4640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc0406d0d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c3f0 tx=0x7fcbf400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007020 con 0x7fcc040ce5f0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf40071c0 con 0x7fcc040ce5f0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007590 con 0x7fcc040ce5f0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040b76e0 con 0x7fcc040ce5f0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.677+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0406c8d0 con 0x7fcc040ce5f0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.678+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc0404f050 con 0x7fcc040ce5f0 2024-03-14T12:18:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.678+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcbf4019020 con 0x7fcc040ce5f0 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcbfaffd640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcbe806bc00 0x7fcbe806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcbf4089f20 con 0x7fcc040ce5f0 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcc0a0f5640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcbe806bc00 0x7fcbe806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcbfaffd640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7fcbe8071c00 0x7fcbe8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 --> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbf400ee50 con 0x7fcbe8071c00 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbf408c620 con 0x7fcc040ce5f0 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcc0a8f6640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7fcbe8071c00 0x7fcbe8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcc0a0f5640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcbe806bc00 0x7fcbe806e0c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcc00035000 tx=0x7fcc00030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.679+0000 7fcc0a8f6640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7fcbe8071c00 0x7fcbe8074020 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.680+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== osd.0 v2:172.21.15.82:6800/3139126855 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcbf400ee50 con 0x7fcbe8071c00 2024-03-14T12:18:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.693+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 --> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcc040cdfe0 con 0x7fcbe8071c00 2024-03-14T12:18:26.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.693+0000 7fcbfaffd640 1 -- 172.21.15.82:0/2232378289 <== osd.0 v2:172.21.15.82:6800/3139126855 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcc040cdfe0 con 0x7fcbe8071c00 2024-03-14T12:18:26.694 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:26.694 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:26.694 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.693+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7fcbe8071c00 msgr2=0x7fcbe8074020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.693+0000 7fcc0c359640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6800/3139126855,v1:172.21.15.82:6801/3139126855] conn(0x7fcbe8071c00 0x7fcbe8074020 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.693+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcbe806bc00 msgr2=0x7fcbe806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0c359640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fcbe806bc00 0x7fcbe806e0c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcc00035000 tx=0x7fcc00030000 comp rx=0 tx=0).stop 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 msgr2=0x7fcc0406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0c359640 1 --2- 172.21.15.82:0/2232378289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc040ce5f0 0x7fcc0406d0d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c3f0 tx=0x7fcbf400c8b0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0a8f6640 1 -- 172.21.15.82:0/2232378289 reap_dead start 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 shutdown_connections 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 >> 172.21.15.82:0/2232378289 conn(0x7fcc040b3780 msgr2=0x7fcc0406f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:26.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 shutdown_connections 2024-03-14T12:18:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.694+0000 7fcc0c359640 1 -- 172.21.15.82:0/2232378289 wait complete. 2024-03-14T12:18:26.704 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-14T12:18:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.805+0000 7f2642f22640 1 Processor -- start 2024-03-14T12:18:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.805+0000 7f2642f22640 1 -- start start 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f2642f22640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c15e670 0x7f263c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f2642f22640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f263c143f60 0x7f263c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f2642f22640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c15ebe0 con 0x7f263c071740 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f2642f22640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c15eec0 con 0x7f263c143f60 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f2642f22640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c15f1a0 con 0x7f263c15e670 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f26414bf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f263c143f60 0x7f263c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f263bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c15e670 0x7f263c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f2640cbe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f263c071740 0x7f263c071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.82:58588/0 (socket says 172.21.15.82:58588) 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f26414bf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f263c143f60 0x7f263c15b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:42796/0 (socket says 172.21.15.82:42796) 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f2640cbe640 1 -- 172.21.15.82:0/2291089636 learned_addr learned my addr 172.21.15.82:0/2291089636 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f263bfff640 1 -- 172.21.15.82:0/2291089636 >> v2:172.21.15.164:3300/0 conn(0x7f263c143f60 msgr2=0x7f263c15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:26.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f263bfff640 1 --2- 172.21.15.82:0/2291089636 >> v2:172.21.15.164:3300/0 conn(0x7f263c143f60 0x7f263c15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f263bfff640 1 -- 172.21.15.82:0/2291089636 >> v1:172.21.15.164:6789/0 conn(0x7f263c071740 legacy=0x7f263c071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f263bfff640 1 -- 172.21.15.82:0/2291089636 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c15f480 con 0x7f263c15e670 2024-03-14T12:18:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f263b7fe640 1 -- 172.21.15.82:0/2291089636 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490063569 0 0) 0x7f263c15ebe0 con 0x7f263c071740 2024-03-14T12:18:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.806+0000 7f26414bf640 1 --2- 172.21.15.82:0/2291089636 >> v2:172.21.15.164:3300/0 conn(0x7f263c143f60 0x7f263c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T12:18:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.807+0000 7f263bfff640 1 --2- 172.21.15.82:0/2291089636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c15e670 0x7f263c0b2940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26300097a0 tx=0x7f263002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2635e1809dfbf0a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.807+0000 7f263b7fe640 1 -- 172.21.15.82:0/2291089636 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630035020 con 0x7f263c15e670 2024-03-14T12:18:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.807+0000 7f263b7fe640 1 -- 172.21.15.82:0/2291089636 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630031830 con 0x7f263c15e670 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.807+0000 7f263b7fe640 1 -- 172.21.15.82:0/2291089636 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630031c00 con 0x7f263c15e670 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 -- 172.21.15.82:0/2291089636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c15e670 msgr2=0x7f263c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 --2- 172.21.15.82:0/2291089636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c15e670 0x7f263c0b2940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26300097a0 tx=0x7f263002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 -- 172.21.15.82:0/2291089636 shutdown_connections 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 --2- 172.21.15.82:0/2291089636 >> v2:172.21.15.164:3300/0 conn(0x7f263c143f60 0x7f263c15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 --2- 172.21.15.82:0/2291089636 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c15e670 0x7f263c0b2940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 -- 172.21.15.82:0/2291089636 >> 172.21.15.82:0/2291089636 conn(0x7f263c0b1c70 msgr2=0x7f263c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 -- 172.21.15.82:0/2291089636 shutdown_connections 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.808+0000 7f2642f22640 1 -- 172.21.15.82:0/2291089636 wait complete. 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.809+0000 7f2642f22640 1 Processor -- start 2024-03-14T12:18:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.809+0000 7f2642f22640 1 -- start start 2024-03-14T12:18:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2642f22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c071740 0x7f263c13f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2642f22640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c143f60 0x7f263c1f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2642f22640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f263c15e670 0x7f263c0c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2642f22640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c144670 con 0x7f263c071740 2024-03-14T12:18:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2642f22640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c15b080 con 0x7f263c15e670 2024-03-14T12:18:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2642f22640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c1447f0 con 0x7f263c143f60 2024-03-14T12:18:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c071740 0x7f263c13f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c071740 0x7f263c13f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55680/0 (socket says 172.21.15.82:55680) 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 -- 172.21.15.82:0/2739808728 learned_addr learned my addr 172.21.15.82:0/2739808728 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f263bfff640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c143f60 0x7f263c1f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f26414bf640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f263c15e670 0x7f263c0c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 -- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c143f60 msgr2=0x7f263c1f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c143f60 0x7f263c1f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 -- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f263c15e670 msgr2=0x7f263c0c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f263c15e670 0x7f263c0c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 -- 172.21.15.82:0/2739808728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c1f6580 con 0x7f263c071740 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.810+0000 7f2640cbe640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c071740 0x7f263c13f040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c430 tx=0x7f262c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.811+0000 7f26397fa640 1 -- 172.21.15.82:0/2739808728 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0074e0 con 0x7f263c071740 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.811+0000 7f26397fa640 1 -- 172.21.15.82:0/2739808728 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c0047e0 con 0x7f263c071740 2024-03-14T12:18:26.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.811+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c13dfc0 con 0x7f263c071740 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.811+0000 7f26397fa640 1 -- 172.21.15.82:0/2739808728 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c004bd0 con 0x7f263c071740 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.811+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c13e580 con 0x7f263c071740 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.812+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c04f050 con 0x7f263c071740 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.812+0000 7f26397fa640 1 -- 172.21.15.82:0/2739808728 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f262c011070 con 0x7f263c071740 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.812+0000 7f26397fa640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f262806bbd0 0x7f262806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.813+0000 7f263bfff640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f262806bbd0 0x7f262806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.813+0000 7f26397fa640 1 -- 172.21.15.82:0/2739808728 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f262c08a2d0 con 0x7f263c071740 2024-03-14T12:18:26.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.813+0000 7f263bfff640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f262806bbd0 0x7f262806e090 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2630036000 tx=0x7f263003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:26.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.817+0000 7f26397fa640 1 -- 172.21.15.82:0/2739808728 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f263c1f6580 con 0x7f263c071740 2024-03-14T12:18:26.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.989+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f263c0d07b0 con 0x7f262806bbd0 2024-03-14T12:18:26.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.989+0000 7f26397fa640 1 -- 172.21.15.82:0/2739808728 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f263c0d07b0 con 0x7f262806bbd0 2024-03-14T12:18:26.989 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 0 to deep-scrub 2024-03-14T12:18:26.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f262806bbd0 msgr2=0x7f262806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f262806bbd0 0x7f262806e090 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2630036000 tx=0x7f263003d040 comp rx=0 tx=0).stop 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c071740 msgr2=0x7f263c13f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c071740 0x7f263c13f040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c430 tx=0x7f262c00c830 comp rx=0 tx=0).stop 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 shutdown_connections 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f263c15e670 0x7f263c0c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f262806bbd0 0x7f262806e090 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f263c143f60 0x7f263c1f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 --2- 172.21.15.82:0/2739808728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c071740 0x7f263c13f040 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 >> 172.21.15.82:0/2739808728 conn(0x7f263c0b1c70 msgr2=0x7f263c15d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 shutdown_connections 2024-03-14T12:18:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:26.991+0000 7f2642f22640 1 -- 172.21.15.82:0/2739808728 wait complete. 2024-03-14T12:18:27.002 INFO:tasks.ceph:Scrubbing osd.1 2024-03-14T12:18:27.003 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:27.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.106+0000 7fbe3249a640 1 Processor -- start 2024-03-14T12:18:27.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.106+0000 7fbe3249a640 1 -- start start 2024-03-14T12:18:27.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe3249a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe2c15a530 0x7fbe2c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe3249a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbe2c15e910 0x7fbe2c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe3249a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2c06dd80 con 0x7fbe2c0cc650 2024-03-14T12:18:27.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe3249a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2c06e340 con 0x7fbe2c15a530 2024-03-14T12:18:27.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe3249a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2c06db30 con 0x7fbe2c15e910 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe2bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe2c15a530 0x7fbe2c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe2bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe2c15a530 0x7fbe2c0cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55696/0 (socket says 172.21.15.82:55696) 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe2bfff640 1 -- 172.21.15.82:0/3275578051 learned_addr learned my addr 172.21.15.82:0/3275578051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe30a37640 1 --2- 172.21.15.82:0/3275578051 >> v2:172.21.15.164:3300/0 conn(0x7fbe2c15e910 0x7fbe2c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe2bfff640 1 -- 172.21.15.82:0/3275578051 >> v2:172.21.15.164:3300/0 conn(0x7fbe2c15e910 msgr2=0x7fbe2c071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe2bfff640 1 --2- 172.21.15.82:0/3275578051 >> v2:172.21.15.164:3300/0 conn(0x7fbe2c15e910 0x7fbe2c071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe2bfff640 1 -- 172.21.15.82:0/3275578051 >> v1:172.21.15.164:6789/0 conn(0x7fbe2c0cc650 legacy=0x7fbe2c15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.107+0000 7fbe2bfff640 1 -- 172.21.15.82:0/3275578051 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c06e060 con 0x7fbe2c15a530 2024-03-14T12:18:27.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe2bfff640 1 --2- 172.21.15.82:0/3275578051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cf390 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0b4da0 tx=0x7fbe1c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a16505375a1077 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe2affd640 1 -- 172.21.15.82:0/3275578051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c03b070 con 0x7fbe2c15a530 2024-03-14T12:18:27.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe2affd640 1 -- 172.21.15.82:0/3275578051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c02ee00 con 0x7fbe2c15a530 2024-03-14T12:18:27.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe2affd640 1 -- 172.21.15.82:0/3275578051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c036660 con 0x7fbe2c15a530 2024-03-14T12:18:27.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe3249a640 1 -- 172.21.15.82:0/3275578051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 msgr2=0x7fbe2c0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe3249a640 1 --2- 172.21.15.82:0/3275578051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cf390 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0b4da0 tx=0x7fbe1c02e050 comp rx=0 tx=0).stop 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe3249a640 1 -- 172.21.15.82:0/3275578051 shutdown_connections 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe3249a640 1 --2- 172.21.15.82:0/3275578051 >> v2:172.21.15.164:3300/0 conn(0x7fbe2c15e910 0x7fbe2c071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe3249a640 1 --2- 172.21.15.82:0/3275578051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cf390 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.108+0000 7fbe3249a640 1 -- 172.21.15.82:0/3275578051 >> 172.21.15.82:0/3275578051 conn(0x7fbe2c0b19b0 msgr2=0x7fbe2c0b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.109+0000 7fbe3249a640 1 -- 172.21.15.82:0/3275578051 shutdown_connections 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.109+0000 7fbe3249a640 1 -- 172.21.15.82:0/3275578051 wait complete. 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.110+0000 7fbe3249a640 1 Processor -- start 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.110+0000 7fbe3249a640 1 -- start start 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.110+0000 7fbe3249a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe2c0cc650 0x7fbe2c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe3249a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe3249a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c15e910 0x7fbe2c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe3249a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c15a910 con 0x7fbe2c15a530 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe3249a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c070af0 con 0x7fbe2c15e910 2024-03-14T12:18:27.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe3249a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c15aa90 con 0x7fbe2c0cc650 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55712/0 (socket says 172.21.15.82:55712) 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 -- 172.21.15.82:0/48016596 learned_addr learned my addr 172.21.15.82:0/48016596 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 -- 172.21.15.82:0/48016596 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe2c0cc650 msgr2=0x7fbe2c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe2c0cc650 0x7fbe2c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 -- 172.21.15.82:0/48016596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c15e910 msgr2=0x7fbe2c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe2c15e910 0x7fbe2c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 -- 172.21.15.82:0/48016596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c159140 con 0x7fbe2c15a530 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe2b7fe640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cdb90 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c400 tx=0x7fbe2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20011790 con 0x7fbe2c15a530 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe20017070 con 0x7fbe2c15a530 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c159420 con 0x7fbe2c15a530 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20011bb0 con 0x7fbe2c15a530 2024-03-14T12:18:27.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.111+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c151550 con 0x7fbe2c15a530 2024-03-14T12:18:27.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.112+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe2c0b4b60 con 0x7fbe2c15a530 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fbe2001c460 con 0x7fbe2c15a530 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe28ff9640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe1806bbd0 0x7fbe1806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbe2008a5e0 con 0x7fbe2c15a530 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe2bfff640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe1806bbd0 0x7fbe1806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe28ff9640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7fbe18071bd0 0x7fbe18073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 --> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe180746a0 con 0x7fbe18071bd0 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe20091050 con 0x7fbe2c15a530 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.113+0000 7fbe30a37640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7fbe18071bd0 0x7fbe18073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.114+0000 7fbe2bfff640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe1806bbd0 0x7fbe1806e090 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c038000 tx=0x7fbe1c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.114+0000 7fbe30a37640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7fbe18071bd0 0x7fbe18073ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.114+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== osd.1 v2:172.21.15.82:6816/2003973578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbe180746a0 con 0x7fbe18071bd0 2024-03-14T12:18:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.127+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 --> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbe2c04f050 con 0x7fbe18071bd0 2024-03-14T12:18:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe28ff9640 1 -- 172.21.15.82:0/48016596 <== osd.1 v2:172.21.15.82:6816/2003973578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbe2c04f050 con 0x7fbe18071bd0 2024-03-14T12:18:27.128 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:27.128 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:27.128 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:27.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7fbe18071bd0 msgr2=0x7fbe18073ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6816/2003973578,v1:172.21.15.82:6817/2003973578] conn(0x7fbe18071bd0 0x7fbe18073ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe1806bbd0 msgr2=0x7fbe1806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fbe1806bbd0 0x7fbe1806e090 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c038000 tx=0x7fbe1c030000 comp rx=0 tx=0).stop 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 msgr2=0x7fbe2c0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 --2- 172.21.15.82:0/48016596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15a530 0x7fbe2c0cdb90 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c400 tx=0x7fbe2000c8c0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe30a37640 1 -- 172.21.15.82:0/48016596 reap_dead start 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 shutdown_connections 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 >> 172.21.15.82:0/48016596 conn(0x7fbe2c0b19b0 msgr2=0x7fbe2c0cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 shutdown_connections 2024-03-14T12:18:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.128+0000 7fbe3249a640 1 -- 172.21.15.82:0/48016596 wait complete. 2024-03-14T12:18:27.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-14T12:18:27.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.240+0000 7f083ca80640 1 Processor -- start 2024-03-14T12:18:27.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.240+0000 7f083ca80640 1 -- start start 2024-03-14T12:18:27.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083ca80640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0838071740 0x7f0838071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083ca80640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838143f60 0x7f083815b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083ca80640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380b3050 con 0x7f083815e670 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083ca80640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0838072060 con 0x7f0838071740 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083ca80640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083815ebe0 con 0x7f0838143f60 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0838071740 0x7f0838071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0838071740 0x7f0838071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55724/0 (socket says 172.21.15.82:55724) 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f083659c640 1 -- 172.21.15.82:0/4287231502 learned_addr learned my addr 172.21.15.82:0/4287231502 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.241+0000 7f0836d9d640 1 --2- 172.21.15.82:0/4287231502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838143f60 0x7f083815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083659c640 1 -- 172.21.15.82:0/4287231502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838143f60 msgr2=0x7f083815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083659c640 1 --2- 172.21.15.82:0/4287231502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838143f60 0x7f083815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083659c640 1 -- 172.21.15.82:0/4287231502 >> v1:172.21.15.164:6789/0 conn(0x7f083815e670 legacy=0x7f08380b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083659c640 1 -- 172.21.15.82:0/4287231502 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083815eec0 con 0x7f0838071740 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083659c640 1 --2- 172.21.15.82:0/4287231502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838071740 0x7f0838071b20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0828009e50 tx=0x7f082802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f687ffd3db3d3e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083559a640 1 -- 172.21.15.82:0/4287231502 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082802cb00 con 0x7f0838071740 2024-03-14T12:18:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083559a640 1 -- 172.21.15.82:0/4287231502 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082802cca0 con 0x7f0838071740 2024-03-14T12:18:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.242+0000 7f083559a640 1 -- 172.21.15.82:0/4287231502 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828034610 con 0x7f0838071740 2024-03-14T12:18:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 -- 172.21.15.82:0/4287231502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838071740 msgr2=0x7f0838071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 --2- 172.21.15.82:0/4287231502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838071740 0x7f0838071b20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0828009e50 tx=0x7f082802bf60 comp rx=0 tx=0).stop 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 -- 172.21.15.82:0/4287231502 shutdown_connections 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 --2- 172.21.15.82:0/4287231502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838143f60 0x7f083815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 --2- 172.21.15.82:0/4287231502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838071740 0x7f0838071b20 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 -- 172.21.15.82:0/4287231502 >> 172.21.15.82:0/4287231502 conn(0x7f08380b1c70 msgr2=0x7f08380cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 -- 172.21.15.82:0/4287231502 shutdown_connections 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.243+0000 7f083ca80640 1 -- 172.21.15.82:0/4287231502 wait complete. 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.244+0000 7f083ca80640 1 Processor -- start 2024-03-14T12:18:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.244+0000 7f083ca80640 1 -- start start 2024-03-14T12:18:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f083ca80640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838071740 0x7f08380c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f083ca80640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838143f60 0x7f083813da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f083ca80640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083815e670 0x7f0838156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f083ca80640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838143ce0 con 0x7f0838143f60 2024-03-14T12:18:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f083ca80640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083815f480 con 0x7f083815e670 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f083ca80640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083815b080 con 0x7f0838071740 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f0835d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838143f60 0x7f083813da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f0835d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838143f60 0x7f083813da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55734/0 (socket says 172.21.15.82:55734) 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f0835d9b640 1 -- 172.21.15.82:0/2371184847 learned_addr learned my addr 172.21.15.82:0/2371184847 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f083659c640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838071740 0x7f08380c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.245+0000 7f0836d9d640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083815e670 0x7f0838156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0835d9b640 1 -- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838071740 msgr2=0x7f08380c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0835d9b640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838071740 0x7f08380c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0835d9b640 1 -- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083815e670 msgr2=0x7f0838156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0835d9b640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083815e670 0x7f0838156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0835d9b640 1 -- 172.21.15.82:0/2371184847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08381417a0 con 0x7f0838143f60 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0835d9b640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838143f60 0x7f083813da50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c400 tx=0x7f082c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0826ffd640 1 -- 172.21.15.82:0/2371184847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c0052d0 con 0x7f0838143f60 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0826ffd640 1 -- 172.21.15.82:0/2371184847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082c005490 con 0x7f0838143f60 2024-03-14T12:18:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0838158620 con 0x7f0838143f60 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f0826ffd640 1 -- 172.21.15.82:0/2371184847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c007260 con 0x7f0838143f60 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.246+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08381411e0 con 0x7f0838143f60 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.247+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083804f050 con 0x7f0838143f60 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.247+0000 7f0826ffd640 1 -- 172.21.15.82:0/2371184847 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f082c007400 con 0x7f0838143f60 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.248+0000 7f0826ffd640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f081806bbd0 0x7f081806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.248+0000 7f083659c640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f081806bbd0 0x7f081806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.248+0000 7f0826ffd640 1 -- 172.21.15.82:0/2371184847 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f082c089d90 con 0x7f0838143f60 2024-03-14T12:18:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.248+0000 7f083659c640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f081806bbd0 0x7f081806e090 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0828035000 tx=0x7f0828030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.252+0000 7f0826ffd640 1 -- 172.21.15.82:0/2371184847 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f082c058ad0 con 0x7f0838143f60 2024-03-14T12:18:27.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.424+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f083813bbd0 con 0x7f081806bbd0 2024-03-14T12:18:27.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.424+0000 7f0826ffd640 1 -- 172.21.15.82:0/2371184847 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f083813bbd0 con 0x7f081806bbd0 2024-03-14T12:18:27.425 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 1 to deep-scrub 2024-03-14T12:18:27.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f081806bbd0 msgr2=0x7f081806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f081806bbd0 0x7f081806e090 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0828035000 tx=0x7f0828030000 comp rx=0 tx=0).stop 2024-03-14T12:18:27.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838143f60 msgr2=0x7f083813da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838143f60 0x7f083813da50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c400 tx=0x7f082c00c8c0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 shutdown_connections 2024-03-14T12:18:27.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083815e670 0x7f0838156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0838143f60 0x7f083813da50 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f081806bbd0 0x7f081806e090 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 --2- 172.21.15.82:0/2371184847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0838071740 0x7f08380c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 >> 172.21.15.82:0/2371184847 conn(0x7f08380b1c70 msgr2=0x7f0838142cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 shutdown_connections 2024-03-14T12:18:27.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.426+0000 7f083ca80640 1 -- 172.21.15.82:0/2371184847 wait complete. 2024-03-14T12:18:27.437 INFO:tasks.ceph:Scrubbing osd.2 2024-03-14T12:18:27.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:27.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.542+0000 7f1770da2640 1 Processor -- start 2024-03-14T12:18:27.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.542+0000 7f1770da2640 1 -- start start 2024-03-14T12:18:27.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f1770da2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 0x7f176c169210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f1770da2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0d16d0 con 0x7f176c0b1f40 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f1770da2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0b30d0 con 0x7f176c0b2650 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f1770da2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cef40 con 0x7f176c166e20 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f176c0b1f40 0x7f176c0ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42836/0 (socket says 172.21.15.82:42836) 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176b7fe640 1 -- 172.21.15.82:0/2073673676 learned_addr learned my addr 172.21.15.82:0/2073673676 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091062078 0 0) 0x7f176c0d16d0 con 0x7f176c0b1f40 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1754003520 con 0x7f176c0b1f40 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176bfff640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 0x7f176c169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442267376 0 0) 0x7f1754003520 con 0x7f176c0b1f40 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 msgr2=0x7f176c169210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 0x7f176c169210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 >> v1:172.21.15.164:6789/0 conn(0x7f176c0b2650 legacy=0x7f176c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c06c960 con 0x7f176c0b1f40 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f1760001670 con 0x7f176c0b1f40 2024-03-14T12:18:27.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 >> v1:172.21.15.82:6789/0 conn(0x7f176c0b1f40 legacy=0x7f176c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f1754003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 0x7f1754004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1754004d40 0x7f17540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cef40 con 0x7f17540039e0 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175400bb20 con 0x7f1754004d40 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.543+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175400bd20 con 0x7f176c166e20 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176b7fe640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f1754003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176affd640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 0x7f1754004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176bfff640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1754004d40 0x7f17540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176b7fe640 1 -- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 msgr2=0x7f1754004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176b7fe640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c166e20 0x7f1754004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176b7fe640 1 -- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1754004d40 msgr2=0x7f17540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176b7fe640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1754004d40 0x7f17540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176b7fe640 1 -- 172.21.15.82:0/2073673676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1754003520 con 0x7f17540039e0 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176bfff640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1754004d40 0x7f17540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:27.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.544+0000 7f176b7fe640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f1754003e00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f176000c210 tx=0x7f1754005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4c0a76cd56490d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760007d30 con 0x7f17540039e0 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1760030070 con 0x7f17540039e0 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f176a7fc640 1 -- 172.21.15.82:0/2073673676 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760005d30 con 0x7f17540039e0 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f1770da2640 1 -- 172.21.15.82:0/2073673676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 msgr2=0x7f1754003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f1770da2640 1 --2- 172.21.15.82:0/2073673676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f1754003e00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f176000c210 tx=0x7f1754005830 comp rx=0 tx=0).stop 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f176bfff640 1 -- 172.21.15.82:0/2073673676 reap_dead start 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f1770da2640 1 -- 172.21.15.82:0/2073673676 shutdown_connections 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f1770da2640 1 -- 172.21.15.82:0/2073673676 >> 172.21.15.82:0/2073673676 conn(0x7f176c0b33b0 msgr2=0x7f176c0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f1770da2640 1 -- 172.21.15.82:0/2073673676 shutdown_connections 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.545+0000 7f1770da2640 1 -- 172.21.15.82:0/2073673676 wait complete. 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.546+0000 7f1770da2640 1 Processor -- start 2024-03-14T12:18:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.546+0000 7f1770da2640 1 -- start start 2024-03-14T12:18:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f1770da2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f176c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f1770da2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176c0b58e0 0x7f176c0af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f1770da2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c0afaa0 0x7f176c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f1770da2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b2ba0 con 0x7f17540039e0 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f1770da2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c06cc00 con 0x7f176c0b58e0 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f1770da2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c06cee0 con 0x7f176c0afaa0 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f176c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f176c158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55760/0 (socket says 172.21.15.82:55760) 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 -- 172.21.15.82:0/3417340564 learned_addr learned my addr 172.21.15.82:0/3417340564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 -- 172.21.15.82:0/3417340564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c0afaa0 msgr2=0x7f176c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f176c0afaa0 0x7f176c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 -- 172.21.15.82:0/3417340564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176c0b58e0 msgr2=0x7f176c0af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f176c0b58e0 0x7f176c0af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.547+0000 7f176b7fe640 1 -- 172.21.15.82:0/3417340564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c158ea0 con 0x7f17540039e0 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.548+0000 7f176b7fe640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f176c158960 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1760000c00 tx=0x7f1760001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.548+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760004bf0 con 0x7f17540039e0 2024-03-14T12:18:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.548+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1760004db0 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.548+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c0b4840 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.548+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17600072b0 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.548+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c0b4e00 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.549+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f176c0b54c0 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.549+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f1760007450 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f1753fff640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f175806bbd0 0x7f175806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f176affd640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f175806bbd0 0x7f175806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f17600ab9b0 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f1753fff640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7f1758071bd0 0x7f1758073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 --> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f176002ee50 con 0x7f1758071bd0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17600b02c0 con 0x7f17540039e0 2024-03-14T12:18:27.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f176bfff640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7f1758071bd0 0x7f1758073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f176affd640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f175806bbd0 0x7f175806e090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f176c15c200 tx=0x7f175c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.550+0000 7f176bfff640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7f1758071bd0 0x7f1758073ff0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.551+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== osd.2 v2:172.21.15.82:6808/2455719014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f176002ee50 con 0x7f1758071bd0 2024-03-14T12:18:27.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 --> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f176c075210 con 0x7f1758071bd0 2024-03-14T12:18:27.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1753fff640 1 -- 172.21.15.82:0/3417340564 <== osd.2 v2:172.21.15.82:6808/2455719014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f176c075210 con 0x7f1758071bd0 2024-03-14T12:18:27.564 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:27.564 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:27.564 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:27.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7f1758071bd0 msgr2=0x7f1758073ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6808/2455719014,v1:172.21.15.82:6809/2455719014] conn(0x7f1758071bd0 0x7f1758073ff0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f175806bbd0 msgr2=0x7f175806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f175806bbd0 0x7f175806e090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f176c15c200 tx=0x7f175c009040 comp rx=0 tx=0).stop 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 msgr2=0x7f176c158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 --2- 172.21.15.82:0/3417340564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17540039e0 0x7f176c158960 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1760000c00 tx=0x7f1760001a40 comp rx=0 tx=0).stop 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f176bfff640 1 -- 172.21.15.82:0/3417340564 reap_dead start 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 shutdown_connections 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 >> 172.21.15.82:0/3417340564 conn(0x7f176c0b33b0 msgr2=0x7f176c06f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 shutdown_connections 2024-03-14T12:18:27.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.564+0000 7f1770da2640 1 -- 172.21.15.82:0/3417340564 wait complete. 2024-03-14T12:18:27.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-14T12:18:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.676+0000 7efc47654640 1 Processor -- start 2024-03-14T12:18:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.676+0000 7efc47654640 1 -- start start 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc47654640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efc4015e670 0x7efc400b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc47654640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efc40143f60 0x7efc4015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc47654640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4015ebe0 con 0x7efc40071740 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc47654640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4015eec0 con 0x7efc40143f60 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc47654640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4015f1a0 con 0x7efc4015e670 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc45bf1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efc40143f60 0x7efc4015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc453f0640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7efc40071740 0x7efc40071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42844/0 (socket says 172.21.15.82:42844) 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc453f0640 1 -- 172.21.15.82:0/3799717753 learned_addr learned my addr 172.21.15.82:0/3799717753 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc45bf1640 1 -- 172.21.15.82:0/3799717753 >> v2:172.21.15.164:3300/0 conn(0x7efc4015e670 msgr2=0x7efc400b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc45bf1640 1 --2- 172.21.15.82:0/3799717753 >> v2:172.21.15.164:3300/0 conn(0x7efc4015e670 0x7efc400b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc45bf1640 1 -- 172.21.15.82:0/3799717753 >> v1:172.21.15.82:6789/0 conn(0x7efc40071740 legacy=0x7efc40071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc45bf1640 1 -- 172.21.15.82:0/3799717753 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4015f480 con 0x7efc40143f60 2024-03-14T12:18:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.677+0000 7efc37fff640 1 -- 172.21.15.82:0/3799717753 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750756685 0 0) 0x7efc4015ebe0 con 0x7efc40071740 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.678+0000 7efc45bf1640 1 --2- 172.21.15.82:0/3799717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc40143f60 0x7efc4015b610 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7efc38009d90 tx=0x7efc3802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4045472b87731d99 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.678+0000 7efc37fff640 1 -- 172.21.15.82:0/3799717753 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38039070 con 0x7efc40143f60 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.678+0000 7efc37fff640 1 -- 172.21.15.82:0/3799717753 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3802cc60 con 0x7efc40143f60 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.678+0000 7efc37fff640 1 -- 172.21.15.82:0/3799717753 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38034660 con 0x7efc40143f60 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 -- 172.21.15.82:0/3799717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc40143f60 msgr2=0x7efc4015b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 --2- 172.21.15.82:0/3799717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc40143f60 0x7efc4015b610 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7efc38009d90 tx=0x7efc3802beb0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 -- 172.21.15.82:0/3799717753 shutdown_connections 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 --2- 172.21.15.82:0/3799717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc40143f60 0x7efc4015b610 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 --2- 172.21.15.82:0/3799717753 >> v2:172.21.15.164:3300/0 conn(0x7efc4015e670 0x7efc400b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 -- 172.21.15.82:0/3799717753 >> 172.21.15.82:0/3799717753 conn(0x7efc400b1c70 msgr2=0x7efc400cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 -- 172.21.15.82:0/3799717753 shutdown_connections 2024-03-14T12:18:27.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.679+0000 7efc47654640 1 -- 172.21.15.82:0/3799717753 wait complete. 2024-03-14T12:18:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.680+0000 7efc47654640 1 Processor -- start 2024-03-14T12:18:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.680+0000 7efc47654640 1 -- start start 2024-03-14T12:18:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc47654640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc40071740 0x7efc401e4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc47654640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc40143f60 0x7efc4015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc47654640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4015e670 0x7efc4012f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc47654640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40144670 con 0x7efc4015e670 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc47654640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4015b080 con 0x7efc40143f60 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc47654640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc401447f0 con 0x7efc40071740 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4015e670 0x7efc4012f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4015e670 0x7efc4012f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55778/0 (socket says 172.21.15.82:55778) 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 -- 172.21.15.82:0/3686188810 learned_addr learned my addr 172.21.15.82:0/3686188810 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc44bef640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc40143f60 0x7efc4015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc453f0640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc40071740 0x7efc401e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 -- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc40071740 msgr2=0x7efc401e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc40071740 0x7efc401e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 -- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc40143f60 msgr2=0x7efc4015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc40143f60 0x7efc4015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 -- 172.21.15.82:0/3686188810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4012f690 con 0x7efc4015e670 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc453f0640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc40071740 0x7efc401e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc45bf1640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4015e670 0x7efc4012f150 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7efc38009eb0 tx=0x7efc38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc35ffb640 1 -- 172.21.15.82:0/3686188810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380463f0 con 0x7efc4015e670 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc35ffb640 1 -- 172.21.15.82:0/3686188810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc380465b0 con 0x7efc4015e670 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4015d3f0 con 0x7efc4015e670 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.681+0000 7efc35ffb640 1 -- 172.21.15.82:0/3686188810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380469a0 con 0x7efc4015e670 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.682+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4015d9b0 con 0x7efc4015e670 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.683+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4004f050 con 0x7efc4015e670 2024-03-14T12:18:27.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.683+0000 7efc35ffb640 1 -- 172.21.15.82:0/3686188810 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7efc38046b40 con 0x7efc4015e670 2024-03-14T12:18:27.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.684+0000 7efc35ffb640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc2406bbd0 0x7efc2406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.684+0000 7efc453f0640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc2406bbd0 0x7efc2406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.684+0000 7efc35ffb640 1 -- 172.21.15.82:0/3686188810 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7efc380aed70 con 0x7efc4015e670 2024-03-14T12:18:27.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.684+0000 7efc453f0640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc2406bbd0 0x7efc2406e090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000f60 tx=0x7efc3c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.688+0000 7efc35ffb640 1 -- 172.21.15.82:0/3686188810 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7efc38041070 con 0x7efc4015e670 2024-03-14T12:18:27.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.860+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7efc4015cfd0 con 0x7efc2406bbd0 2024-03-14T12:18:27.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.860+0000 7efc35ffb640 1 -- 172.21.15.82:0/3686188810 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efc4015cfd0 con 0x7efc2406bbd0 2024-03-14T12:18:27.861 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 2 to deep-scrub 2024-03-14T12:18:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc2406bbd0 msgr2=0x7efc2406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc2406bbd0 0x7efc2406e090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000f60 tx=0x7efc3c009040 comp rx=0 tx=0).stop 2024-03-14T12:18:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4015e670 msgr2=0x7efc4012f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4015e670 0x7efc4012f150 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7efc38009eb0 tx=0x7efc38004a40 comp rx=0 tx=0).stop 2024-03-14T12:18:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 shutdown_connections 2024-03-14T12:18:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4015e670 0x7efc4012f150 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc40143f60 0x7efc4015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7efc2406bbd0 0x7efc2406e090 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 --2- 172.21.15.82:0/3686188810 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efc40071740 0x7efc401e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 >> 172.21.15.82:0/3686188810 conn(0x7efc400b1c70 msgr2=0x7efc400afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 shutdown_connections 2024-03-14T12:18:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.862+0000 7efc47654640 1 -- 172.21.15.82:0/3686188810 wait complete. 2024-03-14T12:18:27.874 INFO:tasks.ceph:Scrubbing osd.3 2024-03-14T12:18:27.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:27.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.977+0000 7f9c4ace8640 1 Processor -- start 2024-03-14T12:18:27.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.977+0000 7f9c4ace8640 1 -- start start 2024-03-14T12:18:27.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.978+0000 7f9c4ace8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 0x7f9c440b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.978+0000 7f9c4ace8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c441629f0 con 0x7f9c4415e1b0 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.978+0000 7f9c4ace8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c440d0aa0 con 0x7f9c44162430 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.978+0000 7f9c4ace8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44162c90 con 0x7f9c440b2720 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.978+0000 7f9c48a84640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9c4415e1b0 0x7f9c44161d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42852/0 (socket says 172.21.15.82:42852) 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.978+0000 7f9c48a84640 1 -- 172.21.15.82:0/3424007394 learned_addr learned my addr 172.21.15.82:0/3424007394 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c49285640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 0x7f9c440b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414750323 0 0) 0x7f9c441629f0 con 0x7f9c4415e1b0 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c2c003520 con 0x7f9c4415e1b0 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2403229887 0 0) 0x7f9c2c003520 con 0x7f9c4415e1b0 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 msgr2=0x7f9c440b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 0x7f9c440b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 >> v1:172.21.15.164:6789/0 conn(0x7f9c44162430 legacy=0x7f9c4406bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c440d0d40 con 0x7f9c4415e1b0 2024-03-14T12:18:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f9c34001670 con 0x7f9c4415e1b0 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 >> v1:172.21.15.82:6789/0 conn(0x7f9c4415e1b0 legacy=0x7f9c44161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 0x7f9c2c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c2c003f60 0x7f9c2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c2c004d40 0x7f9c2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44162c90 con 0x7f9c2c004d40 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c00bb50 con 0x7f9c2c003f60 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c00bd50 con 0x7f9c440b2720 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c49285640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c2c004d40 0x7f9c2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c48a84640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 0x7f9c2c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c43fff640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c2c003f60 0x7f9c2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c49285640 1 -- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 msgr2=0x7f9c2c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c49285640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c440b2720 0x7f9c2c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c49285640 1 -- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c2c003f60 msgr2=0x7f9c2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c49285640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c2c003f60 0x7f9c2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.979+0000 7f9c49285640 1 -- 172.21.15.82:0/3424007394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c003520 con 0x7f9c2c004d40 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.980+0000 7f9c49285640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c2c004d40 0x7f9c2c0051c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009fc0 tx=0x7f9c2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805d01503290a056 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.980+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c039070 con 0x7f9c2c004d40 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.980+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c02cc20 con 0x7f9c2c004d40 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.980+0000 7f9c437fe640 1 -- 172.21.15.82:0/3424007394 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c009c10 con 0x7f9c2c004d40 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.981+0000 7f9c4ace8640 1 -- 172.21.15.82:0/3424007394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c2c004d40 msgr2=0x7f9c2c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.981+0000 7f9c4ace8640 1 --2- 172.21.15.82:0/3424007394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c2c004d40 0x7f9c2c0051c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009fc0 tx=0x7f9c2c005830 comp rx=0 tx=0).stop 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.981+0000 7f9c49285640 1 -- 172.21.15.82:0/3424007394 reap_dead start 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.981+0000 7f9c4ace8640 1 -- 172.21.15.82:0/3424007394 shutdown_connections 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.981+0000 7f9c4ace8640 1 -- 172.21.15.82:0/3424007394 >> 172.21.15.82:0/3424007394 conn(0x7f9c440b3610 msgr2=0x7f9c440b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.981+0000 7f9c4ace8640 1 -- 172.21.15.82:0/3424007394 shutdown_connections 2024-03-14T12:18:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.981+0000 7f9c4ace8640 1 -- 172.21.15.82:0/3424007394 wait complete. 2024-03-14T12:18:27.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.982+0000 7f9c4ace8640 1 Processor -- start 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.982+0000 7f9c4ace8640 1 -- start start 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c4ace8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c44158350 0x7f9c440b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c4ace8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c44158730 0x7f9c440b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c4ace8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c44159120 0x7f9c440b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c4ace8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440b4130 con 0x7f9c44158730 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c4ace8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4415f5a0 con 0x7f9c44158350 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c4ace8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440b42b0 con 0x7f9c44159120 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c44158730 0x7f9c440b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c44158730 0x7f9c440b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55802/0 (socket says 172.21.15.82:55802) 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 -- 172.21.15.82:0/686724965 learned_addr learned my addr 172.21.15.82:0/686724965 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c48a84640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c44158350 0x7f9c440b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c49285640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c44159120 0x7f9c440b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 -- 172.21.15.82:0/686724965 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c44159120 msgr2=0x7f9c440b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c44159120 0x7f9c440b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 -- 172.21.15.82:0/686724965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c44158350 msgr2=0x7f9c440b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c44158350 0x7f9c440b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 -- 172.21.15.82:0/686724965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c440b4f50 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c43fff640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c44158730 0x7f9c440b4700 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800f820 tx=0x7f9c3800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.983+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380050d0 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.984+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c38018070 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.984+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c440b5230 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.984+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380054b0 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.984+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c440b57f0 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.984+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c4404f050 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.985+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f9c3800d6d0 con 0x7f9c44158730 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.985+0000 7f9c417fa640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c3006bbd0 0x7f9c3006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.985+0000 7f9c48a84640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c3006bbd0 0x7f9c3006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.985+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9c3808a5d0 con 0x7f9c44158730 2024-03-14T12:18:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.985+0000 7f9c417fa640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f9c30071bd0 0x7f9c30073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.985+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 --> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c38014e50 con 0x7f9c30071bd0 2024-03-14T12:18:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.985+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c38091050 con 0x7f9c44158730 2024-03-14T12:18:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.986+0000 7f9c49285640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f9c30071bd0 0x7f9c30073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.986+0000 7f9c48a84640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c3006bbd0 0x7f9c3006e090 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9c34001050 tx=0x7f9c34000f80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.986+0000 7f9c49285640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f9c30071bd0 0x7f9c30073ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.986+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== osd.3 v2:172.21.15.82:6824/994456991 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9c38014e50 con 0x7f9c30071bd0 2024-03-14T12:18:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:27.999+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 --> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9c44161840 con 0x7f9c30071bd0 2024-03-14T12:18:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c417fa640 1 -- 172.21.15.82:0/686724965 <== osd.3 v2:172.21.15.82:6824/994456991 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9c44161840 con 0x7f9c30071bd0 2024-03-14T12:18:28.000 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:28.000 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:28.000 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f9c30071bd0 msgr2=0x7f9c30073ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6824/994456991,v1:172.21.15.82:6825/994456991] conn(0x7f9c30071bd0 0x7f9c30073ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c3006bbd0 msgr2=0x7f9c3006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f9c3006bbd0 0x7f9c3006e090 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9c34001050 tx=0x7f9c34000f80 comp rx=0 tx=0).stop 2024-03-14T12:18:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c44158730 msgr2=0x7f9c440b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 --2- 172.21.15.82:0/686724965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c44158730 0x7f9c440b4700 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800f820 tx=0x7f9c3800fce0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c49285640 1 -- 172.21.15.82:0/686724965 reap_dead start 2024-03-14T12:18:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 shutdown_connections 2024-03-14T12:18:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 >> 172.21.15.82:0/686724965 conn(0x7f9c440b3610 msgr2=0x7f9c44159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 shutdown_connections 2024-03-14T12:18:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.000+0000 7f9c4ace8640 1 -- 172.21.15.82:0/686724965 wait complete. 2024-03-14T12:18:28.011 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-14T12:18:28.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.111+0000 7fac12c44640 1 Processor -- start 2024-03-14T12:18:28.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.111+0000 7fac12c44640 1 -- start start 2024-03-14T12:18:28.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac12c44640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac0c15f7a0 0x7fac0c15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac12c44640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c1451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac12c44640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c0ce620 con 0x7fac0c0cdd60 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac12c44640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0c0cf150 con 0x7fac0c15f7a0 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac12c44640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c1600c0 con 0x7fac0c0ce210 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac109e0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac0c15f7a0 0x7fac0c15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac0bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fac0c0cdd60 0x7fac0c144ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42854/0 (socket says 172.21.15.82:42854) 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac0bfff640 1 -- 172.21.15.82:0/837007828 learned_addr learned my addr 172.21.15.82:0/837007828 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac109e0640 1 -- 172.21.15.82:0/837007828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 msgr2=0x7fac0c1451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac111e1640 1 --2- 172.21.15.82:0/837007828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c1451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac0b7fe640 1 -- 172.21.15.82:0/837007828 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356356822 0 0) 0x7fac0c0ce620 con 0x7fac0c0cdd60 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac109e0640 1 --2- 172.21.15.82:0/837007828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c1451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac109e0640 1 -- 172.21.15.82:0/837007828 >> v1:172.21.15.82:6789/0 conn(0x7fac0c0cdd60 legacy=0x7fac0c144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac109e0640 1 -- 172.21.15.82:0/837007828 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c0ce8b0 con 0x7fac0c15f7a0 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.112+0000 7fac111e1640 1 --2- 172.21.15.82:0/837007828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c1451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:28.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.113+0000 7fac109e0640 1 --2- 172.21.15.82:0/837007828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c15f7a0 0x7fac0c15fb80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a44c6fdc9b6e29d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.113+0000 7fac0b7fe640 1 -- 172.21.15.82:0/837007828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac000315a0 con 0x7fac0c15f7a0 2024-03-14T12:18:28.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.113+0000 7fac0b7fe640 1 -- 172.21.15.82:0/837007828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00037070 con 0x7fac0c15f7a0 2024-03-14T12:18:28.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.113+0000 7fac0b7fe640 1 -- 172.21.15.82:0/837007828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00031960 con 0x7fac0c15f7a0 2024-03-14T12:18:28.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 -- 172.21.15.82:0/837007828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c15f7a0 msgr2=0x7fac0c15fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 --2- 172.21.15.82:0/837007828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c15f7a0 0x7fac0c15fb80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac0002b840 comp rx=0 tx=0).stop 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 -- 172.21.15.82:0/837007828 shutdown_connections 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 --2- 172.21.15.82:0/837007828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c1451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 --2- 172.21.15.82:0/837007828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c15f7a0 0x7fac0c15fb80 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 -- 172.21.15.82:0/837007828 >> 172.21.15.82:0/837007828 conn(0x7fac0c0b1c70 msgr2=0x7fac0c0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 -- 172.21.15.82:0/837007828 shutdown_connections 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.114+0000 7fac12c44640 1 -- 172.21.15.82:0/837007828 wait complete. 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.115+0000 7fac12c44640 1 Processor -- start 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.115+0000 7fac12c44640 1 -- start start 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c0cdd60 0x7fac0c13f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c15f7a0 0x7fac0c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c06c4e0 con 0x7fac0c0cdd60 2024-03-14T12:18:28.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c0cee70 con 0x7fac0c15f7a0 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c0cc4f0 con 0x7fac0c0ce210 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c0cdd60 0x7fac0c13f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c0cdd60 0x7fac0c13f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55824/0 (socket says 172.21.15.82:55824) 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 -- 172.21.15.82:0/3769385828 learned_addr learned my addr 172.21.15.82:0/3769385828 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac111e1640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c15f7a0 0x7fac0c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac0bfff640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 -- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 msgr2=0x7fac0c154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 -- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c15f7a0 msgr2=0x7fac0c154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c15f7a0 0x7fac0c154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 -- 172.21.15.82:0/3769385828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c1e5a90 con 0x7fac0c0cdd60 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac109e0640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c0cdd60 0x7fac0c13f2e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac097fa640 1 -- 172.21.15.82:0/3769385828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00004230 con 0x7fac0c0cdd60 2024-03-14T12:18:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac097fa640 1 -- 172.21.15.82:0/3769385828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac000043f0 con 0x7fac0c0cdd60 2024-03-14T12:18:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0c1e5d70 con 0x7fac0c0cdd60 2024-03-14T12:18:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac097fa640 1 -- 172.21.15.82:0/3769385828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00032880 con 0x7fac0c0cdd60 2024-03-14T12:18:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.116+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c1e6330 con 0x7fac0c0cdd60 2024-03-14T12:18:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.117+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac0c04f050 con 0x7fac0c0cdd60 2024-03-14T12:18:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.118+0000 7fac097fa640 1 -- 172.21.15.82:0/3769385828 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fac00032a20 con 0x7fac0c0cdd60 2024-03-14T12:18:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.118+0000 7fac097fa640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fabf006bbd0 0x7fabf006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.118+0000 7fac097fa640 1 -- 172.21.15.82:0/3769385828 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fac0002f030 con 0x7fac0c0cdd60 2024-03-14T12:18:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.118+0000 7fac0bfff640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fabf006bbd0 0x7fabf006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.119+0000 7fac0bfff640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fabf006bbd0 0x7fabf006e090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fac0c1e59c0 tx=0x7fabf8007c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.123+0000 7fac097fa640 1 -- 172.21.15.82:0/3769385828 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fac00041070 con 0x7fac0c0cdd60 2024-03-14T12:18:28.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.293+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fac0c159dd0 con 0x7fabf006bbd0 2024-03-14T12:18:28.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.294+0000 7fac097fa640 1 -- 172.21.15.82:0/3769385828 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fac0c159dd0 con 0x7fabf006bbd0 2024-03-14T12:18:28.294 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 3 to deep-scrub 2024-03-14T12:18:28.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fabf006bbd0 msgr2=0x7fabf006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fabf006bbd0 0x7fabf006e090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fac0c1e59c0 tx=0x7fabf8007c00 comp rx=0 tx=0).stop 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c0cdd60 msgr2=0x7fac0c13f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c0cdd60 0x7fac0c13f2e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac000046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 shutdown_connections 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c15f7a0 0x7fac0c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fabf006bbd0 0x7fabf006e090 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fac0c0ce210 0x7fac0c154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 --2- 172.21.15.82:0/3769385828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c0cdd60 0x7fac0c13f2e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 >> 172.21.15.82:0/3769385828 conn(0x7fac0c0b1c70 msgr2=0x7fac0c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 shutdown_connections 2024-03-14T12:18:28.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.296+0000 7fac12c44640 1 -- 172.21.15.82:0/3769385828 wait complete. 2024-03-14T12:18:28.307 INFO:tasks.ceph:Scrubbing osd.4 2024-03-14T12:18:28.307 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:28.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.410+0000 7f4820197640 1 Processor -- start 2024-03-14T12:18:28.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.410+0000 7f4820197640 1 -- start start 2024-03-14T12:18:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f4820197640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4818159c90 0x7f481815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f4820197640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48180cfd10 0x7f48180d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f4820197640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180b3540 con 0x7f48180b4a30 2024-03-14T12:18:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f4820197640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180ce050 con 0x7f48180cfd10 2024-03-14T12:18:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f4820197640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180d06a0 con 0x7f4818159c90 2024-03-14T12:18:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481e734640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48180cfd10 0x7f48180d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481d732640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f48180b4a30 0x7f48180b2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42856/0 (socket says 172.21.15.82:42856) 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481d732640 1 -- 172.21.15.82:0/807647798 learned_addr learned my addr 172.21.15.82:0/807647798 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481df33640 1 --2- 172.21.15.82:0/807647798 >> v2:172.21.15.164:3300/0 conn(0x7f4818159c90 0x7f481815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481e734640 1 -- 172.21.15.82:0/807647798 >> v2:172.21.15.164:3300/0 conn(0x7f4818159c90 msgr2=0x7f481815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481e734640 1 --2- 172.21.15.82:0/807647798 >> v2:172.21.15.164:3300/0 conn(0x7f4818159c90 0x7f481815cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481e734640 1 -- 172.21.15.82:0/807647798 >> v1:172.21.15.82:6789/0 conn(0x7f48180b4a30 legacy=0x7f48180b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.411+0000 7f481e734640 1 -- 172.21.15.82:0/807647798 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481815d460 con 0x7f48180cfd10 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f481e734640 1 --2- 172.21.15.82:0/807647798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f48180d0160 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f48180722a0 tx=0x7f481002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293a3767b52ea801 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f481cf31640 1 -- 172.21.15.82:0/807647798 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810039070 con 0x7f48180cfd10 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f481cf31640 1 -- 172.21.15.82:0/807647798 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481002cc80 con 0x7f48180cfd10 2024-03-14T12:18:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f481cf31640 1 -- 172.21.15.82:0/807647798 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810034660 con 0x7f48180cfd10 2024-03-14T12:18:28.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f4820197640 1 -- 172.21.15.82:0/807647798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 msgr2=0x7f48180d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f4820197640 1 --2- 172.21.15.82:0/807647798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f48180d0160 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f48180722a0 tx=0x7f481002bed0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f4820197640 1 -- 172.21.15.82:0/807647798 shutdown_connections 2024-03-14T12:18:28.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f4820197640 1 --2- 172.21.15.82:0/807647798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f48180d0160 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f4820197640 1 --2- 172.21.15.82:0/807647798 >> v2:172.21.15.164:3300/0 conn(0x7f4818159c90 0x7f481815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.412+0000 7f4820197640 1 -- 172.21.15.82:0/807647798 >> 172.21.15.82:0/807647798 conn(0x7f48180b19b0 msgr2=0x7f48180b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.413+0000 7f4820197640 1 -- 172.21.15.82:0/807647798 shutdown_connections 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.413+0000 7f4820197640 1 -- 172.21.15.82:0/807647798 wait complete. 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 Processor -- start 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 -- start start 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48180b4a30 0x7f48181f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f4818151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4818159c90 0x7f481814d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4818072690 con 0x7f48180cfd10 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4818072970 con 0x7f48180b4a30 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f4820197640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180ce280 con 0x7f4818159c90 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f481d732640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f4818151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f481d732640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f4818151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55840/0 (socket says 172.21.15.82:55840) 2024-03-14T12:18:28.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.414+0000 7f481d732640 1 -- 172.21.15.82:0/1468321268 learned_addr learned my addr 172.21.15.82:0/1468321268 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f481d732640 1 -- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4818159c90 msgr2=0x7f481814d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f481d732640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4818159c90 0x7f481814d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f481d732640 1 -- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48180b4a30 msgr2=0x7f48181f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f481df33640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48180b4a30 0x7f48181f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f481d732640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48180b4a30 0x7f48181f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f481d732640 1 -- 172.21.15.82:0/1468321268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818151780 con 0x7f48180cfd10 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f481d732640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f4818151f90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f480800c440 tx=0x7f480800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808017070 con 0x7f48180cfd10 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808005ce0 con 0x7f48180cfd10 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818151a60 con 0x7f48180cfd10 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080072f0 con 0x7f48180cfd10 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.415+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4818158930 con 0x7f48180cfd10 2024-03-14T12:18:28.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.416+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48181526e0 con 0x7f48180cfd10 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.416+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f480801a070 con 0x7f48180cfd10 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f480e7fc640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f47fc06bbd0 0x7f47fc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f480808a2a0 con 0x7f48180cfd10 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f481df33640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f47fc06bbd0 0x7f47fc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f480e7fc640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f47fc071bd0 0x7f47fc073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 --> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47fc0746a0 con 0x7f47fc071bd0 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f480808a730 con 0x7f48180cfd10 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f481df33640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f47fc06bbd0 0x7f47fc06e090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4814006fd0 tx=0x7f4814008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.417+0000 7f481e734640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f47fc071bd0 0x7f47fc073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.418+0000 7f481e734640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f47fc071bd0 0x7f47fc073ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.419+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== osd.4 v2:172.21.15.164:6800/2779428821 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f47fc0746a0 con 0x7f47fc071bd0 2024-03-14T12:18:28.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.431+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 --> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f48180ae720 con 0x7f47fc071bd0 2024-03-14T12:18:28.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f480e7fc640 1 -- 172.21.15.82:0/1468321268 <== osd.4 v2:172.21.15.164:6800/2779428821 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f48180ae720 con 0x7f47fc071bd0 2024-03-14T12:18:28.432 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:28.432 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:28.432 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f47fc071bd0 msgr2=0x7f47fc073ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.164:6800/2779428821,v1:172.21.15.164:6801/2779428821] conn(0x7f47fc071bd0 0x7f47fc073ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f47fc06bbd0 msgr2=0x7f47fc06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f47fc06bbd0 0x7f47fc06e090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4814006fd0 tx=0x7f4814008040 comp rx=0 tx=0).stop 2024-03-14T12:18:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 msgr2=0x7f4818151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 --2- 172.21.15.82:0/1468321268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48180cfd10 0x7f4818151f90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f480800c440 tx=0x7f480800c900 comp rx=0 tx=0).stop 2024-03-14T12:18:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f481e734640 1 -- 172.21.15.82:0/1468321268 reap_dead start 2024-03-14T12:18:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 shutdown_connections 2024-03-14T12:18:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 >> 172.21.15.82:0/1468321268 conn(0x7f48180b19b0 msgr2=0x7f481806af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 shutdown_connections 2024-03-14T12:18:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.432+0000 7f4820197640 1 -- 172.21.15.82:0/1468321268 wait complete. 2024-03-14T12:18:28.443 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-14T12:18:28.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.544+0000 7f183c7b2640 1 Processor -- start 2024-03-14T12:18:28.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.544+0000 7f183c7b2640 1 -- start start 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183c7b2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1834071740 0x7f1834071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183c7b2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1834143f60 0x7f183415b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183c7b2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340b3050 con 0x7f183415e670 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183c7b2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1834072060 con 0x7f1834071740 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183c7b2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183415ebe0 con 0x7f1834143f60 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183a54e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1834071740 0x7f1834071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183a54e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1834071740 0x7f1834071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55850/0 (socket says 172.21.15.82:55850) 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.545+0000 7f183a54e640 1 -- 172.21.15.82:0/3721715297 learned_addr learned my addr 172.21.15.82:0/3721715297 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:28.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.546+0000 7f183a54e640 1 -- 172.21.15.82:0/3721715297 >> v2:172.21.15.164:3300/0 conn(0x7f1834143f60 msgr2=0x7f183415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.546+0000 7f183a54e640 1 --2- 172.21.15.82:0/3721715297 >> v2:172.21.15.164:3300/0 conn(0x7f1834143f60 0x7f183415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.546+0000 7f183a54e640 1 -- 172.21.15.82:0/3721715297 >> v1:172.21.15.164:6789/0 conn(0x7f183415e670 legacy=0x7f18340b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.546+0000 7f183a54e640 1 -- 172.21.15.82:0/3721715297 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183415eec0 con 0x7f1834071740 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.546+0000 7f183a54e640 1 --2- 172.21.15.82:0/3721715297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834071740 0x7f1834071b20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f183415f790 tx=0x7f182c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a7f7ba2c5389ab server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.546+0000 7f183954c640 1 -- 172.21.15.82:0/3721715297 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c035020 con 0x7f1834071740 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183954c640 1 -- 172.21.15.82:0/3721715297 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c031870 con 0x7f1834071740 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183954c640 1 -- 172.21.15.82:0/3721715297 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c031c40 con 0x7f1834071740 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 -- 172.21.15.82:0/3721715297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834071740 msgr2=0x7f1834071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 --2- 172.21.15.82:0/3721715297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834071740 0x7f1834071b20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f183415f790 tx=0x7f182c02b910 comp rx=0 tx=0).stop 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 -- 172.21.15.82:0/3721715297 shutdown_connections 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 --2- 172.21.15.82:0/3721715297 >> v2:172.21.15.164:3300/0 conn(0x7f1834143f60 0x7f183415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 --2- 172.21.15.82:0/3721715297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834071740 0x7f1834071b20 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 -- 172.21.15.82:0/3721715297 >> 172.21.15.82:0/3721715297 conn(0x7f18340b1c70 msgr2=0x7f18340cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 -- 172.21.15.82:0/3721715297 shutdown_connections 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.547+0000 7f183c7b2640 1 -- 172.21.15.82:0/3721715297 wait complete. 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.548+0000 7f183c7b2640 1 Processor -- start 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.548+0000 7f183c7b2640 1 -- start start 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183c7b2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1834071740 0x7f18341f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183c7b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834143f60 0x7f1834157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183c7b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183415e670 0x7f18340cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183c7b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834143ce0 con 0x7f1834143f60 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183c7b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183415f480 con 0x7f183415e670 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183c7b2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183415b080 con 0x7f1834071740 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834143f60 0x7f1834157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834143f60 0x7f1834157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55854/0 (socket says 172.21.15.82:55854) 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 -- 172.21.15.82:0/2282831399 learned_addr learned my addr 172.21.15.82:0/2282831399 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183a54e640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1834071740 0x7f18341f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f183ad4f640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183415e670 0x7f18340cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 -- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1834071740 msgr2=0x7f18341f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1834071740 0x7f18341f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 -- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183415e670 msgr2=0x7f18340cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183415e670 0x7f18340cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.549+0000 7f1839d4d640 1 -- 172.21.15.82:0/2282831399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18341f3180 con 0x7f1834143f60 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.550+0000 7f1839d4d640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834143f60 0x7f1834157fb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f182400c410 tx=0x7f182400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.550+0000 7f182affd640 1 -- 172.21.15.82:0/2282831399 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824017070 con 0x7f1834143f60 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.550+0000 7f182affd640 1 -- 172.21.15.82:0/2282831399 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240035f0 con 0x7f1834143f60 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.550+0000 7f182affd640 1 -- 172.21.15.82:0/2282831399 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824003a20 con 0x7f1834143f60 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.550+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183415bbb0 con 0x7f1834143f60 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.550+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183415c170 con 0x7f1834143f60 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.551+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183404f050 con 0x7f1834143f60 2024-03-14T12:18:28.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.551+0000 7f182affd640 1 -- 172.21.15.82:0/2282831399 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f182401a070 con 0x7f1834143f60 2024-03-14T12:18:28.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.551+0000 7f182affd640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f181806bbd0 0x7f181806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.551+0000 7f182affd640 1 -- 172.21.15.82:0/2282831399 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1824089bf0 con 0x7f1834143f60 2024-03-14T12:18:28.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.551+0000 7f183a54e640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f181806bbd0 0x7f181806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.552+0000 7f183a54e640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f181806bbd0 0x7f181806e090 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f182c036000 tx=0x7f182c03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.556+0000 7f182affd640 1 -- 172.21.15.82:0/2282831399 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f18240010c0 con 0x7f1834143f60 2024-03-14T12:18:28.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.726+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f18340d07b0 con 0x7f181806bbd0 2024-03-14T12:18:28.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.727+0000 7f182affd640 1 -- 172.21.15.82:0/2282831399 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f18340d07b0 con 0x7f181806bbd0 2024-03-14T12:18:28.727 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 4 to deep-scrub 2024-03-14T12:18:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.728+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f181806bbd0 msgr2=0x7f181806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.728+0000 7f183c7b2640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f181806bbd0 0x7f181806e090 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f182c036000 tx=0x7f182c03c040 comp rx=0 tx=0).stop 2024-03-14T12:18:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.728+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834143f60 msgr2=0x7f1834157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834143f60 0x7f1834157fb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f182400c410 tx=0x7f182400c8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 shutdown_connections 2024-03-14T12:18:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f183415e670 0x7f18340cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1834143f60 0x7f1834157fb0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f181806bbd0 0x7f181806e090 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 --2- 172.21.15.82:0/2282831399 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1834071740 0x7f18341f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 >> 172.21.15.82:0/2282831399 conn(0x7f18340b1c70 msgr2=0x7f18340b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 shutdown_connections 2024-03-14T12:18:28.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.729+0000 7f183c7b2640 1 -- 172.21.15.82:0/2282831399 wait complete. 2024-03-14T12:18:28.740 INFO:tasks.ceph:Scrubbing osd.5 2024-03-14T12:18:28.740 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.843+0000 7f4c0ed48640 1 Processor -- start 2024-03-14T12:18:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c0ed48640 1 -- start start 2024-03-14T12:18:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c0ed48640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4c080d06b0 0x7f4c0806d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c0ed48640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0806bec0 con 0x7f4c080cda50 2024-03-14T12:18:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c0ed48640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0806c1a0 con 0x7f4c080ce5e0 2024-03-14T12:18:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c0ed48640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0806c7f0 con 0x7f4c080d06b0 2024-03-14T12:18:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c0d2e5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4c080d06b0 0x7f4c0806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c07fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4c080cda50 0x7f4c0806cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42872/0 (socket says 172.21.15.82:42872) 2024-03-14T12:18:28.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.844+0000 7f4c07fff640 1 -- 172.21.15.82:0/3484635078 learned_addr learned my addr 172.21.15.82:0/3484635078 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:28.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.845+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3484635078 >> v1:172.21.15.164:6789/0 conn(0x7f4c080ce5e0 legacy=0x7f4c080ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:28.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.845+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3484635078 >> v1:172.21.15.82:6789/0 conn(0x7f4c080cda50 legacy=0x7f4c0806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.845+0000 7f4c077fe640 1 -- 172.21.15.82:0/3484635078 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084988131 0 0) 0x7f4c0806bec0 con 0x7f4c080cda50 2024-03-14T12:18:28.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.845+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3484635078 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0806c480 con 0x7f4c080d06b0 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.845+0000 7f4c0d2e5640 1 --2- 172.21.15.82:0/3484635078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c0806d390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009d90 tx=0x7f4c0002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6aa8bba12e6f20 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c077fe640 1 -- 172.21.15.82:0/3484635078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00039070 con 0x7f4c080d06b0 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c077fe640 1 -- 172.21.15.82:0/3484635078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0002cc60 con 0x7f4c080d06b0 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c077fe640 1 -- 172.21.15.82:0/3484635078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00034660 con 0x7f4c080d06b0 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3484635078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 msgr2=0x7f4c0806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c0ed48640 1 --2- 172.21.15.82:0/3484635078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c0806d390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009d90 tx=0x7f4c0002beb0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3484635078 shutdown_connections 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c0ed48640 1 --2- 172.21.15.82:0/3484635078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c0806d390 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3484635078 >> 172.21.15.82:0/3484635078 conn(0x7f4c080b3780 msgr2=0x7f4c080b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3484635078 shutdown_connections 2024-03-14T12:18:28.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.846+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3484635078 wait complete. 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.847+0000 7f4c0ed48640 1 Processor -- start 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.847+0000 7f4c0ed48640 1 -- start start 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0ed48640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c080cda50 0x7f4c080c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0ed48640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c080ce5e0 0x7f4c0815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0ed48640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c080c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0ed48640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080d0d60 con 0x7f4c080d06b0 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0ed48640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080d0be0 con 0x7f4c080cda50 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0ed48640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0815e370 con 0x7f4c080ce5e0 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0d2e5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c080c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0d2e5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c080c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55868/0 (socket says 172.21.15.82:55868) 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.848+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3768016874 learned_addr learned my addr 172.21.15.82:0/3768016874 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0cae4640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c080cda50 0x7f4c080c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c07fff640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c080ce5e0 0x7f4c0815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c080ce5e0 msgr2=0x7f4c0815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0d2e5640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c080ce5e0 0x7f4c0815a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c080cda50 msgr2=0x7f4c080c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0d2e5640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c080cda50 0x7f4c080c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3768016874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0815a7e0 con 0x7f4c080d06b0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0d2e5640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c080c6de0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009eb0 tx=0x7f4c00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00044020 con 0x7f4c080d06b0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0003f470 con 0x7f4c080d06b0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00039040 con 0x7f4c080d06b0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c080c7320 con 0x7f4c080d06b0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.849+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c080c6020 con 0x7f4c080d06b0 2024-03-14T12:18:28.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.850+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c0804f050 con 0x7f4c080d06b0 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4c00047020 con 0x7f4c080d06b0 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c057fa640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4bf406bbd0 0x7f4bf406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4c000ada00 con 0x7f4c080d06b0 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c0cae4640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4bf406bbd0 0x7f4bf406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c057fa640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7f4bf4071bd0 0x7f4bf4073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 --> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c0002cc60 con 0x7f4bf4071bd0 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c000ade90 con 0x7f4c080d06b0 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.851+0000 7f4c0cae4640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4bf406bbd0 0x7f4bf406e090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80015d0 tx=0x7f4bf8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.852+0000 7f4c07fff640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7f4bf4071bd0 0x7f4bf4073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.852+0000 7f4c07fff640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7f4bf4071bd0 0x7f4bf4073ff0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.853+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== osd.5 v2:172.21.15.164:6817/2718563770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4c0002cc60 con 0x7f4bf4071bd0 2024-03-14T12:18:28.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.866+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 --> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4c080b74b0 con 0x7f4bf4071bd0 2024-03-14T12:18:28.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.866+0000 7f4c057fa640 1 -- 172.21.15.82:0/3768016874 <== osd.5 v2:172.21.15.164:6817/2718563770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4c080b74b0 con 0x7f4bf4071bd0 2024-03-14T12:18:28.867 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:28.867 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:28.867 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7f4bf4071bd0 msgr2=0x7f4bf4073ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.164:6817/2718563770,v1:172.21.15.164:6819/2718563770] conn(0x7f4bf4071bd0 0x7f4bf4073ff0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4bf406bbd0 msgr2=0x7f4bf406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4bf406bbd0 0x7f4bf406e090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80015d0 tx=0x7f4bf8009040 comp rx=0 tx=0).stop 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 msgr2=0x7f4c080c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 --2- 172.21.15.82:0/3768016874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080d06b0 0x7f4c080c6de0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009eb0 tx=0x7f4c00004a40 comp rx=0 tx=0).stop 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0d2e5640 1 -- 172.21.15.82:0/3768016874 reap_dead start 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 shutdown_connections 2024-03-14T12:18:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 >> 172.21.15.82:0/3768016874 conn(0x7f4c080b3780 msgr2=0x7f4c0806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 shutdown_connections 2024-03-14T12:18:28.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.867+0000 7f4c0ed48640 1 -- 172.21.15.82:0/3768016874 wait complete. 2024-03-14T12:18:28.878 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-14T12:18:28.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.978+0000 7f4300759640 1 Processor -- start 2024-03-14T12:18:28.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.978+0000 7f4300759640 1 -- start start 2024-03-14T12:18:28.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.978+0000 7f4300759640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42f80b5710 0x7f42f815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f4300759640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f42f80b5af0 0x7f42f815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f4300759640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f815dcd0 0x7f42f806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f4300759640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f815e150 con 0x7f42f80b5af0 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f4300759640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f815e430 con 0x7f42f80b5710 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f4300759640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f815e710 con 0x7f42f815dcd0 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f42f80b5af0 0x7f42f815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f42f80b5af0 0x7f42f815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55880/0 (socket says 172.21.15.82:55880) 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 -- 172.21.15.82:0/168074055 learned_addr learned my addr 172.21.15.82:0/168074055 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fecf6640 1 --2- 172.21.15.82:0/168074055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f815dcd0 0x7f42f806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fe4f5640 1 --2- 172.21.15.82:0/168074055 >> v2:172.21.15.164:3300/0 conn(0x7f42f80b5710 0x7f42f815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 -- 172.21.15.82:0/168074055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f815dcd0 msgr2=0x7f42f806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 --2- 172.21.15.82:0/168074055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f815dcd0 0x7f42f806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 -- 172.21.15.82:0/168074055 >> v2:172.21.15.164:3300/0 conn(0x7f42f80b5710 msgr2=0x7f42f815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:28.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 --2- 172.21.15.82:0/168074055 >> v2:172.21.15.164:3300/0 conn(0x7f42f80b5710 0x7f42f815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.979+0000 7f42fdcf4640 1 -- 172.21.15.82:0/168074055 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f815e9f0 con 0x7f42f80b5af0 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f42fdcf4640 1 --2- 172.21.15.82:0/168074055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f80b5af0 0x7f42f815d790 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f42f80b48b0 tx=0x7f42e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eba8a53b6df15c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f42fd4f3640 1 -- 172.21.15.82:0/168074055 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8035020 con 0x7f42f80b5af0 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f42fd4f3640 1 -- 172.21.15.82:0/168074055 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8031870 con 0x7f42f80b5af0 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f42fd4f3640 1 -- 172.21.15.82:0/168074055 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8031c40 con 0x7f42f80b5af0 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f4300759640 1 -- 172.21.15.82:0/168074055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f80b5af0 msgr2=0x7f42f815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f4300759640 1 --2- 172.21.15.82:0/168074055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f80b5af0 0x7f42f815d790 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f42f80b48b0 tx=0x7f42e802b910 comp rx=0 tx=0).stop 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f4300759640 1 -- 172.21.15.82:0/168074055 shutdown_connections 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f4300759640 1 --2- 172.21.15.82:0/168074055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f815dcd0 0x7f42f806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f4300759640 1 --2- 172.21.15.82:0/168074055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f80b5af0 0x7f42f815d790 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f4300759640 1 --2- 172.21.15.82:0/168074055 >> v2:172.21.15.164:3300/0 conn(0x7f42f80b5710 0x7f42f815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.980+0000 7f4300759640 1 -- 172.21.15.82:0/168074055 >> 172.21.15.82:0/168074055 conn(0x7f42f8072ee0 msgr2=0x7f42f809a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:28.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.981+0000 7f4300759640 1 -- 172.21.15.82:0/168074055 shutdown_connections 2024-03-14T12:18:28.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.981+0000 7f4300759640 1 -- 172.21.15.82:0/168074055 wait complete. 2024-03-14T12:18:28.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 Processor -- start 2024-03-14T12:18:28.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 -- start start 2024-03-14T12:18:28.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f80b5710 0x7f42f80d2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f80b5af0 0x7f42f8150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f815dcd0 0x7f42f8152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8156650 con 0x7f42f815dcd0 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b5f80 con 0x7f42f80b5710 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f4300759640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f814d870 con 0x7f42f80b5af0 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f42fecf6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f815dcd0 0x7f42f8152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f42fecf6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f815dcd0 0x7f42f8152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55894/0 (socket says 172.21.15.82:55894) 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.982+0000 7f42fecf6640 1 -- 172.21.15.82:0/3656894342 learned_addr learned my addr 172.21.15.82:0/3656894342 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42fecf6640 1 -- 172.21.15.82:0/3656894342 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f80b5af0 msgr2=0x7f42f8150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42fecf6640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f80b5af0 0x7f42f8150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42fecf6640 1 -- 172.21.15.82:0/3656894342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f80b5710 msgr2=0x7f42f80d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42fecf6640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f80b5710 0x7f42f80d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42fecf6640 1 -- 172.21.15.82:0/3656894342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f814daa0 con 0x7f42f815dcd0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42fecf6640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f815dcd0 0x7f42f8152880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f42f0007e40 tx=0x7f42f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42eeffd640 1 -- 172.21.15.82:0/3656894342 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0014620 con 0x7f42f815dcd0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42eeffd640 1 -- 172.21.15.82:0/3656894342 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0019070 con 0x7f42f815dcd0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f8151980 con 0x7f42f815dcd0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f42eeffd640 1 -- 172.21.15.82:0/3656894342 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0014a20 con 0x7f42f815dcd0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.983+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f81e5d20 con 0x7f42f815dcd0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.984+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f815bbe0 con 0x7f42f815dcd0 2024-03-14T12:18:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.984+0000 7f42eeffd640 1 -- 172.21.15.82:0/3656894342 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f42f0014bc0 con 0x7f42f815dcd0 2024-03-14T12:18:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.985+0000 7f42eeffd640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f42dc06bc00 0x7f42dc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.985+0000 7f42eeffd640 1 -- 172.21.15.82:0/3656894342 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f42f008b160 con 0x7f42f815dcd0 2024-03-14T12:18:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.985+0000 7f42fe4f5640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f42dc06bc00 0x7f42dc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.985+0000 7f42fe4f5640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f42dc06bc00 0x7f42dc06e0c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f42f400b7f0 tx=0x7f42f4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:28.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:28.989+0000 7f42eeffd640 1 -- 172.21.15.82:0/3656894342 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f42f815bbe0 con 0x7f42f815dcd0 2024-03-14T12:18:29.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.158+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f42f80b43a0 con 0x7f42dc06bc00 2024-03-14T12:18:29.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.159+0000 7f42eeffd640 1 -- 172.21.15.82:0/3656894342 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f42f80b43a0 con 0x7f42dc06bc00 2024-03-14T12:18:29.159 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 5 to deep-scrub 2024-03-14T12:18:29.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f42dc06bc00 msgr2=0x7f42dc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f42dc06bc00 0x7f42dc06e0c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f42f400b7f0 tx=0x7f42f4008040 comp rx=0 tx=0).stop 2024-03-14T12:18:29.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f815dcd0 msgr2=0x7f42f8152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f815dcd0 0x7f42f8152880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f42f0007e40 tx=0x7f42f000e4b0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 shutdown_connections 2024-03-14T12:18:29.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f815dcd0 0x7f42f8152880 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f42dc06bc00 0x7f42dc06e0c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42f80b5af0 0x7f42f8150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 --2- 172.21.15.82:0/3656894342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42f80b5710 0x7f42f80d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 >> 172.21.15.82:0/3656894342 conn(0x7f42f8072ee0 msgr2=0x7f42f80af160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.160+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 shutdown_connections 2024-03-14T12:18:29.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.161+0000 7f4300759640 1 -- 172.21.15.82:0/3656894342 wait complete. 2024-03-14T12:18:29.171 INFO:tasks.ceph:Scrubbing osd.6 2024-03-14T12:18:29.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:29.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.275+0000 7f6a0ce8c640 1 Processor -- start 2024-03-14T12:18:29.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.275+0000 7f6a0ce8c640 1 -- start start 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a0ce8c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 0x7f6a080cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a0ce8c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a080ade30 con 0x7f6a080ad650 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a0ce8c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a080ae110 con 0x7f6a080cfb90 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a0ce8c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a080ae3f0 con 0x7f6a0815bf40 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a06d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a080ad650 0x7f6a0806d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42878/0 (socket says 172.21.15.82:42878) 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a06d9d640 1 -- 172.21.15.82:0/1030380383 learned_addr learned my addr 172.21.15.82:0/1030380383 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515922323 0 0) 0x7f6a080ade30 con 0x7f6a080ad650 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a0659c640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 0x7f6a080cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69f4003520 con 0x7f6a080ad650 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884810395 0 0) 0x7f69f4003520 con 0x7f6a080ad650 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 msgr2=0x7f6a080cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 0x7f6a080cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 >> v1:172.21.15.164:6789/0 conn(0x7f6a080cfb90 legacy=0x7f6a080acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0806d100 con 0x7f6a080ad650 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6a000019f0 con 0x7f6a080ad650 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 >> v1:172.21.15.82:6789/0 conn(0x7f6a080ad650 legacy=0x7f6a0806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 0x7f69f40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f69f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69f4004d40 0x7f69f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a080ae3f0 con 0x7f69f4003f60 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f400baa0 con 0x7f69f4004d40 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.276+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f400bca0 con 0x7f6a0815bf40 2024-03-14T12:18:29.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f69fffff640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f69f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f6a0659c640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 0x7f69f40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f6a06d9d640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69f4004d40 0x7f69f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f69fffff640 1 -- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 msgr2=0x7f69f40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f69fffff640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0815bf40 0x7f69f40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f69fffff640 1 -- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69f4004d40 msgr2=0x7f69f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f69fffff640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69f4004d40 0x7f69f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f69fffff640 1 -- 172.21.15.82:0/1030380383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4003520 con 0x7f69f4003f60 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f6a06d9d640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69f4004d40 0x7f69f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.277+0000 7f69fffff640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f69f4004800 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f69f0001750 tx=0x7f69f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c43daffa4af0f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0001370 con 0x7f69f4003f60 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f00052d0 con 0x7f69f4003f60 2024-03-14T12:18:29.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a05d9b640 1 -- 172.21.15.82:0/1030380383 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0003830 con 0x7f69f4003f60 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1030380383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 msgr2=0x7f69f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a0ce8c640 1 --2- 172.21.15.82:0/1030380383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f69f4004800 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f69f0001750 tx=0x7f69f4005830 comp rx=0 tx=0).stop 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a06d9d640 1 -- 172.21.15.82:0/1030380383 reap_dead start 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1030380383 shutdown_connections 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1030380383 >> 172.21.15.82:0/1030380383 conn(0x7f6a080d0110 msgr2=0x7f6a080d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1030380383 shutdown_connections 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.278+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1030380383 wait complete. 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.279+0000 7f6a0ce8c640 1 Processor -- start 2024-03-14T12:18:29.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0ce8c640 1 -- start start 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0ce8c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f6a08156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0ce8c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0809bea0 0x7f6a080b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0ce8c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a0809b140 0x7f6a0809b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0ce8c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0806ae90 con 0x7f69f4003f60 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0ce8c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0806ad10 con 0x7f6a0809b140 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0ce8c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0815c320 con 0x7f6a0809bea0 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f6a08156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f6a08156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55906/0 (socket says 172.21.15.82:55906) 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a0659c640 1 -- 172.21.15.82:0/1555000467 learned_addr learned my addr 172.21.15.82:0/1555000467 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:29.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f69fffff640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0809bea0 0x7f6a080b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.280+0000 7f6a06d9d640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a0809b140 0x7f6a0809b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0659c640 1 -- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0809bea0 msgr2=0x7f6a080b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0659c640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0809bea0 0x7f6a080b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0659c640 1 -- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a0809b140 msgr2=0x7f6a0809b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0659c640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a0809b140 0x7f6a0809b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0659c640 1 -- 172.21.15.82:0/1555000467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a081574e0 con 0x7f69f4003f60 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f69fffff640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6a0809bea0 0x7f6a080b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a06d9d640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a0809b140 0x7f6a0809b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0659c640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f6a08156fa0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f69f80119a0 tx=0x7f69f8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800faf0 con 0x7f69f4003f60 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f800fc90 con 0x7f69f4003f60 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8014440 con 0x7f69f4003f60 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a080cc140 con 0x7f69f4003f60 2024-03-14T12:18:29.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.281+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a080cb660 con 0x7f69f4003f60 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.282+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a080ce770 con 0x7f69f4003f60 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.282+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f69f8012040 con 0x7f69f4003f60 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f69feffd640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69e806bbd0 0x7f69e806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f69f808aeb0 con 0x7f69f4003f60 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f69fffff640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69e806bbd0 0x7f69e806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f69feffd640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f69e8071bd0 0x7f69e8073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 --> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69f801fe50 con 0x7f69e8071bd0 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69f808d5b0 con 0x7f69f4003f60 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f6a06d9d640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f69e8071bd0 0x7f69e8073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f69fffff640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69e806bbd0 0x7f69e806e090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f69f0033000 tx=0x7f69f0009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.283+0000 7f6a06d9d640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f69e8071bd0 0x7f69e8073ff0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.284+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== osd.6 v2:172.21.15.164:6808/3403808739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f69f801fe50 con 0x7f69e8071bd0 2024-03-14T12:18:29.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.297+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 --> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6a0806f3c0 con 0x7f69e8071bd0 2024-03-14T12:18:29.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f69feffd640 1 -- 172.21.15.82:0/1555000467 <== osd.6 v2:172.21.15.164:6808/3403808739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6a0806f3c0 con 0x7f69e8071bd0 2024-03-14T12:18:29.298 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:29.298 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:29.298 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:29.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f69e8071bd0 msgr2=0x7f69e8073ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.164:6808/3403808739,v1:172.21.15.164:6809/3403808739] conn(0x7f69e8071bd0 0x7f69e8073ff0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69e806bbd0 msgr2=0x7f69e806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f69e806bbd0 0x7f69e806e090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f69f0033000 tx=0x7f69f0009030 comp rx=0 tx=0).stop 2024-03-14T12:18:29.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 msgr2=0x7f6a08156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 --2- 172.21.15.82:0/1555000467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f4003f60 0x7f6a08156fa0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f69f80119a0 tx=0x7f69f8011e60 comp rx=0 tx=0).stop 2024-03-14T12:18:29.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a06d9d640 1 -- 172.21.15.82:0/1555000467 reap_dead start 2024-03-14T12:18:29.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 shutdown_connections 2024-03-14T12:18:29.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 >> 172.21.15.82:0/1555000467 conn(0x7f6a080d0110 msgr2=0x7f6a0806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 shutdown_connections 2024-03-14T12:18:29.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.298+0000 7f6a0ce8c640 1 -- 172.21.15.82:0/1555000467 wait complete. 2024-03-14T12:18:29.309 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-14T12:18:29.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.410+0000 7f4dad659640 1 Processor -- start 2024-03-14T12:18:29.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.410+0000 7f4dad659640 1 -- start start 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.410+0000 7f4dad659640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 0x7f4da80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.410+0000 7f4dad659640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80cfcf0 con 0x7f4da80d2390 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.410+0000 7f4dad659640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80d4a50 con 0x7f4da80d0e80 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.410+0000 7f4dad659640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80d4d30 con 0x7f4da80d1b90 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.410+0000 7f4da6ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4da80d2390 0x7f4da80d2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42884/0 (socket says 172.21.15.82:42884) 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da6ffd640 1 -- 172.21.15.82:0/1829856326 learned_addr learned my addr 172.21.15.82:0/1829856326 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127784812 0 0) 0x7f4da80cfcf0 con 0x7f4da80d2390 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d90003520 con 0x7f4da80d2390 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da67fc640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 0x7f4da80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3935435246 0 0) 0x7f4d90003520 con 0x7f4da80d2390 2024-03-14T12:18:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 msgr2=0x7f4da80c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 0x7f4da80c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 >> v1:172.21.15.164:6789/0 conn(0x7f4da80d0e80 legacy=0x7f4da80c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80c7a30 con 0x7f4da80d2390 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f4d98001670 con 0x7f4da80d2390 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 >> v1:172.21.15.82:6789/0 conn(0x7f4da80d2390 legacy=0x7f4da80d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4d90003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d90004800 0x7f4d90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 0x7f4d90005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80d4d30 con 0x7f4d900039e0 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000bb20 con 0x7f4d90004800 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000bd20 con 0x7f4da80d1b90 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4d90003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da67fc640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d90004800 0x7f4d90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da77fe640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 0x7f4d90005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da6ffd640 1 -- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 msgr2=0x7f4d90005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80d1b90 0x7f4d90005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da6ffd640 1 -- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d90004800 msgr2=0x7f4d90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d90004800 0x7f4d90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da6ffd640 1 -- 172.21.15.82:0/1829856326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d90003520 con 0x7f4d900039e0 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.411+0000 7f4da67fc640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d90004800 0x7f4d90004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.412+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4d90003e00 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800c210 tx=0x7f4d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e650720fc0f270 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.412+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98007d30 con 0x7f4d900039e0 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.412+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98030070 con 0x7f4d900039e0 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.412+0000 7f4da5ffb640 1 -- 172.21.15.82:0/1829856326 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9802cd60 con 0x7f4d900039e0 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.413+0000 7f4dad659640 1 -- 172.21.15.82:0/1829856326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 msgr2=0x7f4d90003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.413+0000 7f4dad659640 1 --2- 172.21.15.82:0/1829856326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4d90003e00 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800c210 tx=0x7f4d90005830 comp rx=0 tx=0).stop 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.413+0000 7f4da77fe640 1 -- 172.21.15.82:0/1829856326 reap_dead start 2024-03-14T12:18:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.413+0000 7f4dad659640 1 -- 172.21.15.82:0/1829856326 shutdown_connections 2024-03-14T12:18:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.413+0000 7f4dad659640 1 -- 172.21.15.82:0/1829856326 >> 172.21.15.82:0/1829856326 conn(0x7f4da80d3db0 msgr2=0x7f4da80d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.413+0000 7f4dad659640 1 -- 172.21.15.82:0/1829856326 shutdown_connections 2024-03-14T12:18:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.413+0000 7f4dad659640 1 -- 172.21.15.82:0/1829856326 wait complete. 2024-03-14T12:18:29.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.414+0000 7f4dad659640 1 Processor -- start 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.414+0000 7f4dad659640 1 -- start start 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4dad659640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4da815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4dad659640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80b1030 0x7f4da80b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4dad659640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4da80b14b0 0x7f4da80b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4dad659640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80ca590 con 0x7f4d900039e0 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4dad659640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80c7ff0 con 0x7f4da80b14b0 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4dad659640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80ca710 con 0x7f4da80b1030 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4da815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4da815e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55932/0 (socket says 172.21.15.82:55932) 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 -- 172.21.15.82:0/3151643349 learned_addr learned my addr 172.21.15.82:0/3151643349 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da77fe640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4da80b14b0 0x7f4da80b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da67fc640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80b1030 0x7f4da80b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 -- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80b1030 msgr2=0x7f4da80b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80b1030 0x7f4da80b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 -- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4da80b14b0 msgr2=0x7f4da80b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4da80b14b0 0x7f4da80b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.415+0000 7f4da6ffd640 1 -- 172.21.15.82:0/3151643349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80b2cc0 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.416+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4da815e0e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4d98001970 tx=0x7f4d98001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.416+0000 7f4d8f7fe640 1 -- 172.21.15.82:0/3151643349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004bf0 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.416+0000 7f4d8f7fe640 1 -- 172.21.15.82:0/3151643349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98004d90 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.416+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da80b2f60 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.416+0000 7f4d8f7fe640 1 -- 172.21.15.82:0/3151643349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98007290 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.416+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da80b3520 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.417+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da804f050 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.417+0000 7f4d8f7fe640 1 -- 172.21.15.82:0/3151643349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4d98048020 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.417+0000 7f4d8f7fe640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d9406bbd0 0x7f4d9406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.418+0000 7f4d8f7fe640 1 -- 172.21.15.82:0/3151643349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4d980abb80 con 0x7f4d900039e0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.418+0000 7f4da67fc640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d9406bbd0 0x7f4d9406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.418+0000 7f4da67fc640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d9406bbd0 0x7f4d9406e090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009a60 tx=0x7f4d9c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.422+0000 7f4d8f7fe640 1 -- 172.21.15.82:0/3151643349 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4d9807a300 con 0x7f4d900039e0 2024-03-14T12:18:29.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.596+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f4da80d2170 con 0x7f4d9406bbd0 2024-03-14T12:18:29.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.596+0000 7f4d8f7fe640 1 -- 172.21.15.82:0/3151643349 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4da80d2170 con 0x7f4d9406bbd0 2024-03-14T12:18:29.597 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 6 to deep-scrub 2024-03-14T12:18:29.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d9406bbd0 msgr2=0x7f4d9406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d9406bbd0 0x7f4d9406e090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009a60 tx=0x7f4d9c008040 comp rx=0 tx=0).stop 2024-03-14T12:18:29.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 msgr2=0x7f4da815e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4da815e0e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4d98001970 tx=0x7f4d98001a40 comp rx=0 tx=0).stop 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 shutdown_connections 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4da80b14b0 0x7f4da80b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f4d9406bbd0 0x7f4d9406e090 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4da80b1030 0x7f4da80b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 --2- 172.21.15.82:0/3151643349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d900039e0 0x7f4da815e0e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.598+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 >> 172.21.15.82:0/3151643349 conn(0x7f4da80d3db0 msgr2=0x7f4da8161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.599+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 shutdown_connections 2024-03-14T12:18:29.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.599+0000 7f4dad659640 1 -- 172.21.15.82:0/3151643349 wait complete. 2024-03-14T12:18:29.610 INFO:tasks.ceph:Scrubbing osd.7 2024-03-14T12:18:29.610 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-14T12:18:29.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.713+0000 7f6def47e640 1 Processor -- start 2024-03-14T12:18:29.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.714+0000 7f6def47e640 1 -- start start 2024-03-14T12:18:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.714+0000 7f6def47e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 0x7f6de80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.714+0000 7f6def47e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de80ae4f0 con 0x7f6de806cf60 2024-03-14T12:18:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.714+0000 7f6def47e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de806af60 con 0x7f6de80cfaa0 2024-03-14T12:18:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.714+0000 7f6def47e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de806ad10 con 0x7f6de815dfd0 2024-03-14T12:18:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6deda1b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6de806cf60 0x7f6de80acf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42890/0 (socket says 172.21.15.82:42890) 2024-03-14T12:18:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6deda1b640 1 -- 172.21.15.82:0/3567543567 learned_addr learned my addr 172.21.15.82:0/3567543567 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ded21a640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 0x7f6de80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408587477 0 0) 0x7f6de80ae4f0 con 0x7f6de806cf60 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd4003520 con 0x7f6de806cf60 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806521996 0 0) 0x7f6dd4003520 con 0x7f6de806cf60 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 msgr2=0x7f6de80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 0x7f6de80ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 >> v1:172.21.15.164:6789/0 conn(0x7f6de80cfaa0 legacy=0x7f6de806ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de806b240 con 0x7f6de806cf60 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 169930418 0 0) 0x7f6de00019f0 con 0x7f6de806cf60 2024-03-14T12:18:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 >> v1:172.21.15.82:6789/0 conn(0x7f6de806cf60 legacy=0x7f6de80acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 0x7f6dd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd4004d40 0x7f6dd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de806ad10 con 0x7f6dd40039e0 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd400ba70 con 0x7f6dd4004d40 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd400bc70 con 0x7f6de815dfd0 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ded21a640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6deca19640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 0x7f6dd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6deda1b640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd4004d40 0x7f6dd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ded21a640 1 -- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 msgr2=0x7f6dd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ded21a640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de815dfd0 0x7f6dd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ded21a640 1 -- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd4004d40 msgr2=0x7f6dd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ded21a640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd4004d40 0x7f6dd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.715+0000 7f6ded21a640 1 -- 172.21.15.82:0/3567543567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd4003520 con 0x7f6dd40039e0 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.716+0000 7f6ded21a640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003e00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009a90 tx=0x7f6dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55065468d1077f2a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.716+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4039070 con 0x7f6dd40039e0 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.716+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de402c680 con 0x7f6dd40039e0 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.716+0000 7f6ddffff640 1 -- 172.21.15.82:0/3567543567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4009690 con 0x7f6dd40039e0 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.717+0000 7f6def47e640 1 -- 172.21.15.82:0/3567543567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dd40039e0 msgr2=0x7f6dd4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.717+0000 7f6def47e640 1 --2- 172.21.15.82:0/3567543567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dd40039e0 0x7f6dd4003e00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009a90 tx=0x7f6dd4005830 comp rx=0 tx=0).stop 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.717+0000 7f6deda1b640 1 -- 172.21.15.82:0/3567543567 reap_dead start 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.717+0000 7f6def47e640 1 -- 172.21.15.82:0/3567543567 shutdown_connections 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.717+0000 7f6def47e640 1 -- 172.21.15.82:0/3567543567 >> 172.21.15.82:0/3567543567 conn(0x7f6de80d0110 msgr2=0x7f6de80d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.717+0000 7f6def47e640 1 -- 172.21.15.82:0/3567543567 shutdown_connections 2024-03-14T12:18:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.717+0000 7f6def47e640 1 -- 172.21.15.82:0/3567543567 wait complete. 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.718+0000 7f6def47e640 1 Processor -- start 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.718+0000 7f6def47e640 1 -- start start 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6def47e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd40039e0 0x7f6de8072590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6def47e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de81538a0 0x7f6de809cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6def47e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de815fbb0 0x7f6de809d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6def47e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de806c150 con 0x7f6de815fbb0 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6def47e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80cfec0 con 0x7f6dd40039e0 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6def47e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de815e3b0 con 0x7f6de81538a0 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de815fbb0 0x7f6de809d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de815fbb0 0x7f6de809d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55942/0 (socket says 172.21.15.82:55942) 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 -- 172.21.15.82:0/2359259594 learned_addr learned my addr 172.21.15.82:0/2359259594 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6ded21a640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd40039e0 0x7f6de8072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deca19640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de81538a0 0x7f6de809cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 -- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de81538a0 msgr2=0x7f6de809cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de81538a0 0x7f6de809cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 -- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd40039e0 msgr2=0x7f6de8072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd40039e0 0x7f6de8072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 -- 172.21.15.82:0/2359259594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de8072ad0 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deca19640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6de81538a0 0x7f6de809cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.719+0000 7f6deda1b640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de815fbb0 0x7f6de809d0f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6de8162c70 tx=0x7f6de000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.720+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0007aa0 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.720+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de0018070 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.720+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de80b50a0 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.720+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0013420 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.720+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de80b5660 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.720+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6de804f050 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.721+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6de00135c0 con 0x7f6de815fbb0 2024-03-14T12:18:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.721+0000 7f6dddffb640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6dcc06bc00 0x7f6dcc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.722+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6de008ab10 con 0x7f6de815fbb0 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.722+0000 7f6ded21a640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6dcc06bc00 0x7f6dcc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.722+0000 7f6dddffb640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f6dcc071c00 0x7f6dcc074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.722+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 --> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6de001ee50 con 0x7f6dcc071c00 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.722+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6de008f2c0 con 0x7f6de815fbb0 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.722+0000 7f6ded21a640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6dcc06bc00 0x7f6dcc06e0c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6de4036000 tx=0x7f6de4030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.722+0000 7f6deca19640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f6dcc071c00 0x7f6dcc074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.723+0000 7f6deca19640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f6dcc071c00 0x7f6dcc074020 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.723+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== osd.7 v2:172.21.15.164:6816/369064365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6de001ee50 con 0x7f6dcc071c00 2024-03-14T12:18:29.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.736+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 --> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6de8066f90 con 0x7f6dcc071c00 2024-03-14T12:18:29.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6dddffb640 1 -- 172.21.15.82:0/2359259594 <== osd.7 v2:172.21.15.164:6816/369064365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6de8066f90 con 0x7f6dcc071c00 2024-03-14T12:18:29.737 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-14T12:18:29.737 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T12:18:29.737 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-14T12:18:29.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f6dcc071c00 msgr2=0x7f6dcc074020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.164:6816/369064365,v1:172.21.15.164:6818/369064365] conn(0x7f6dcc071c00 0x7f6dcc074020 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6dcc06bc00 msgr2=0x7f6dcc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f6dcc06bc00 0x7f6dcc06e0c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6de4036000 tx=0x7f6de4030000 comp rx=0 tx=0).stop 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de815fbb0 msgr2=0x7f6de809d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 --2- 172.21.15.82:0/2359259594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de815fbb0 0x7f6de809d0f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6de8162c70 tx=0x7f6de000e530 comp rx=0 tx=0).stop 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6deda1b640 1 -- 172.21.15.82:0/2359259594 reap_dead start 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 shutdown_connections 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 >> 172.21.15.82:0/2359259594 conn(0x7f6de80d0110 msgr2=0x7f6de80acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 shutdown_connections 2024-03-14T12:18:29.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.738+0000 7f6def47e640 1 -- 172.21.15.82:0/2359259594 wait complete. 2024-03-14T12:18:29.748 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-14T12:18:29.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.849+0000 7f40b7be6640 1 Processor -- start 2024-03-14T12:18:29.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.849+0000 7f40b7be6640 1 -- start start 2024-03-14T12:18:29.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b7be6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b015e670 0x7f40b00b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b7be6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f40b0143f60 0x7f40b015b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b7be6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b015ebe0 con 0x7f40b0071740 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b7be6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b015eec0 con 0x7f40b0143f60 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b7be6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b015f1a0 con 0x7f40b015e670 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b5181640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b015e670 0x7f40b00b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b6183640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f40b0143f60 0x7f40b015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b6183640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f40b0143f60 0x7f40b015b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.82:43018/0 (socket says 172.21.15.82:43018) 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b6183640 1 -- 172.21.15.82:0/279042024 learned_addr learned my addr 172.21.15.82:0/279042024 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b5181640 1 -- 172.21.15.82:0/279042024 >> v2:172.21.15.164:3300/0 conn(0x7f40b0143f60 msgr2=0x7f40b015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b5181640 1 --2- 172.21.15.82:0/279042024 >> v2:172.21.15.164:3300/0 conn(0x7f40b0143f60 0x7f40b015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b5181640 1 -- 172.21.15.82:0/279042024 >> v1:172.21.15.164:6789/0 conn(0x7f40b0071740 legacy=0x7f40b0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.850+0000 7f40b5181640 1 -- 172.21.15.82:0/279042024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b015f480 con 0x7f40b015e670 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.851+0000 7f40b5181640 1 --2- 172.21.15.82:0/279042024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b015e670 0x7f40b00b2940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40a00097a0 tx=0x7f40a002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8738a2549172802c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.851+0000 7f40b4980640 1 -- 172.21.15.82:0/279042024 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0035020 con 0x7f40b015e670 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.851+0000 7f40b4980640 1 -- 172.21.15.82:0/279042024 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a0031830 con 0x7f40b015e670 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.851+0000 7f40b4980640 1 -- 172.21.15.82:0/279042024 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0031c00 con 0x7f40b015e670 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 -- 172.21.15.82:0/279042024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b015e670 msgr2=0x7f40b00b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 --2- 172.21.15.82:0/279042024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b015e670 0x7f40b00b2940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40a00097a0 tx=0x7f40a002b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 -- 172.21.15.82:0/279042024 shutdown_connections 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 --2- 172.21.15.82:0/279042024 >> v2:172.21.15.164:3300/0 conn(0x7f40b0143f60 0x7f40b015b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 --2- 172.21.15.82:0/279042024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b015e670 0x7f40b00b2940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 -- 172.21.15.82:0/279042024 >> 172.21.15.82:0/279042024 conn(0x7f40b00b1c70 msgr2=0x7f40b00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 -- 172.21.15.82:0/279042024 shutdown_connections 2024-03-14T12:18:29.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.852+0000 7f40b7be6640 1 -- 172.21.15.82:0/279042024 wait complete. 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.853+0000 7f40b7be6640 1 Processor -- start 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.853+0000 7f40b7be6640 1 -- start start 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b7be6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b0071740 0x7f40b012db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b7be6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b0143f60 0x7f40b0157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b7be6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b015e670 0x7f40b0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b7be6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0144670 con 0x7f40b015e670 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b7be6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b015b080 con 0x7f40b0071740 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b7be6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b01447f0 con 0x7f40b0143f60 2024-03-14T12:18:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b015e670 0x7f40b0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b015e670 0x7f40b0070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55956/0 (socket says 172.21.15.82:55956) 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 -- 172.21.15.82:0/1396032974 learned_addr learned my addr 172.21.15.82:0/1396032974 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b5181640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b0143f60 0x7f40b0157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b5982640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b0071740 0x7f40b012db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 -- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b0143f60 msgr2=0x7f40b0157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b0143f60 0x7f40b0157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 -- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b0071740 msgr2=0x7f40b012db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b0071740 0x7f40b012db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.854+0000 7f40b6183640 1 -- 172.21.15.82:0/1396032974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b013ce70 con 0x7f40b015e670 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.855+0000 7f40b6183640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b015e670 0x7f40b0070630 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009d90 tx=0x7f40a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.855+0000 7f40a67fc640 1 -- 172.21.15.82:0/1396032974 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a800cd20 con 0x7f40b015e670 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.855+0000 7f40a67fc640 1 -- 172.21.15.82:0/1396032974 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8003990 con 0x7f40b015e670 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.855+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b013d150 con 0x7f40b015e670 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.855+0000 7f40a67fc640 1 -- 172.21.15.82:0/1396032974 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003d60 con 0x7f40b015e670 2024-03-14T12:18:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.855+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b013d710 con 0x7f40b015e670 2024-03-14T12:18:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.856+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b004f050 con 0x7f40b015e670 2024-03-14T12:18:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.856+0000 7f40a67fc640 1 -- 172.21.15.82:0/1396032974 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f40a8012050 con 0x7f40b015e670 2024-03-14T12:18:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.857+0000 7f40a67fc640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f409406bbd0 0x7f409406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.857+0000 7f40b5982640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f409406bbd0 0x7f409406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.857+0000 7f40a67fc640 1 -- 172.21.15.82:0/1396032974 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f40a808ab60 con 0x7f40b015e670 2024-03-14T12:18:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.857+0000 7f40b5982640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f409406bbd0 0x7f409406e090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0016e0 tx=0x7f40ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:29.861+0000 7f40a67fc640 1 -- 172.21.15.82:0/1396032974 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f40a8010040 con 0x7f40b015e670 2024-03-14T12:18:30.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.030+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f40b00d07b0 con 0x7f409406bbd0 2024-03-14T12:18:30.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.031+0000 7f40a67fc640 1 -- 172.21.15.82:0/1396032974 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f40b00d07b0 con 0x7f409406bbd0 2024-03-14T12:18:30.031 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 7 to deep-scrub 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f409406bbd0 msgr2=0x7f409406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f409406bbd0 0x7f409406e090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0016e0 tx=0x7f40ac009040 comp rx=0 tx=0).stop 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b015e670 msgr2=0x7f40b0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b015e670 0x7f40b0070630 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009d90 tx=0x7f40a800c490 comp rx=0 tx=0).stop 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 shutdown_connections 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b015e670 0x7f40b0070630 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f409406bbd0 0x7f409406e090 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40b0143f60 0x7f40b0157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 --2- 172.21.15.82:0/1396032974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b0071740 0x7f40b012db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.032+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 >> 172.21.15.82:0/1396032974 conn(0x7f40b00b1c70 msgr2=0x7f40b00b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:30.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.033+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 shutdown_connections 2024-03-14T12:18:30.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.033+0000 7f40b7be6640 1 -- 172.21.15.82:0/1396032974 wait complete. 2024-03-14T12:18:30.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:18:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.144+0000 7ff47359e640 1 Processor -- start 2024-03-14T12:18:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.144+0000 7ff47359e640 1 -- start start 2024-03-14T12:18:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff47359e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff4740d0cf0 0x7ff4740b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff47359e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4740b0790 con 0x7ff4740b51f0 2024-03-14T12:18:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff47359e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4740cf9e0 con 0x7ff474161620 2024-03-14T12:18:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff47359e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4740b3470 con 0x7ff4740d0cf0 2024-03-14T12:18:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff471d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff4740d0cf0 0x7ff4740b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff472d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff4740b51f0 0x7ff4740b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42896/0 (socket says 172.21.15.82:42896) 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff472d9d640 1 -- 172.21.15.82:0/50986766 learned_addr learned my addr 172.21.15.82:0/50986766 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff471d9b640 1 -- 172.21.15.82:0/50986766 >> v1:172.21.15.164:6789/0 conn(0x7ff474161620 legacy=0x7ff4740d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff471d9b640 1 -- 172.21.15.82:0/50986766 >> v1:172.21.15.82:6789/0 conn(0x7ff4740b51f0 legacy=0x7ff4740b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff47159a640 1 -- 172.21.15.82:0/50986766 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908886978 0 0) 0x7ff4740b0790 con 0x7ff4740b51f0 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.145+0000 7ff471d9b640 1 -- 172.21.15.82:0/50986766 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740b36b0 con 0x7ff4740d0cf0 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.146+0000 7ff471d9b640 1 --2- 172.21.15.82:0/50986766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740d0cf0 0x7ff4740b3eb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff4680097a0 tx=0x7ff46802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b2f9b02583e4a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.146+0000 7ff47159a640 1 -- 172.21.15.82:0/50986766 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468035020 con 0x7ff4740d0cf0 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.146+0000 7ff47159a640 1 -- 172.21.15.82:0/50986766 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff468031830 con 0x7ff4740d0cf0 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.146+0000 7ff47159a640 1 -- 172.21.15.82:0/50986766 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468031c00 con 0x7ff4740d0cf0 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.147+0000 7ff47359e640 1 -- 172.21.15.82:0/50986766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740d0cf0 msgr2=0x7ff4740b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.147+0000 7ff47359e640 1 --2- 172.21.15.82:0/50986766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740d0cf0 0x7ff4740b3eb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff4680097a0 tx=0x7ff46802b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.147+0000 7ff47359e640 1 -- 172.21.15.82:0/50986766 shutdown_connections 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.147+0000 7ff47359e640 1 --2- 172.21.15.82:0/50986766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740d0cf0 0x7ff4740b3eb0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.147+0000 7ff47359e640 1 -- 172.21.15.82:0/50986766 >> 172.21.15.82:0/50986766 conn(0x7ff4740d3bd0 msgr2=0x7ff4740d3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.147+0000 7ff47359e640 1 -- 172.21.15.82:0/50986766 shutdown_connections 2024-03-14T12:18:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.147+0000 7ff47359e640 1 -- 172.21.15.82:0/50986766 wait complete. 2024-03-14T12:18:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.148+0000 7ff47359e640 1 Processor -- start 2024-03-14T12:18:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.148+0000 7ff47359e640 1 -- start start 2024-03-14T12:18:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740b51f0 0x7ff474155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4740d0cf0 0x7ff4740c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47359e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff474161620 0x7ff4740b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47359e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740cece0 con 0x7ff4740b51f0 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47359e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740b3a20 con 0x7ff4740d0cf0 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47359e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740ce890 con 0x7ff474161620 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740b51f0 0x7ff474155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740b51f0 0x7ff474155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55972/0 (socket says 172.21.15.82:55972) 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 -- 172.21.15.82:0/2566370834 learned_addr learned my addr 172.21.15.82:0/2566370834 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff471d9b640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4740d0cf0 0x7ff4740c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff472d9d640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff474161620 0x7ff4740b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 -- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff474161620 msgr2=0x7ff4740b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff474161620 0x7ff4740b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 -- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4740d0cf0 msgr2=0x7ff4740c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4740d0cf0 0x7ff4740c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 -- 172.21.15.82:0/2566370834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740cadb0 con 0x7ff4740b51f0 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff471d9b640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4740d0cf0 0x7ff4740c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff47259c640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740b51f0 0x7ff474155a10 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff46400b410 tx=0x7ff46400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.149+0000 7ff462ffd640 1 -- 172.21.15.82:0/2566370834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464015020 con 0x7ff4740b51f0 2024-03-14T12:18:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.150+0000 7ff462ffd640 1 -- 172.21.15.82:0/2566370834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff464011830 con 0x7ff4740b51f0 2024-03-14T12:18:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.150+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4740cb090 con 0x7ff4740b51f0 2024-03-14T12:18:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.150+0000 7ff462ffd640 1 -- 172.21.15.82:0/2566370834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464011c20 con 0x7ff4740b51f0 2024-03-14T12:18:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.150+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4740cb650 con 0x7ff4740b51f0 2024-03-14T12:18:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.151+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff474075210 con 0x7ff4740b51f0 2024-03-14T12:18:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.151+0000 7ff462ffd640 1 -- 172.21.15.82:0/2566370834 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff464018020 con 0x7ff4740b51f0 2024-03-14T12:18:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.151+0000 7ff462ffd640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff45406bc00 0x7ff45406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.152+0000 7ff471d9b640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff45406bc00 0x7ff45406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.152+0000 7ff462ffd640 1 -- 172.21.15.82:0/2566370834 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff46408a1e0 con 0x7ff4740b51f0 2024-03-14T12:18:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.152+0000 7ff471d9b640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff45406bc00 0x7ff45406e0c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff468036000 tx=0x7ff46803c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:30.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.156+0000 7ff462ffd640 1 -- 172.21.15.82:0/2566370834 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff46401f070 con 0x7ff4740b51f0 2024-03-14T12:18:30.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.324+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4740cf5b0 con 0x7ff45406bc00 2024-03-14T12:18:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.325+0000 7ff462ffd640 1 -- 172.21.15.82:0/2566370834 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13352 (secure 0 0 0) 0x7ff4740cf5b0 con 0x7ff45406bc00 2024-03-14T12:18:30.325 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:30.326 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:18:30.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff45406bc00 msgr2=0x7ff45406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff45406bc00 0x7ff45406e0c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff468036000 tx=0x7ff46803c040 comp rx=0 tx=0).stop 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740b51f0 msgr2=0x7ff474155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740b51f0 0x7ff474155a10 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff46400b410 tx=0x7ff46400b8d0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 shutdown_connections 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7ff45406bc00 0x7ff45406e0c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff474161620 0x7ff4740b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4740d0cf0 0x7ff4740c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 --2- 172.21.15.82:0/2566370834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4740b51f0 0x7ff474155a10 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 >> 172.21.15.82:0/2566370834 conn(0x7ff4740d3bd0 msgr2=0x7ff4740d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 shutdown_connections 2024-03-14T12:18:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:30.327+0000 7ff47359e640 1 -- 172.21.15.82:0/2566370834 wait complete. 2024-03-14T12:18:30.338 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-03-14T12:18:29.554837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266228,"kb_used_data":8456,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708492,"statfs":{"total":773094113280,"available":772821495808,"internally_reserved":0,"allocated":8658944,"data_stored":4331224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":263835187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.071029"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":274,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-14T12:18:23.508119+0000","last_change":"2024-03-14T12:13:41.697029+0000","last_active":"2024-03-14T12:18:23.508119+0000","last_peered":"2024-03-14T12:18:23.508119+0000","last_clean":"2024-03-14T12:18:23.508119+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:18:23.508119+0000","last_undegraded":"2024-03-14T12:18:23.508119+0000","last_fullsized":"2024-03-14T12:18:23.508119+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T12:13:40.517100+0000","last_clean_scrub_stamp":"2024-03-14T12:13:40.517100+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:24:54.445548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1527808,"data_stored":984179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36256,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":984179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:18:30.342 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-14T12:13:40.517100+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=12, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=12, tm_min=18, tm_sec=25, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T12:18:30.342 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-14T12:18:50.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T12:18:50.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.456+0000 7f726fae9640 1 Processor -- start 2024-03-14T12:18:50.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.456+0000 7f726fae9640 1 -- start start 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726fae9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f726815e670 0x7f72680b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726fae9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726815ebe0 con 0x7f7268143f60 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726fae9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726815eec0 con 0x7f7268071740 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726fae9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726815f1a0 con 0x7f726815e670 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726d084640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f726815e670 0x7f72680b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726e086640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7268143f60 0x7f726815b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38478/0 (socket says 172.21.15.82:38478) 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726e086640 1 -- 172.21.15.82:0/4291206363 learned_addr learned my addr 172.21.15.82:0/4291206363 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726d084640 1 -- 172.21.15.82:0/4291206363 >> v1:172.21.15.164:6789/0 conn(0x7f7268071740 legacy=0x7f7268071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726d084640 1 -- 172.21.15.82:0/4291206363 >> v1:172.21.15.82:6789/0 conn(0x7f7268143f60 legacy=0x7f726815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.458+0000 7f726d084640 1 -- 172.21.15.82:0/4291206363 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726815f480 con 0x7f726815e670 2024-03-14T12:18:50.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.457+0000 7f726c883640 1 -- 172.21.15.82:0/4291206363 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310724113 0 0) 0x7f726815ebe0 con 0x7f7268143f60 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.458+0000 7f726d084640 1 --2- 172.21.15.82:0/4291206363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f726815e670 0x7f72680b2940 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f725800ba70 tx=0x7f725802e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ca2c3521113481 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.458+0000 7f726c883640 1 -- 172.21.15.82:0/4291206363 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725803b070 con 0x7f726815e670 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.458+0000 7f726c883640 1 -- 172.21.15.82:0/4291206363 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725802ee30 con 0x7f726815e670 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.458+0000 7f726c883640 1 -- 172.21.15.82:0/4291206363 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258036640 con 0x7f726815e670 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.459+0000 7f726fae9640 1 -- 172.21.15.82:0/4291206363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f726815e670 msgr2=0x7f72680b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.459+0000 7f726fae9640 1 --2- 172.21.15.82:0/4291206363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f726815e670 0x7f72680b2940 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f725800ba70 tx=0x7f725802e080 comp rx=0 tx=0).stop 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.459+0000 7f726fae9640 1 -- 172.21.15.82:0/4291206363 shutdown_connections 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.459+0000 7f726fae9640 1 --2- 172.21.15.82:0/4291206363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f726815e670 0x7f72680b2940 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.459+0000 7f726fae9640 1 -- 172.21.15.82:0/4291206363 >> 172.21.15.82:0/4291206363 conn(0x7f72680b1c70 msgr2=0x7f72680cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.459+0000 7f726fae9640 1 -- 172.21.15.82:0/4291206363 shutdown_connections 2024-03-14T12:18:50.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.459+0000 7f726fae9640 1 -- 172.21.15.82:0/4291206363 wait complete. 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.460+0000 7f726fae9640 1 Processor -- start 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.460+0000 7f726fae9640 1 -- start start 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726fae9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7268071740 0x7f726815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726fae9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7268143f60 0x7f72680ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726fae9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f726815e670 0x7f7268158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726fae9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7268144670 con 0x7f7268143f60 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726fae9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726815b080 con 0x7f7268071740 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726fae9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72681447f0 con 0x7f726815e670 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7268143f60 0x7f72680ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7268143f60 0x7f72680ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38866/0 (socket says 172.21.15.82:38866) 2024-03-14T12:18:50.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 -- 172.21.15.82:0/495541426 learned_addr learned my addr 172.21.15.82:0/495541426 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d885640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7268071740 0x7f726815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726e086640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f726815e670 0x7f7268158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 -- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f726815e670 msgr2=0x7f7268158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f726815e670 0x7f7268158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 -- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7268071740 msgr2=0x7f726815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7268071740 0x7f726815a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d084640 1 -- 172.21.15.82:0/495541426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72680cd690 con 0x7f7268143f60 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.461+0000 7f726d885640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7268071740 0x7f726815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.462+0000 7f726d084640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7268143f60 0x7f72680ce730 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f725800ba70 tx=0x7f72580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.462+0000 7f725e7fc640 1 -- 172.21.15.82:0/495541426 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258046070 con 0x7f7268143f60 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.462+0000 7f725e7fc640 1 -- 172.21.15.82:0/495541426 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258042670 con 0x7f7268143f60 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.462+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680cd970 con 0x7f7268143f60 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.462+0000 7f725e7fc640 1 -- 172.21.15.82:0/495541426 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725803b040 con 0x7f7268143f60 2024-03-14T12:18:50.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.462+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72680cdf30 con 0x7f7268143f60 2024-03-14T12:18:50.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.463+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726804f050 con 0x7f7268143f60 2024-03-14T12:18:50.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.463+0000 7f725e7fc640 1 -- 172.21.15.82:0/495541426 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7258049070 con 0x7f7268143f60 2024-03-14T12:18:50.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.464+0000 7f725e7fc640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f724c06bbd0 0x7f724c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.464+0000 7f726d885640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f724c06bbd0 0x7f724c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.464+0000 7f725e7fc640 1 -- 172.21.15.82:0/495541426 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f72580af2c0 con 0x7f7268143f60 2024-03-14T12:18:50.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.464+0000 7f726d885640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f724c06bbd0 0x7f724c06e090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7264007e90 tx=0x7f72640073a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:50.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.468+0000 7f725e7fc640 1 -- 172.21.15.82:0/495541426 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f725807e000 con 0x7f7268143f60 2024-03-14T12:18:50.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.643+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 --> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72680d07b0 con 0x7f724c06bbd0 2024-03-14T12:18:50.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.644+0000 7f725e7fc640 1 -- 172.21.15.82:0/495541426 <== mgr.4114 v2:172.21.15.82:6832/583731528 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13359 (secure 0 0 0) 0x7f72680d07b0 con 0x7f724c06bbd0 2024-03-14T12:18:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:50.644 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f724c06bbd0 msgr2=0x7f724c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f724c06bbd0 0x7f724c06e090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7264007e90 tx=0x7f72640073a0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7268143f60 msgr2=0x7f72680ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7268143f60 0x7f72680ce730 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f725800ba70 tx=0x7f72580046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 shutdown_connections 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f724c06bbd0 0x7f724c06e090 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f726815e670 0x7f7268158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7268143f60 0x7f72680ce730 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 --2- 172.21.15.82:0/495541426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7268071740 0x7f726815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 >> 172.21.15.82:0/495541426 conn(0x7f72680b1c70 msgr2=0x7f72680b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 shutdown_connections 2024-03-14T12:18:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.646+0000 7f726fae9640 1 -- 172.21.15.82:0/495541426 wait complete. 2024-03-14T12:18:50.657 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-03-14T12:18:49.559259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266216,"kb_used_data":8456,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708504,"statfs":{"total":773094113280,"available":772821508096,"internally_reserved":0,"allocated":8658944,"data_stored":4331224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":263835187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002673"},"pg_stats":[{"pgid":"1.0","version":"76'119","reported_seq":285,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-14T12:18:30.292939+0000","last_change":"2024-03-14T12:18:30.292977+0000","last_active":"2024-03-14T12:18:30.292939+0000","last_peered":"2024-03-14T12:18:30.292939+0000","last_clean":"2024-03-14T12:18:30.292939+0000","last_became_active":"2024-03-14T12:13:41.696038+0000","last_became_peered":"2024-03-14T12:13:41.696038+0000","last_unstale":"2024-03-14T12:18:30.292939+0000","last_undegraded":"2024-03-14T12:18:30.292939+0000","last_fullsized":"2024-03-14T12:18:30.292939+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'119","last_scrub_stamp":"2024-03-14T12:18:30.292850+0000","last_deep_scrub":"76'119","last_deep_scrub_stamp":"2024-03-14T12:18:30.292850+0000","last_clean_scrub_stamp":"2024-03-14T12:18:30.292850+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:02:58.835012+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1527808,"data_stored":984179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36256,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":933888,"data_stored":393811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1527808,"data_stored":984179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T12:18:50.658 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-14T12:18:50.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.760+0000 7f8273634640 1 Processor -- start 2024-03-14T12:18:50.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.760+0000 7f8273634640 1 -- start start 2024-03-14T12:18:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.761+0000 7f8273634640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f826c1549d0 0x7f826c06daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.761+0000 7f8273634640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.761+0000 7f8273634640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c0b2f70 con 0x7f826c0cf180 2024-03-14T12:18:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.761+0000 7f8273634640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c154db0 con 0x7f826c1549d0 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.761+0000 7f8273634640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c06d610 con 0x7f826c15e570 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.761+0000 7f82713d0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f826c1549d0 0x7f826c06daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.761+0000 7f8270bcf640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f826c0cf180 0x7f826c15de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:38484/0 (socket says 172.21.15.82:38484) 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f8270bcf640 1 -- 172.21.15.82:0/2638693825 learned_addr learned my addr 172.21.15.82:0/2638693825 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f8271bd1640 1 --2- 172.21.15.82:0/2638693825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f82713d0640 1 -- 172.21.15.82:0/2638693825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 msgr2=0x7f826c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f82713d0640 1 --2- 172.21.15.82:0/2638693825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f82713d0640 1 -- 172.21.15.82:0/2638693825 >> v1:172.21.15.82:6789/0 conn(0x7f826c0cf180 legacy=0x7f826c15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f8263fff640 1 -- 172.21.15.82:0/2638693825 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642050780 0 0) 0x7f826c0b2f70 con 0x7f826c0cf180 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f82713d0640 1 -- 172.21.15.82:0/2638693825 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c0b29b0 con 0x7f826c1549d0 2024-03-14T12:18:50.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.762+0000 7f8271bd1640 1 --2- 172.21.15.82:0/2638693825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:50.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.763+0000 7f82713d0640 1 --2- 172.21.15.82:0/2638693825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c1549d0 0x7f826c06daa0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f825c0097a0 tx=0x7f825c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd47591cb93c68ba server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.763+0000 7f8263fff640 1 -- 172.21.15.82:0/2638693825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c034070 con 0x7f826c1549d0 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.763+0000 7f8263fff640 1 -- 172.21.15.82:0/2638693825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c030610 con 0x7f826c1549d0 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.763+0000 7f8263fff640 1 -- 172.21.15.82:0/2638693825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0309e0 con 0x7f826c1549d0 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 -- 172.21.15.82:0/2638693825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c1549d0 msgr2=0x7f826c06daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 --2- 172.21.15.82:0/2638693825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c1549d0 0x7f826c06daa0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f825c0097a0 tx=0x7f825c02b960 comp rx=0 tx=0).stop 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 -- 172.21.15.82:0/2638693825 shutdown_connections 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 --2- 172.21.15.82:0/2638693825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 --2- 172.21.15.82:0/2638693825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c1549d0 0x7f826c06daa0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 -- 172.21.15.82:0/2638693825 >> 172.21.15.82:0/2638693825 conn(0x7f826c06aef0 msgr2=0x7f826c06b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 -- 172.21.15.82:0/2638693825 shutdown_connections 2024-03-14T12:18:50.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.764+0000 7f8273634640 1 -- 172.21.15.82:0/2638693825 wait complete. 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.765+0000 7f8273634640 1 Processor -- start 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.765+0000 7f8273634640 1 -- start start 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8273634640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c0cf180 0x7f826c1f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8273634640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f826c1549d0 0x7f826c140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8273634640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8273634640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c06d2a0 con 0x7f826c0cf180 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8273634640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c06cfc0 con 0x7f826c1549d0 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8273634640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c15ec30 con 0x7f826c15e570 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c0cf180 0x7f826c1f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c0cf180 0x7f826c1f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38884/0 (socket says 172.21.15.82:38884) 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 -- 172.21.15.82:0/1601820427 learned_addr learned my addr 172.21.15.82:0/1601820427 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8271bd1640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8270bcf640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f826c1549d0 0x7f826c140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 -- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 msgr2=0x7f826c143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 -- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f826c1549d0 msgr2=0x7f826c140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f826c1549d0 0x7f826c140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 -- 172.21.15.82:0/1601820427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c140fb0 con 0x7f826c0cf180 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8270bcf640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f826c1549d0 0x7f826c140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f8271bd1640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.766+0000 7f82713d0640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c0cf180 0x7f826c1f2ea0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f825c000c00 tx=0x7f825c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.767+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c034040 con 0x7f826c0cf180 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.767+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c030410 con 0x7f826c0cf180 2024-03-14T12:18:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.767+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c141360 con 0x7f826c0cf180 2024-03-14T12:18:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.767+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c004250 con 0x7f826c0cf180 2024-03-14T12:18:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.767+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c141920 con 0x7f826c0cf180 2024-03-14T12:18:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.768+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c04f050 con 0x7f826c0cf180 2024-03-14T12:18:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.769+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f825c037070 con 0x7f826c0cf180 2024-03-14T12:18:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.769+0000 7f8261ffb640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f825006bbd0 0x7f825006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.769+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f825c0ac450 con 0x7f826c0cf180 2024-03-14T12:18:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.769+0000 7f8270bcf640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f825006bbd0 0x7f825006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.770+0000 7f8270bcf640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f825006bbd0 0x7f825006e090 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f826c15a260 tx=0x7f8268007c10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.774+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f825c07b2b0 con 0x7f826c0cf180 2024-03-14T12:18:50.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.958+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f826c06cad0 con 0x7f826c0cf180 2024-03-14T12:18:50.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.961+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v18)=0 v18) v1 ==== 123+0+0 (secure 0 0 0) 0x7f825c07abb0 con 0x7f826c0cf180 2024-03-14T12:18:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.962+0000 7f8261ffb640 1 -- 172.21.15.82:0/1601820427 <== mon.0 v2:172.21.15.82:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f825c07ed00 con 0x7f826c0cf180 2024-03-14T12:18:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f825006bbd0 msgr2=0x7f825006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f825006bbd0 0x7f825006e090 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f826c15a260 tx=0x7f8268007c10 comp rx=0 tx=0).stop 2024-03-14T12:18:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c0cf180 msgr2=0x7f826c1f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c0cf180 0x7f826c1f2ea0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f825c000c00 tx=0x7f825c0046c0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 shutdown_connections 2024-03-14T12:18:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7f825006bbd0 0x7f825006e090 secure :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f826c15a260 tx=0x7f8268007c10 comp rx=0 tx=0).stop 2024-03-14T12:18:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f826c15e570 0x7f826c143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f826c1549d0 0x7f826c140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 --2- 172.21.15.82:0/1601820427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c0cf180 0x7f826c1f2ea0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 >> 172.21.15.82:0/1601820427 conn(0x7f826c06aef0 msgr2=0x7f826c0cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 shutdown_connections 2024-03-14T12:18:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:50.964+0000 7f8273634640 1 -- 172.21.15.82:0/1601820427 wait complete. 2024-03-14T12:18:50.975 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T12:18:50.975 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T12:18:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.078+0000 7fae99769640 1 Processor -- start 2024-03-14T12:18:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.078+0000 7fae99769640 1 -- start start 2024-03-14T12:18:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.078+0000 7fae99769640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae94071740 0x7fae94071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.078+0000 7fae99769640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae94143f60 0x7fae9415b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.078+0000 7fae99769640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940b3050 con 0x7fae9415e670 2024-03-14T12:18:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.078+0000 7fae99769640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae94072060 con 0x7fae94071740 2024-03-14T12:18:51.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.078+0000 7fae99769640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9415ebe0 con 0x7fae94143f60 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae92ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae94071740 0x7fae94071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae92ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae94071740 0x7fae94071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38898/0 (socket says 172.21.15.82:38898) 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae92ffd640 1 -- 172.21.15.82:0/3080851001 learned_addr learned my addr 172.21.15.82:0/3080851001 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae937fe640 1 --2- 172.21.15.82:0/3080851001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae94143f60 0x7fae9415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae92ffd640 1 -- 172.21.15.82:0/3080851001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae94143f60 msgr2=0x7fae9415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae92ffd640 1 --2- 172.21.15.82:0/3080851001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae94143f60 0x7fae9415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae92ffd640 1 -- 172.21.15.82:0/3080851001 >> v1:172.21.15.164:6789/0 conn(0x7fae9415e670 legacy=0x7fae940b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.079+0000 7fae92ffd640 1 -- 172.21.15.82:0/3080851001 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9415eec0 con 0x7fae94071740 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.080+0000 7fae92ffd640 1 --2- 172.21.15.82:0/3080851001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94071740 0x7fae94071b20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fae84009870 tx=0x7fae8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509c35a7bc9d9233 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.080+0000 7fae91ffb640 1 -- 172.21.15.82:0/3080851001 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84035020 con 0x7fae94071740 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.080+0000 7fae91ffb640 1 -- 172.21.15.82:0/3080851001 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fae84031640 con 0x7fae94071740 2024-03-14T12:18:51.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.080+0000 7fae91ffb640 1 -- 172.21.15.82:0/3080851001 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8402d040 con 0x7fae94071740 2024-03-14T12:18:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 -- 172.21.15.82:0/3080851001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94071740 msgr2=0x7fae94071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 --2- 172.21.15.82:0/3080851001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94071740 0x7fae94071b20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fae84009870 tx=0x7fae8402b8f0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 -- 172.21.15.82:0/3080851001 shutdown_connections 2024-03-14T12:18:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 --2- 172.21.15.82:0/3080851001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae94143f60 0x7fae9415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 --2- 172.21.15.82:0/3080851001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94071740 0x7fae94071b20 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 -- 172.21.15.82:0/3080851001 >> 172.21.15.82:0/3080851001 conn(0x7fae940b1c70 msgr2=0x7fae940cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 -- 172.21.15.82:0/3080851001 shutdown_connections 2024-03-14T12:18:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.081+0000 7fae99769640 1 -- 172.21.15.82:0/3080851001 wait complete. 2024-03-14T12:18:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.082+0000 7fae99769640 1 Processor -- start 2024-03-14T12:18:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.082+0000 7fae99769640 1 -- start start 2024-03-14T12:18:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae99769640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94071740 0x7fae9415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae99769640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94143f60 0x7fae940708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae99769640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae9415e670 0x7fae9414df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae99769640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94143ce0 con 0x7fae94143f60 2024-03-14T12:18:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae99769640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9415f480 con 0x7fae94071740 2024-03-14T12:18:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae99769640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9415b080 con 0x7fae9415e670 2024-03-14T12:18:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94143f60 0x7fae940708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94143f60 0x7fae940708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38912/0 (socket says 172.21.15.82:38912) 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 -- 172.21.15.82:0/4201032883 learned_addr learned my addr 172.21.15.82:0/4201032883 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae937fe640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae9415e670 0x7fae9414df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae92ffd640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94071740 0x7fae9415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 -- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae9415e670 msgr2=0x7fae9414df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae9415e670 0x7fae9414df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 -- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94071740 msgr2=0x7fae9415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94071740 0x7fae9415ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 -- 172.21.15.82:0/4201032883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9415d1f0 con 0x7fae94143f60 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.083+0000 7fae927fc640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94143f60 0x7fae940708a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fae8800c460 tx=0x7fae8800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.084+0000 7fae7b7fe640 1 -- 172.21.15.82:0/4201032883 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88016020 con 0x7fae94143f60 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.084+0000 7fae7b7fe640 1 -- 172.21.15.82:0/4201032883 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fae88012870 con 0x7fae94143f60 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.084+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9415d4c0 con 0x7fae94143f60 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.084+0000 7fae7b7fe640 1 -- 172.21.15.82:0/4201032883 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88012c80 con 0x7fae94143f60 2024-03-14T12:18:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.084+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9415da80 con 0x7fae94143f60 2024-03-14T12:18:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.085+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9404f050 con 0x7fae94143f60 2024-03-14T12:18:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.085+0000 7fae7b7fe640 1 -- 172.21.15.82:0/4201032883 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fae88019020 con 0x7fae94143f60 2024-03-14T12:18:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.086+0000 7fae7b7fe640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fae8006bbd0 0x7fae8006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T12:18:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.086+0000 7fae7b7fe640 1 -- 172.21.15.82:0/4201032883 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fae8808a330 con 0x7fae94143f60 2024-03-14T12:18:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.086+0000 7fae92ffd640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fae8006bbd0 0x7fae8006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T12:18:51.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.086+0000 7fae92ffd640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fae8006bbd0 0x7fae8006e090 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae84036000 tx=0x7fae84008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T12:18:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.090+0000 7fae7b7fe640 1 -- 172.21.15.82:0/4201032883 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fae88020070 con 0x7fae94143f60 2024-03-14T12:18:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.300+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae940b3240 con 0x7fae94143f60 2024-03-14T12:18:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.301+0000 7fae7b7fe640 1 -- 172.21.15.82:0/4201032883 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fae88012e20 con 0x7fae94143f60 2024-03-14T12:18:51.301 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-14T12:18:51.301 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2847024147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2847024147},{"type":"v1","addr":"172.21.15.164:6835","nonce":2847024147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1725452585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1725452585},{"type":"v1","addr":"172.21.15.164:6834","nonce":1725452585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3678536227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3678536227},{"type":"v1","addr":"172.21.15.82:6835","nonce":3678536227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":7215,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2123357224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2123357224},{"type":"v1","addr":"172.21.15.82:6837","nonce":2123357224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[]} 2024-03-14T12:18:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 42 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fae8006bbd0 msgr2=0x7fae8006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fae8006bbd0 0x7fae8006e090 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae84036000 tx=0x7fae84008030 comp rx=0 tx=0).stop 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94143f60 msgr2=0x7fae940708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94143f60 0x7fae940708a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fae8800c460 tx=0x7fae8800c920 comp rx=0 tx=0).stop 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 shutdown_connections 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:6832/583731528,v1:172.21.15.82:6833/583731528] conn(0x7fae8006bbd0 0x7fae8006e090 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae9415e670 0x7fae9414df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae94143f60 0x7fae940708a0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 --2- 172.21.15.82:0/4201032883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae94071740 0x7fae9415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 >> 172.21.15.82:0/4201032883 conn(0x7fae940b1c70 msgr2=0x7fae940cd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 shutdown_connections 2024-03-14T12:18:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-14T12:18:51.303+0000 7fae99769640 1 -- 172.21.15.82:0/4201032883 wait complete. 2024-03-14T12:18:51.313 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-14T12:18:51.313 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-14T12:18:51.314 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T12:18:51.314 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.322 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T12:18:51.322 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T12:18:51.323 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.376 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T12:18:51.376 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T12:18:51.376 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.388 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T12:18:51.388 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T12:18:51.388 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.440 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T12:18:51.441 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-14T12:18:51.441 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-14T12:18:51.441 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.478 INFO:tasks.ceph.osd.0:Stopped 2024-03-14T12:18:51.478 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-14T12:18:51.478 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.548 INFO:tasks.ceph.osd.1:Stopped 2024-03-14T12:18:51.548 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-14T12:18:51.549 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.618 INFO:tasks.ceph.osd.2:Stopped 2024-03-14T12:18:51.618 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-14T12:18:51.618 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.697 INFO:tasks.ceph.osd.3:Stopped 2024-03-14T12:18:51.697 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-14T12:18:51.697 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.731 INFO:tasks.ceph.osd.4:Stopped 2024-03-14T12:18:51.731 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-14T12:18:51.732 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.807 INFO:tasks.ceph.osd.5:Stopped 2024-03-14T12:18:51.807 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-14T12:18:51.807 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.889 INFO:tasks.ceph.osd.6:Stopped 2024-03-14T12:18:51.889 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-14T12:18:51.890 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.934 INFO:tasks.ceph.osd.7:Stopped 2024-03-14T12:18:51.934 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-14T12:18:51.934 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-14T12:18:51.934 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.960 INFO:tasks.ceph.mgr.y:Stopped 2024-03-14T12:18:51.960 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-14T12:18:51.960 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:51.989 INFO:tasks.ceph.mgr.x:Stopped 2024-03-14T12:18:51.989 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-14T12:18:51.990 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-14T12:18:51.990 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:52.011 INFO:tasks.ceph.mon.a:Stopped 2024-03-14T12:18:52.011 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-14T12:18:52.011 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:52.043 INFO:tasks.ceph.mon.b:Stopped 2024-03-14T12:18:52.043 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-14T12:18:52.043 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T12:18:52.096 INFO:tasks.ceph.mon.c:Stopped 2024-03-14T12:18:52.108 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-14T12:18:52.109 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'missing required features' | head -n 1 2024-03-14T12:18:52.141 INFO:teuthology.orchestra.run.smithi082.stdout:2024-03-14T12:18:05.109706+0000 mds.c (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi082: (6651), after waiting 48.0601 seconds during MDS startup 2024-03-14T12:18:52.142 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-14T12:18:52.142 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'missing required features' | head -n 1 2024-03-14T12:18:52.214 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'missing required features' | head -n 1 2024-03-14T12:18:52.287 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'missing required features' | head -n 1 2024-03-14T12:18:52.360 INFO:teuthology.orchestra.run.smithi082.stdout:2024-03-14T12:18:05.109706+0000 mds.c (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi082: (6651), after waiting 48.0601 seconds during MDS startup 2024-03-14T12:18:52.360 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:18:52.360 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-14T12:18:52.479 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:52.481 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:52.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:18:52.585 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-14T12:18:52.685 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:18:52.686 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-14T12:18:52.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:18:52.778 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-14T12:18:52.871 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:18:52.872 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-14T12:18:53.059 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:18:53.059 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-14T12:18:53.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:18:53.145 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-14T12:18:53.233 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:18:53.234 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-14T12:18:53.283 INFO:tasks.ceph:Archiving mon data... 2024-03-14T12:18:53.284 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/data/mon.a.tgz 2024-03-14T12:18:53.284 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-03-14T12:18:53.300 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.CAJiUiLUHz 2024-03-14T12:18:53.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.CAJiUiLUHz 2024-03-14T12:18:53.466 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.CAJiUiLUHz 2024-03-14T12:18:53.551 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.CAJiUiLUHz is 425KB 2024-03-14T12:18:53.649 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.CAJiUiLUHz 2024-03-14T12:18:53.664 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/data/mon.b.tgz 2024-03-14T12:18:53.664 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-03-14T12:18:53.681 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.KO7zakQoL5 2024-03-14T12:18:53.681 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KO7zakQoL5 2024-03-14T12:18:53.867 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.KO7zakQoL5 2024-03-14T12:18:53.952 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.KO7zakQoL5 is 551KB 2024-03-14T12:18:54.050 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.KO7zakQoL5 2024-03-14T12:18:54.065 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/data/mon.c.tgz 2024-03-14T12:18:54.065 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-03-14T12:18:54.119 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.tJYE7892zv 2024-03-14T12:18:54.120 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tJYE7892zv 2024-03-14T12:18:54.293 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.tJYE7892zv 2024-03-14T12:18:54.377 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.tJYE7892zv is 551KB 2024-03-14T12:18:54.448 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.tJYE7892zv 2024-03-14T12:18:54.463 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-14T12:18:54.463 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-14T12:18:54.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-14T12:18:54.548 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-14T12:18:54.584 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-14T12:18:54.585 INFO:tasks.ceph:Archiving crash dumps... 2024-03-14T12:18:54.586 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/remote/smithi082/crash 2024-03-14T12:18:54.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-14T12:18:54.616 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/remote/smithi164/crash 2024-03-14T12:18:54.617 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-14T12:18:54.650 INFO:tasks.ceph:Shutting down logrotate 2024-03-14T12:18:54.650 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:54.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:54.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:54.771 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-14T12:18:54.798 INFO:tasks.ceph:Compressing logs... 2024-03-14T12:18:54.798 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-14T12:18:54.813 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-14T12:18:54.893 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2024-03-14T12:18:54.893 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.50960.log 2024-03-14T12:18:54.893 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37523.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-14T12:18:54.893 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.1.50960.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.37603.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.50960.log.gz 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37563.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37723.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44539.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37603.log.gz 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37563.log.gz 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37683.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.1.44539.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37723.log.gz 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44539.log.gz 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37683.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37683.log.gz 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37643.log 2024-03-14T12:18:54.894 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.50472.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37643.log.gz 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37483.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.1.50472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.50472.log.gz 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.49995.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37483.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37483.log.gz 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr: /var/log/ceph/ceph-osd.5.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.1.49995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.49995.log.gz 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37763.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37763.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 87.6% 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37763.log.gz 2024-03-14T12:18:54.895 INFO:teuthology.orchestra.run.smithi164.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43552.log 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.1.43552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43552.log.gz 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43942.log 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.b.log: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.3.43942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43942.log.gz 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.d.log: 93.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-14T12:18:54.896 INFO:teuthology.orchestra.run.smithi164.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-14T12:18:54.938 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.c.log: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-14T12:18:55.012 INFO:teuthology.orchestra.run.smithi164.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-14T12:18:55.045 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45249.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56758.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64429.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49812.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45249.log.gz 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60123.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56758.log.gz 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49812.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64429.log.gz 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47583.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.49812.log.gz -5 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose --/var/log/ceph/ceph-client.admin.60123.log: /var/log/ceph/ceph-client.admin.57883.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50106.log 2024-03-14T12:18:55.046 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60123.log.gz 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47583.log.gz 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62122.log 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50106.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57883.log.gz 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61169.log 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50106.log.gz 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55750.log 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52758.log 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62122.log.gz 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63022.log 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61169.log.gz 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52758.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55750.log.gz 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44409.log 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63022.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.52758.log.gz -- 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr: /var/log/ceph/ceph-mon.a.log 2024-03-14T12:18:55.047 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44409.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63022.log.gz 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44409.log.gz 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49896.log 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5/var/log/ceph/ceph-client.admin.43631.log: --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49188.log 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 89.1% 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49896.log.gz 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49188.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49020.log 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49188.log.gz 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4%/var/log/ceph/ceph-client.admin.49020.log: -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2024-03-14T12:18:55.048 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59871.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0%/var/log/ceph/ceph-client.admin.47751.log: -- replaced with /var/log/ceph/ceph-client.admin.49020.log.gz 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45041.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59871.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz /var/log/ceph/ceph-client.admin.63066.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59871.log.gz 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56981.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63066.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45041.log.gz -5 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63538.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56981.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63066.log.gz 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65244.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56981.log.gz 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52217.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60924.log 2024-03-14T12:18:55.049 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63538.log.gz 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52217.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65244.log.gz 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47121.log 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.52217.log.gz -5 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60924.log: /var/log/ceph/ceph-client.admin.47121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59573.log 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60924.log.gz 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45585.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47121.log.gz 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61127.log 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48006.log 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64676.log 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59573.log.gz 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48006.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61127.log.gz 2024-03-14T12:18:55.050 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65748.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48006.log.gz 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58607.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65748.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64676.log.gz 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55440.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr: 87.4%/var/log/ceph/ceph-client.admin.58607.log: -- replaced with /var/log/ceph/ceph-client.admin.65748.log.gz 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63108.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55440.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58607.log.gz 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60483.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63108.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55440.log.gz 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54123.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63108.log.gz 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57799.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.54123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54123.log.gz 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44717.log 2024-03-14T12:18:55.051 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60483.log.gz 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49686.log 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44717.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57799.log.gz 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53576.log 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49686.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44717.log.gz 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55194.log 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53141.log 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49686.log.gz 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53576.log.gz 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55918.log 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55194.log.gz 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53267.log 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53141.log.gz 2024-03-14T12:18:55.052 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55918.log.gz 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52343.log 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:gzip/var/log/ceph/ceph-client.admin.53267.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60291.log 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56081.log 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53267.log.gz 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52343.log.gz 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53786.log 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60291.log.gz 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44713.log 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53786.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.56081.log.gz /var/log/ceph/ceph-client.admin.54401.log 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53786.log.gz 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52469.log 2024-03-14T12:18:55.053 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44713.log.gz 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65622.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64219.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54401.log.gz 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52469.log.gz 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60165.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8%/var/log/ceph/ceph-client.admin.64219.log: -- replaced with /var/log/ceph/ceph-client.admin.65622.log.gz 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61786.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61085.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64219.log.gz 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61786.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60165.log.gz 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53828.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61786.log.gz 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61432.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58255.log 2024-03-14T12:18:55.054 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61085.log.gz 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53828.log.gz 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53448.log 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58255.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61432.log.gzgzip 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53448.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.58255.log.gz --verbose 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.56800.log 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53448.log.gz 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61390.log 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3%/var/log/ceph/ceph-client.admin.56800.log: -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65664.log 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56800.log.gz 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:gzip/var/log/ceph/ceph-client.admin.61390.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65882.log 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49980.log 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61390.log.gz 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65882.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.65664.log.gz 2024-03-14T12:18:55.055 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44157.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49980.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65882.log.gz 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37320.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44157.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.49980.log.gz 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44157.log.gz 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55068.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48553.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37320.log.gz 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63406.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55068.log.gz 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50601.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45333.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52091.log 2024-03-14T12:18:55.056 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50601.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45333.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63406.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45753.log 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45333.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50150.log 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51624.log 89.2% 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52091.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50150.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45753.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47499.log 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51624.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50150.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55834.log 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51624.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65202.log 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65202.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55834.log.gz 2024-03-14T12:18:55.057 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50769.log 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65202.log.gz 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53936.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54317.log 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50769.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62206.log 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54317.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50769.log.gz 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52674.log 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54317.log.gz 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50643.log 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56716.log 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62206.log.gz 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50643.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52674.log.gz 2024-03-14T12:18:55.058 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45669.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50643.log.gz 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61211.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53870.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56716.log.gz 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45669.log.gz 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58297.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61211.log.gz 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63604.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59232.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58297.log.gz 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63604.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53870.log.gz 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44714.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52427.log/var/log/ceph/ceph-client.admin.59232.log: 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58775.log 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% 91.3% -- replaced with /var/log/ceph/ceph-client.admin.59232.log.gz -- replaced with /var/log/ceph/ceph-client.admin.63604.log.gz 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.059 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52427.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44714.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59274.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52427.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61744.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59274.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58775.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61744.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59274.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59004.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49356.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61744.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59004.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58817.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59004.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53353.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65034.log 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49356.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58817.log.gz 2024-03-14T12:18:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62374.log 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65034.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53353.log.gz 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57319.log 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62374.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.65034.log.gz --verbose 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.52049.log 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57319.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62374.log.gz 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48090.log 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57319.log.gz 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50727.log 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48645.log 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52049.log.gz 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50727.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48090.log.gz 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56207.log 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50727.log.gz 2024-03-14T12:18:55.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55960.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58044.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48645.log.gz 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56207.log.gz 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49146.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55960.log.gz 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50287.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62080.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49146.log.gz 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50287.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58044.log.gz 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57193.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45711.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50287.log.gz 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62080.log: /var/log/ceph/ceph-client.admin.57193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64177.log 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62080.log.gz 2024-03-14T12:18:55.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45711.log: gzip -5 --verbose 87.9% -- -- replaced with /var/log/ceph/ceph-client.admin.57193.log.gz /var/log/ceph/ceph-client.admin.51870.log 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64177.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45711.log.gz 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65832.log 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51870.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64177.log.gz 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56470.log 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51870.log.gz 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65706.log 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56470.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.65832.log.gz -5 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59829.log 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56470.log.gz 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57841.log 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65706.log.gz 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62938.log 2024-03-14T12:18:55.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57841.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.59829.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60670.log 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57841.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62938.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31234.log 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62938.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.60670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60670.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.tmp-client.admin.31234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57277.log 0.0% 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.31234.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62602.log 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57277.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55236.log 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62602.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57277.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47709.log 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62602.log.gz 2024-03-14T12:18:55.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47709.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55236.log.gz 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43799.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47709.log.gz 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50373.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58523.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53618.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58523.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50373.log.gz 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53309.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58523.log.gz 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44241.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53618.log: /var/log/ceph/ceph-client.admin.53309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44535.log 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53618.log.gz 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53309.log.gz 2024-03-14T12:18:55.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57235.log 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48132.log 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44535.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44241.log.gz 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64616.log 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44535.log.gz 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61337.log 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57235.log.gz 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48132.log: /var/log/ceph/ceph-client.admin.64616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59449.log 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48132.log.gz 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7%/var/log/ceph/ceph-client.admin.61337.log: -- replaced with /var/log/ceph/ceph-client.admin.64616.log.gz 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52632.log 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59449.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61337.log.gz 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53225.log 2024-03-14T12:18:55.066 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52632.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59449.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43989.log 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52632.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45627.log 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53225.log: /var/log/ceph/ceph-client.admin.43989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52301.log 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53225.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43989.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51786.log 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45627.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59046.log 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51786.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.52301.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62458.log 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6%/var/log/ceph/ceph-client.admin.59046.log: -- replaced with /var/log/ceph/ceph-client.admin.51786.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44493.log 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59046.log.gz 2024-03-14T12:18:55.067 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43465.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62290.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62458.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43465.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44493.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52590.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43465.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52842.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56375.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62290.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52842.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52590.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52842.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37690.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47625.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56375.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62038.log 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-03-14T12:18:55.068 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37690.log.gz 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51828.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62038.log.gz 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54520.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44283.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54520.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65580.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54520.log.gz 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65160.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4%/var/log/ceph/ceph-client.admin.65580.log: -- replaced with /var/log/ceph/ceph-client.admin.44283.log.gz 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65160.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65580.log.gz -5 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.2.46781.log 2024-03-14T12:18:55.069 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50559.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.65160.log.gz 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58110.log 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.2.46781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46781.log.gz 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65118.log 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62248.log 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62248.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65118.log.gz 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 91.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.58110.log.gz --verbose 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.51708.log 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62248.log.gz 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56291.log 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2024-03-14T12:18:55.070 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51708.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44115.log 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56291.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44115.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45795.log 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45417.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44115.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65496.log 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62686.log 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65496.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45795.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48768.log 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65496.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61295.log 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62686.log.gz 2024-03-14T12:18:55.071 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44199.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61295.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48768.log.gz 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65538.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44199.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61295.log.gz 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59088.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44199.log.gz 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8%/var/log/ceph/ceph-client.admin.59088.log: -- replaced with /var/log/ceph/ceph-client.admin.65538.log.gz 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45050.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47457.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59088.log.gz 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63915.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57151.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58381.log 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45050.log.gz 2024-03-14T12:18:55.072 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57151.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63915.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52175.log 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57151.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55624.log 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53532.log 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58381.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55624.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52175.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48689.log 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55624.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59657.log 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48689.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53532.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55398.log 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8%/var/log/ceph/ceph-client.admin.59657.log: -- replaced with /var/log/ceph/ceph-client.admin.48689.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63873.log 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59657.log.gz 2024-03-14T12:18:55.073 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54201.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55398.log.gz 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57427.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63873.log.gz 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54883.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54201.log.gz 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57692.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54883.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57427.log.gz 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48294.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45543.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57692.log.gz 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9%/var/log/ceph/ceph-client.admin.48294.log: -- replaced with /var/log/ceph/ceph-client.admin.54883.log.gz 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48294.log.gz 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62164.log 2024-03-14T12:18:55.074 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.1.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45543.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54817.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47667.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62164.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58565.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60039.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54817.log: /var/log/ceph/ceph-client.admin.58565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61558.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58565.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60039.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54817.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57967.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50517.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60039.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57967.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61558.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57614.log 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50517.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57967.log.gz 2024-03-14T12:18:55.075 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61474.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50517.log.gz 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.57614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57614.log.gz 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45501.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52884.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61474.log.gz 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45501.log.gz 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59615.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52884.log: /var/log/ceph/ceph-client.admin.49314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52259.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52884.log.gz 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43947.log 2024-03-14T12:18:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45040.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52259.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59615.log.gz 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43947.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.45291.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52259.log.gz 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63789.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43947.log.gz 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45291.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.45040.log.gz 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57361.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56888.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63789.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45291.log.gz 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57361.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63789.log.gz 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56249.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56888.log: /var/log/ceph/ceph-client.admin.46911.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57361.log.gz 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56020.log 2024-03-14T12:18:55.077 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56888.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56249.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53183.log 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56249.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56632.log 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56020.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64471.log 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53183.log: /var/log/ceph/ceph-client.admin.56632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64303.log 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53183.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64471.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56632.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61600.log 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64471.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47079.log 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64303.log.gz 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37571.log 2024-03-14T12:18:55.078 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61600.log: /var/log/ceph/ceph-client.admin.47079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59997.log 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61600.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47079.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63295.log 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45051.log 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59997.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37571.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.63295.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59997.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60375.log 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63295.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55482.log 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52800.log 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45051.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60375.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54359.log 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55482.log.gz 2024-03-14T12:18:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56844.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62812.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52800.log.gz 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63670.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56844.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54359.log.gz 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61660.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62812.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56844.log.gz 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51358.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62812.log.gz 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49728.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61660.log: /var/log/ceph/ceph-client.admin.51358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58481.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr: 90.7% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61660.log.gz -- replaced with /var/log/ceph/ceph-client.admin.63670.log.gz 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47373.log 2024-03-14T12:18:55.080 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49728.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51358.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43757.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58481.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49230.log 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47373.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49104.log 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47373.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59913.log 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49104.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49230.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62896.log 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49104.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48596.log 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64908.log 88.2% 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59913.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48596.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62896.log.gz 2024-03-14T12:18:55.081 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44073.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48596.log.gz 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63831.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63957.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44073.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64908.log.gz 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58213.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44073.log.gz 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63957.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64261.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63831.log.gz 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63957.log.gz 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54275.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56550.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58213.log.gz 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61996.log 2024-03-14T12:18:55.082 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64261.log.gz 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56550.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54275.log.gz 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61828.log 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61996.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.56550.log.gz --verbose 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.62416.log 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45047.log 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61996.log.gz 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61828.log.gz 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48510.log 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54949.log 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45047.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62416.log.gz 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45047.log.gz 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54949.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48510.log.gz 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51954.log 2024-03-14T12:18:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61870.log 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54949.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51954.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51912.log 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61870.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51954.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44716.log 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51912.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61870.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51912.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr: /var/log/ceph/ceph-client.admin.56165.log 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48936.log 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56165.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44716.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47289.log 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56165.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64992.log 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47289.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48936.log.gz 2024-03-14T12:18:55.084 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43887.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47289.log.gz 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59787.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59531.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64992.log.gz 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43887.log: /var/log/ceph/ceph-client.admin.59787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49524.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43887.log.gz 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59531.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61253.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59787.log.gz 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59531.log.gz 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44367.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44715.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61253.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49524.log.gz 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60748.log 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61253.log.gz 2024-03-14T12:18:55.085 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44367.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57109.log 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52716.log 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60748.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44715.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51442.log 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60748.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55540.log 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57109.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52716.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46017.log 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60417.log 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55540.log: /var/log/ceph/ceph-client.admin.46017.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55540.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51442.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60417.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46017.log.gz 2024-03-14T12:18:55.086 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53744.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43369.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60417.log.gz 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65286.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59699.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53744.log: /var/log/ceph/ceph-client.admin.65286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55666.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53744.log.gz 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65286.log.gz 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50853.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55666.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59699.log.gz 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51106.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55666.log.gz 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63472.log 2024-03-14T12:18:55.087 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.51106.log: gzip -5 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50853.log.gz 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51106.log.gz 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48336.log 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.63472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50475.log 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63472.log.gz 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48336.log.gz 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52529.log 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50475.log.gz 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50194.log 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52529.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45879.log 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52529.log.gz 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61516.log 2024-03-14T12:18:55.088 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50194.log.gz 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56674.log 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61516.log.gz 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45879.log.gz 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55792.log 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56674.log.gz 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48978.log 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46995.log: /var/log/ceph/ceph-client.admin.55792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64345.log 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55792.log.gz 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50022.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48978.log.gz 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-03-14T12:18:55.089 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64051.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64345.log.gz 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55314.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49062.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64051.log.gz 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50811.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49062.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55314.log.gz 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65454.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50811.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49062.log.gz 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44711.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50811.log.gz 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52133.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44711.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60249.log 2024-03-14T12:18:55.090 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65454.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52133.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44711.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45042.log 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52133.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose/var/log/ceph/ceph-client.admin.60249.log: -- /var/log/ceph/ceph-client.admin.62542.log 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45042.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60249.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55110.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45042.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62542.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59148.log 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55110.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64093.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.091 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59955.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59148.log.gz 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64093.log.gz 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60882.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59955.log.gz 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64135.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60882.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65328.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56123.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60882.log.gz 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64135.log.gz 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: -5/var/log/ceph/ceph-client.admin.65328.log: --verbose -- /var/log/ceph/ceph-client.admin.49938.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56123.log: gzip -5 88.8% --verbose -- /var/log/ceph/ceph-client.admin.59743.log 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65328.log.gz 2024-03-14T12:18:55.092 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56123.log.gz 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47895.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59743.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48174.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49938.log.gz 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59190.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59743.log.gz 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47895.log.gz 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:gzip/var/log/ceph/ceph-client.admin.48174.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58733.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59190.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48174.log.gz 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65370.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59190.log.gz 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50245.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58733.log: /var/log/ceph/ceph-client.admin.65370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61702.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51582.log 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65370.log.gz 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58733.log.gz 2024-03-14T12:18:55.093 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61702.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50245.log.gz 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47037.log 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61702.log.gz 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55356.log 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51582.log: /var/log/ceph/ceph-client.admin.47037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58691.log 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55356.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51582.log.gz 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47037.log.gz 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55356.log.gz/var/log/ceph/ceph-client.admin.58691.log: 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53490.log 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.0.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58691.log.gz 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64950.log 2024-03-14T12:18:55.094 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55582.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53490.log.gz 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62980.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64950.log.gz 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65076.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52979.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62980.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55582.log.gz 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65076.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62980.log.gz 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44706.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47541.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65076.log.gz 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 88.8% 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52979.log.gz 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44706.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50919.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47541.log: /var/log/ceph/ceph-client.admin.51226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60333.log 2024-03-14T12:18:55.095 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50919.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50919.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49440.log 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64387.log 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49440.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60333.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45053.log 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46350.log 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64387.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49440.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45053.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64387.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45053.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.46350.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.0.46350.log.gz 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.0.54707.log 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47247.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-14T12:18:55.096 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.54707.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53702.log 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.0.54707.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47793.log 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53702.log: /var/log/ceph/ceph-client.admin.43588.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53702.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62500.log 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43277.log 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47793.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43277.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62500.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60966.log 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43277.log.gz 2024-03-14T12:18:55.097 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51666.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.60966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62728.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51666.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60966.log.gz 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51666.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61008.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62728.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61008.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62728.log.gz 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61008.log.gz 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57023.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60207.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57023.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57023.log.gz 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52385.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60207.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60207.log.gz 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57067.log 2024-03-14T12:18:55.098 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52385.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52385.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49272.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57067.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57067.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48420.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49272.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49272.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54443.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48420.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48420.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50433.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54443.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54443.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56939.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50433.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50433.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55152.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56939.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56939.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45375.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55152.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55152.log.gz 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58649.log 2024-03-14T12:18:55.099 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45375.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45375.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58919.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58649.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58649.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45052.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58919.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58919.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61954.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45052.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45052.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53660.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61954.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61954.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45459.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53660.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53660.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50685.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45459.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45459.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60840.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50685.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50685.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44031.log 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60840.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60840.log.gz 2024-03-14T12:18:55.100 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50329.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44031.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44031.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58962.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50329.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50329.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60081.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58962.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58962.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47331.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.60081.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60081.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53397.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47331.log: gzip -5 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47331.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58439.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64574.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53397.log: /var/log/ceph/ceph-client.admin.58439.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53397.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58439.log.gz 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58339.log 2024-03-14T12:18:55.101 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64574.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64574.log.gz 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64531.log 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.58339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56333.log 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58339.log.gz 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45837.log 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56333.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64531.log.gz 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56333.log.gz 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65791.log 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65412.log 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45837.log.gz 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.65791.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48852.log 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr: 87.3%/var/log/ceph/ceph-client.admin.65412.log: -- replaced with /var/log/ceph/ceph-client.admin.65791.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43546.log 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65412.log.gz 2024-03-14T12:18:55.102 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48048.log 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48852.log.gz/var/log/ceph/ceph-client.admin.43546.log: 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43546.log.gz 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr: /var/log/ceph/ceph-client.admin.51540.log 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53059.log 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48048.log.gz 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51540.log: gzip -5 --verbose -- 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51540.log.gz 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr: /var/log/ceph/ceph-client.admin.62854.log 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57925.log 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53059.log.gz 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62854.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62854.log.gz 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.58859.log 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49482.log 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9%/var/log/ceph/ceph-client.admin.58859.log: -- replaced with /var/log/ceph/ceph-client.admin.57925.log.gz 2024-03-14T12:18:55.103 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51184.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58859.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55708.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49482.log: /var/log/ceph/ceph-client.admin.51184.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49482.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51184.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46953.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55708.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5/var/log/ceph/ceph-mds.a.log: --verbose -- /var/log/ceph/ceph-client.admin.62770.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62332.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46953.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62770.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64718.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47835.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62332.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.64718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59369.log 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47835.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64718.log.gz 2024-03-14T12:18:55.104 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47835.log.gz 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61912.log 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.59369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55876.log 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59369.log.gz 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.61912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46481.log 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55876.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61912.log.gz 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55876.log.gz 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62644.log 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46481.log.gz 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2024-03-14T12:18:55.105 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.62644.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62644.log.gz 2024-03-14T12:18:55.106 INFO:teuthology.orchestra.run.smithi164.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-14T12:18:55.134 INFO:teuthology.orchestra.run.smithi164.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-14T12:18:55.159 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-14T12:18:55.178 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-14T12:18:55.224 INFO:teuthology.orchestra.run.smithi082.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-14T12:18:55.227 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-14T12:18:55.230 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-14T12:18:55.230 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m0.374s 2024-03-14T12:18:55.230 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m1.576s 2024-03-14T12:18:55.230 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.075s 2024-03-14T12:18:55.282 INFO:teuthology.orchestra.run.smithi082.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-14T12:18:55.297 INFO:teuthology.orchestra.run.smithi082.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-14T12:18:55.333 INFO:teuthology.orchestra.run.smithi082.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-14T12:18:55.494 INFO:teuthology.orchestra.run.smithi082.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-14T12:18:56.087 INFO:teuthology.orchestra.run.smithi082.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-14T12:18:56.089 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-14T12:18:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m1.263s 2024-03-14T12:18:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:user 0m3.368s 2024-03-14T12:18:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m0.362s 2024-03-14T12:18:56.089 INFO:tasks.ceph:Archiving logs... 2024-03-14T12:18:56.089 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/remote/smithi082/log 2024-03-14T12:18:56.090 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-14T12:18:56.955 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/remote/smithi164/log 2024-03-14T12:18:56.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-14T12:18:57.206 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-14T12:18:57.217 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-14T12:18:57.218 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-14T12:18:57.220 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-14T12:18:57.319 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-14T12:18:57.329 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-14T12:18:57.329 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T12:18:57.332 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T12:18:57.345 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-03-14T12:18:57.347 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-03-14T12:18:57.350 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T12:18:57.350 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-03-14T12:18:57.350 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +1619us[+1623us] +/- 130ms 2024-03-14T12:18:57.350 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -1212us[-1208us] +/- 58ms 2024-03-14T12:18:57.350 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +698us[ +702us] +/- 72ms 2024-03-14T12:18:57.350 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:18:57.352 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T12:18:57.352 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-03-14T12:18:57.352 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +1644us[+1644us] +/- 130ms 2024-03-14T12:18:57.352 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -1192us[-1171us] +/- 58ms 2024-03-14T12:18:57.352 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +689us[ +689us] +/- 72ms 2024-03-14T12:18:57.353 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T12:18:57.353 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-14T12:18:57.401 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-14T12:18:57.402 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-14T12:18:57.411 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-14T12:18:57.492 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-14T12:18:57.566 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-14T12:18:57.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:18:57.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-14T12:18:57.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:18:57.630 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-14T12:18:57.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-14T12:18:57.650 INFO:teuthology.task.internal:Duration was 741.651988 seconds 2024-03-14T12:18:57.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-14T12:18:57.660 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-14T12:18:57.660 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-14T12:18:57.663 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-14T12:18:57.705 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T12:18:57.713 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T12:18:58.076 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-14T12:18:58.076 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-03-14T12:18:58.077 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-14T12:18:58.096 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-03-14T12:18:58.096 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-14T12:18:58.120 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-14T12:18:58.120 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T12:18:58.140 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T12:18:58.189 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-14T12:18:58.189 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-14T12:18:58.210 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-14T12:18:58.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-14T12:18:58.341 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-14T12:18:58.342 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-14T12:18:58.368 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-14T12:18:58.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-14T12:18:58.409 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-14T12:18:58.411 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-14T12:18:58.434 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-03-14T12:18:58.460 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-03-14T12:18:58.476 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-14T12:18:58.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:18:58.504 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-14T12:18:58.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T12:18:58.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-14T12:18:58.542 INFO:teuthology.task.internal:Transferring archived files... 2024-03-14T12:18:58.543 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/remote/smithi082 2024-03-14T12:18:58.543 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-14T12:18:58.582 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019/remote/smithi164 2024-03-14T12:18:58.582 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-14T12:18:58.618 INFO:teuthology.task.internal:Removing archive directory... 2024-03-14T12:18:58.618 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-14T12:18:58.621 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-14T12:18:58.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-14T12:18:58.680 INFO:teuthology.task.internal:Not uploading archives. 2024-03-14T12:18:58.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-14T12:18:58.690 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-14T12:18:58.691 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-14T12:18:58.693 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-14T12:18:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 14 12:18 /home/ubuntu/cephtest 2024-03-14T12:18:58.722 INFO:teuthology.orchestra.run.smithi164.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 14 12:18 /home/ubuntu/cephtest 2024-03-14T12:18:58.723 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-14T12:18:58.734 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-14T12:18:58.756 INFO:teuthology.nuke:Checking targets against current locks 2024-03-14T12:18:58.783 DEBUG:teuthology.nuke:shortname: smithi082 2024-03-14T12:18:58.783 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T12:18:58.808 DEBUG:teuthology.nuke:shortname: smithi164 2024-03-14T12:18:58.808 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T12:18:58.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 11:57:05.778361', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtnm40UtVIa2GFsWgd9HtRYh6qFTSDq+2CKW1kvAiwvrH0JAREZrFSK3nki8uLh601My7RmfbPU0NlKowuAyZo='} 2024-03-14T12:18:58.836 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-14_11:45:06-fs:functional-main-distro-default-smithi/7600019', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 11:57:05.776750', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFD15MB9iYeb6MrVg/H9dAD9sptnJFZ1tfewH5VGxfYwEJX0gSLWLl1rzNrxUSikF25Rg8xYpZjDfqYrDMW2F5Y='} 2024-03-14T12:18:58.859 INFO:teuthology.orchestra.console.smithi082:Power off 2024-03-14T12:18:58.859 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T12:18:58.888 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-03-14T12:18:58.888 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:18:58.903 INFO:teuthology.orchestra.console.smithi164:Power off 2024-03-14T12:18:58.903 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T12:18:58.912 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-14T12:18:58.926 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-03-14T12:18:58.926 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:18:58.950 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-03-14T12:19:02.913 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:19:02.951 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:19:03.024 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-14T12:19:03.061 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-03-14T12:19:07.026 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:19:07.062 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:19:07.137 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-14T12:19:07.172 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-03-14T12:19:11.139 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:19:11.173 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T12:19:11.250 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-03-14T12:19:11.282 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-03-14T12:19:11.352 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-03-14T12:19:11.383 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-03-14T12:19:11.488 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin} duration: 741.6519877910614 failure_reason: '"2024-03-14T12:18:05.109706+0000 mds.c (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi082: (6651), after waiting 48.0601 seconds during MDS startup" in cluster log' flavor: default owner: scheduled_jcollin@teuthology success: false 2024-03-14T12:19:11.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T12:19:11.567 INFO:teuthology.run:FAIL